Trajectory prediction method and system based on road map

Abstract

The invention relates to a track prediction method and system based on a road map. The method comprises the following steps: firstly, collecting high-precision map data and low-precision map data, respectively forming a data set, then receiving road map data, and converting global spherical coordinates in the road map data into local city coordinates for subsequent track prediction; establishing and training a teacher model according to the high-precision map data set, establishing and training a student initial model according to the low-precision map data set, and performing a knowledge distillation process to obtain a student model after knowledge distillation; and finally, generating trajectory prediction by using the student model after knowledge distillation. Compared with the prior art, the method has the advantages of realizing accurate trajectory prediction without detailed map support, improving the adaptability and generalization ability of trajectory prediction, realizing efficient and low-cost trajectory prediction in a resource-constrained environment, enhancing the universality and accuracy of the model in various geographical environments, and the like.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving perception technology, and in particular to a trajectory prediction method and system based on a road map. Background Art

[0002] To ensure that driverless vehicles can drive safely and efficiently in complex and changing road environments, the driverless vehicle system must be able to detect and accurately interpret the surrounding environment information in real time. This includes not only identifying the location, speed and possible movement of other vehicles, pedestrians and obstacles, but also predicting the future actions and paths of these traffic participants. Such capabilities are the basis for autonomous vehicles to generate safe driving paths, avoid potential collision risks, and provide a comfortable riding experience. Trajectory prediction involves a large amount of data analysis and processing, which requires the help of advanced algorithms and powerful computing power, and is one of the key points of autonomous driving technology research.

[0003] Since trajectory prediction needs to consider road topology and lane geometry information, existing trajectory prediction methods usually have two methods to obtain environmental map information. One is to convert various elements in the traffic scene into a bird's eye view (BEV) picture by rendering, and input it into a convolutional neural network for feature extraction to provide geographic information of the traffic scene. Another method is to vectorize various elements in the traffic scene, convert them into matrix inputs, and perform feature extraction through a recurrent neural network and a multi-layer perceptron to extract relevant information of the scene. Both methods rely on datasets equipped with high-definition maps for development and evaluation.

[0004] Although using high-precision maps can provide centimeter-level lane geometry information, the creation and maintenance of such maps is usually only partially automated. In today's world where road structures are changing rapidly, using such high-precision maps is usually costly. In addition, the locality and update frequency of high-precision maps cannot meet the rapidly evolving needs of autonomous driving.

[0005] Existing literature references:

[0006] "VectorNet:Encoding HD Maps and Agent Dynamics From VectorizedRepresentation";

[0007] "HiVT: Hierarchical Vector Transformer for Multi-Agent MotionPrediction"; "Multimodal motion prediction with stacked transformers";

[0008] "Multiple trajectory prediction with deep temporal and spatialconvolutional neural networks";

[0009] "MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction";

[0010] "Covernet: Multimodal behavior prediction using trajectory sets". Summary of the invention

[0011] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a trajectory prediction method and system based on a road map, aiming to solve the high cost and limited availability problems of trajectory prediction relying on high-definition maps.

[0012] The purpose of the present invention can be achieved by the following technical solutions:

[0013] According to one aspect of the present invention, a trajectory prediction method based on a road map is provided, and the method steps include:

[0014] S1. Collect high-precision map data and low-precision map data to form a high-precision map dataset and a low-precision map dataset respectively;

[0015] S2, receiving initial road map data, and converting global spherical coordinates in the road map data into local city coordinates to obtain processed road map data;

[0016] S3. Establish and train the teacher model based on the high-precision map dataset, and establish and train the student initial model based on the low-precision map dataset;

[0017] S4, perform knowledge distillation on the student initial model to obtain a student model after knowledge distillation;

[0018] S5. Use the student model after knowledge distillation to generate trajectory prediction based on the road map information processed in S2.

[0019] According to the preferred technical solution, both high-precision map data and road map data are extracted from OpenStreetMap.

[0020] According to the preferred technical solution, the global spherical coordinates in S2 use the WGS-84 coordinate system, and the local city coordinates use the Cartesian coordinate system;

[0021] The specific steps for converting global spherical coordinates in road map data into local city coordinates are:

[0022] S21, converting the global spherical coordinates into universal transverse Mercator coordinates;

[0023] S22. Adjust the universal transverse Mercator coordinates to an origin that is consistent with the local city coordinate system.

[0024] According to the preferred technical solution, the S3 middle school student initial model does not use high-precision map data during training to ensure that it is not dependent on high-precision maps in practical applications.

[0025] According to the preferred technical solution, the total loss function is used to train the student initial model during the S4 knowledge distillation process; wherein the total loss function includes a trajectory prediction loss function and a distillation loss function.

[0026] According to the preferred technical solution, the process of training the student initial model using the total loss function is specifically as follows:

[0027] On the one hand, the trajectory prediction loss function is used to minimize the prediction error;

[0028] On the other hand, a distillation loss function is used to minimize the difference between the embedding vectors of the map between the teacher model and the student model.

[0029] According to the preferred technical solution, the trajectory prediction loss function is the original loss function of the trajectory prediction network, ie, the student initial model.

[0030] According to the preferred technical solution, the distillation loss is calculated by the following formula:

[0031]

[0032] L dist is the distillation loss, ξ i and ξ′ i are the i-th element of the embedding vectors of the teacher model and the student model respectively; and It is the embedding vector obtained after the teacher model and the student initial model are processed by the neural network encoder.

[0033] According to the preferred technical solution, the total loss is calculated by the following formula:

[0034] Ltotal=α·Lmodel+βL dist (ξ′,ξ);

[0035] Among them, Ltotal is the total loss, Lmodel is the trajectory prediction loss, α, β are scaling factors used to balance the prediction loss and distillation loss, L dist is the distillation loss, and It is the embedding vector obtained after the teacher model and the student initial model are processed by the neural network encoder.

[0036] According to another aspect of the present invention, a trajectory prediction system based on a road map is provided, the system applies a trajectory prediction method based on a road map as described above, the system comprises a data processing module, a trajectory prediction module, and a knowledge distillation enhancement module;

[0037] The data processing module is used to process input road map data;

[0038] The knowledge distillation enhancement module is used to optimize the performance of the trajectory prediction module;

[0039] The trajectory prediction module is used to generate a trajectory prediction based on the processed road map data.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] 1. A trajectory prediction method based on a road map in the present invention uses a high-precision map dataset and a low-precision map dataset to establish and train a teacher model and a student model respectively, and performs a knowledge distillation process to obtain a student model after knowledge distillation, and finally uses it to generate trajectory prediction. The method applies knowledge distillation to learn and transfer knowledge from high-precision maps to the prediction model of ordinary road maps. This enables the model based on a simplified map to imitate the decision-making process of the high-precision map model, thereby achieving accurate trajectory prediction without detailed map support.

[0042] 2. A trajectory prediction method based on a road map in the present invention uses a high-precision map dataset and a low-precision map dataset to establish and train a teacher model and a student model respectively. The model only relies on information in ordinary road maps without high-precision lane-level details. The model combines the existing neural network architecture with a new algorithm design to adapt to environments with only basic road information, thereby achieving accurate trajectory prediction without detailed map support. Improve adaptability and generalization capabilities. Due to the universal availability of road maps around the world, our method is able to maintain stable prediction performance in different regions and under changing environmental conditions, which is crucial for the widespread deployment and practical application of autonomous driving systems.

[0043] 3. The student model in the trajectory prediction method based on the road map in the present invention uses the original loss function of the trajectory prediction network for training to minimize the prediction error; on the other hand, it uses the distillation loss function for training to minimize the difference in the embedding vector of the map between the teacher model and the student model. This reduces the dependence on high-precision maps and provides a technical solution for achieving efficient and low-cost trajectory prediction in resource-constrained environments, which is expected to promote the widespread application of autonomous driving technology, especially in areas where high-precision map data is difficult to obtain or updated in a timely manner.

[0044] 4. The present invention also includes a coordinate conversion method in a trajectory prediction method based on a road map, which converts global spherical coordinates into local city coordinates, wherein the global spherical coordinates use the WGS-84 coordinate system and the local city coordinates use the Cartesian coordinate system; first convert the global spherical coordinates into universal transverse Mercator coordinates; and then adjust the universal transverse Mercator coordinates to an origin that is consistent with the local city coordinate system. This further enhances the versatility and accuracy of the model in a variety of geographical environments. It can provide trajectory prediction support in a wider range of environments for autonomous driving systems, reduce reliance on high-precision maps that are high in maintenance and cost, and thus improve the feasibility of autonomous driving technology on a global scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the method steps of a trajectory prediction system based on a road map in the present invention;

[0046] Figure 2 A schematic diagram is provided for summarizing the method of using knowledge distillation during training in the present invention;

[0047] Figure 3a The trajectory prediction result using the high-precision map in the Argoverse dataset on road one in the embodiment;

[0048] Figure 3b The trajectory prediction result based on the road map using the method of the present invention in the Argoverse dataset on road one in the embodiment;

[0049] Figure 4a This is the trajectory prediction result using the high-precision map in the Argoverse dataset on Road 2 in the embodiment;

[0050] Figure 4b This is the trajectory prediction result based on the road map using the method of the present invention in the Argoverse dataset on Road 2 in the embodiment. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] Example

[0053] In this embodiment, an innovative method for trajectory prediction using ordinary road maps instead of high-precision maps is applied. The core of this method is to use ordinary road maps, which can only provide road-level topological and geometric information, but are widely available and low-cost worldwide. Compared with high-precision maps, ordinary road maps are easier to update and maintain, and adapt to the rapidly changing urban traffic environment. By using widely available road maps, this method can achieve effective trajectory prediction in the absence of high-precision lane-level detailed information. In addition, this method adopts a knowledge distillation technique to improve the prediction accuracy and system adaptability by extracting key information from a small amount of high-precision map data and distilling it into a road map-based prediction model.

[0054] In this embodiment, a trajectory prediction model based on a road map is developed, which relies only on information in a common road map without high-precision lane-level details. The model combines the existing neural network architecture and knowledge distillation algorithm to adapt to the environment with only basic road information.

[0055] In this embodiment, the application of knowledge distillation technology: a new application method is proposed to learn and transfer knowledge from high-precision maps to the prediction model of ordinary road maps through knowledge distillation. This enables the model based on simplified maps to imitate the decision-making process of the high-precision map model, thereby achieving accurate trajectory prediction without detailed map support.

[0056] In this embodiment, Table 1 compares the performance of the present invention with other methods on the Argoverse validation set. From the prediction accuracy, it can be seen that the prediction accuracy of the road-level map can be close to that of the high-precision map. Specifically, the network trained using the lane-level map is at the same order of magnitude as the results of the high-precision map in terms of minimum average prediction accuracy (minADE), minimum end point prediction accuracy (minFDE), and miss rate (MR).

[0057] Table 1 Performance comparison of the present invention on the Argoverse validation set

[0058]

[0059] In this embodiment, Table 2 compares the trajectory prediction methods based on the navigation map and knowledge distillation strategy proposed in the present invention with other methods without map information. It can be seen that the trajectory prediction network based on the low-cost road map proposed in the present invention can bring stable trajectory prediction advantages compared with those without map information.

[0060] Table 2 Comparison of the method of the present invention with other methods that do not use high-precision maps

[0061]

[0062] In this embodiment, the method of using knowledge distillation during training is as follows Figure 2 As shown: firstly, high-precision map data is collected, road map data is received, and the global spherical coordinates in the road map data are converted into local city coordinates;

[0063] In this embodiment, road map data is obtained from publicly available sources. The preliminary data is provided in the form of global spherical coordinates, and each point contains longitude and latitude information. In order to adapt to the application requirements of the local urban environment, the system first converts these spherical coordinates into Universal Transverse Mercator (UTM) coordinates, and then adjusts these UTM coordinates to align with the origin of the local urban coordinate system to facilitate subsequent data processing and analysis.

[0064] In this embodiment, the road map data comes from OpenStreetMap.

[0065] In this embodiment, the local city coordinate system is based on a Cartesian coordinate grid, where the coordinates are defined relative to a fixed origin specific to each city, which facilitates more accurate navigation, path planning and simulation by the autonomous driving system in an urban environment.

[0066] This embodiment also includes a coordinate conversion method, which converts global spherical coordinates into local city coordinates, further enhancing the versatility and accuracy of the model in a variety of geographical environments. The specific steps of converting global spherical coordinates to a Cartesian coordinate system at a specific point in the city are:

[0067] S21. Convert the WGS-84 coordinate system to the Universal Transvers Mercator (UTM) coordinate system;

[0068] S22. Adjust the UTM coordinates to an origin that is consistent with the local city coordinate system.

[0069] Secondly, a teacher model and a student model are established respectively; the teacher model is trained using high-precision map data, and the student model is trained using the local city coordinate data obtained by the conversion in 1, and combined with knowledge distillation, the student model learns the prediction ability from the teacher model;

[0070] In this embodiment, two neural network models are constructed: a teacher model and a student model. The teacher model uses high-precision map data during the training phase to capture the details of complex environments and generate high-precision trajectory predictions. The student model uses road map data in converted local city coordinates. The prediction ability is learned from the teacher model through the knowledge distillation process. The student model learns the prediction strategy and behavior of the teacher model and can make effective predictions even without direct access to high-precision maps. This process involves calculating the difference in the latent embeddings output by the two models and optimizing the performance of the student model through distillation loss.

[0071] In this embodiment, the use of high-precision maps is restricted during the training of the student model to ensure that high-precision maps are not relied upon in actual applications.

[0072] In this embodiment, knowledge distillation includes using a distillation loss function that aims to minimize the difference in embedding vectors after encoding the input map by the teacher model and the student model.

[0073] The embedding vectors obtained after the teacher model and the student model are processed by the neural network encoder are expressed as and The goal of the loss function is to minimize the distance between the teacher model and the student model in the embedding space. By comparing the difference between these two dt-dimensional vectors, the following distillation loss L can be formed: dist

[0074]

[0075] ξ i and ξ′ i are the i-th element of the teacher and student embedding vectors, respectively. This loss function calculates the average of the squared differences between the two vectors in each dimension.

[0076] In this example, the total loss of the student model during training is calculated by the following formula:

[0077] Ltotal=α·Lmodel+βL dist (ξ′,ξ)

[0078] Among them, Lmodel is the trajectory prediction loss of the student model, α, β are scaling factors used to balance the prediction loss and distillation loss, and Ltotal is the total loss.

[0079] Finally, the student model after knowledge distillation is used to generate trajectory predictions.

[0080] In this embodiment, experimental verification is performed on an actual traffic dataset, and the Argoverse dataset is used to evaluate the performance of the model.

[0081] In this embodiment, the model is deployed and applied in an autonomous driving vehicle. The student model is deployed in the vehicle and receives real-time environmental data and road information collected by on-board sensors and GPS. Using this data, and relying on the knowledge previously learned by the teacher model, the student model is able to generate accurate trajectory predictions and maintain efficient and accurate navigation capabilities even in an environment lacking high-precision maps. This allows the autonomous driving system to operate in a wide range of geographic areas, especially in areas where high-precision maps are incomplete or not updated in a timely manner.

[0082] In this embodiment, Figure 3a-3b and Figure 4a-4b As shown, two sections of roads are selected, namely Road 1 and Road 2, and the method in this embodiment is compared with the traditional trajectory prediction method relying on high-precision maps on Road 1 and Road 2 respectively. Figure 3a and Figure 4a The image shown is the result of using a high-precision map. Figure 3b and Figure 4b The images shown are the results of using this method with a simple low-precision map. It is not difficult to see that the network trained using the road map can also perform stable multimodal trajectory prediction output in traffic environments such as normal lanes and intersections based on rough maps. This result shows that even without high-precision lane information, road maps can support effective trajectory prediction, demonstrating the effectiveness and advantages of this method.

[0083] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A trajectory prediction method based on a road map, characterized in that: The method steps include: S1. Collect high-precision map data and low-precision map data to form a high-precision map dataset and a low-precision map dataset respectively; S2, receiving initial road map data, and converting global spherical coordinates in the road map data into local city coordinates to obtain processed road map data; S3. Establish and train the teacher model based on the high-precision map dataset, and establish and train the student initial model based on the low-precision map dataset; S4, perform knowledge distillation on the student initial model to obtain a student model after knowledge distillation; S5. Use the student model after knowledge distillation to generate trajectory prediction based on the road map information processed in S2.

2. A trajectory prediction method based on a road map according to claim 1, characterized in that: The high-precision map data and road map data are both extracted from OpenStreetMap.

3. The method for trajectory prediction based on a road map according to claim 1, characterized in that: The global spherical coordinates in S2 use the WGS-84 coordinate system, and the local city coordinates use the Cartesian coordinate system; The specific steps for converting global spherical coordinates in road map data into local city coordinates are: S21, converting the global spherical coordinates into universal transverse Mercator coordinates; S22. Adjust the universal transverse Mercator coordinates to an origin that is consistent with the local city coordinate system.

4. The method for trajectory prediction based on a road map according to claim 1, characterized in that: The S3 student initial model does not use high-precision map data during training to ensure that it is not dependent on high-precision maps in actual applications.

5. The method for trajectory prediction based on a road map according to claim 1, characterized in that: In the S4 knowledge distillation process, the student initial model is trained using a total loss function; wherein the total loss function includes a trajectory prediction loss function and a distillation loss function.

6. A trajectory prediction method based on a road map according to claim 5, characterized in that: The process of training the student initial model using the total loss function is specifically as follows: On the one hand, the trajectory prediction loss function is used to minimize the prediction error; On the other hand, a distillation loss function is used to minimize the difference between the embedding vectors of the map between the teacher model and the student model.

7. A method for trajectory prediction based on a road map according to claim 6, characterized in that: The trajectory prediction loss function is the original loss function of the trajectory prediction network, i.e., the student initial model.

8. The method for trajectory prediction based on a road map according to claim 6, characterized in that: The distillation loss is calculated by the following formula: L dist is the distillation loss, ξ i and i ′ are the i-th element of the embedding vectors of the teacher model and the student model respectively; and It is the embedding vector obtained after the teacher model and the student initial model are processed by the neural network encoder.

9. A trajectory prediction method based on a road map according to claim 8, characterized in that: The total loss is calculated by the following formula: Ltotal=α·Lmodel+βL dist (ξ′,ξ); Among them, Ltotal is the total loss, Lmodel is the trajectory prediction loss, α, β are scaling factors used to balance the prediction loss and distillation loss, L dist is the distillation loss, and It is the embedding vector obtained after the teacher model and the student initial model are processed by the neural network encoder.

10. A trajectory prediction system based on a road map, characterized in that: The system applies a trajectory prediction method based on a road map as described in any one of claims 1 to 9, and the system includes a data processing module, a trajectory prediction module, and a knowledge distillation enhancement module; The data processing module is used to process input road map data; The knowledge distillation enhancement module is used to optimize the performance of the trajectory prediction module; The trajectory prediction module is used to generate a trajectory prediction based on the processed road map data.

Images