Vehicle track prediction method, prediction model training method and road end equipment

Abstract

The embodiment of the invention discloses a vehicle track prediction method, a prediction model training method and road end equipment, which are used in intersections with traffic lights and can improve the prediction accuracy of vehicle tracks. The prediction method comprises the following steps: obtaining graph data; and according to the graph data, through a vehicle track prediction model, predicting a driving track of the vehicle, wherein the graph data at least comprises a dynamic vector graph, and the dynamic vector graph comprises the position, the state, the controlled lane and the remaining time of the traffic light; wherein the vehicle trajectory prediction model is specifically used for predicting the driving trajectory of the vehicle by extracting the graph features related to the traffic light in the dynamic vector graph.

Description

Technical Field

[0001] The present invention relates to vehicle trajectory prediction technology, and in particular to a vehicle trajectory prediction method, a training method for a prediction model, and a roadside device. Background Art

[0002] With the rapid development of autonomous driving technology, autonomous vehicles are increasingly widely used in various road scenarios. However, in complex intersection (such as, crossroads) scenarios, the problem of vehicle trajectory prediction remains a technical problem to be solved urgently. Most of the existing trajectory prediction methods rely on in-vehicle sensors of intelligent vehicles, such as radar, lidar (LiDAR), cameras, etc., to obtain surrounding environment information and perform trajectory prediction. However, in practical applications, these sensors are restricted by various factors such as their own types, installation angles, heights, etc., resulting in limited vision of autonomous vehicles and being unable to comprehensively and accurately obtain the complete traffic conditions at intersections.

[0003] Specifically, the vision limitation of in-vehicle sensors may cause autonomous vehicles to fail to timely and accurately sense traffic participants such as vehicles, pedestrians, and non-motor vehicles from other directions when approaching intersections, thereby reducing the prediction accuracy of the trajectory prediction algorithm at intersections. This decrease in prediction accuracy not only causes autonomous vehicles to run slowly, reduces the traffic efficiency at intersections, but may also lead to traffic congestion and even increase the risk of traffic accidents.

[0004] In order to overcome the vision limitation of in-vehicle sensors, some researchers have proposed a solution to deploy drones at intersections to obtain intersection trajectory information without occlusion. Drones can overlook the entire intersection from the air, collect traffic data in real time, and provide more comprehensive and accurate information for the trajectory prediction algorithm. However, this solution also faces many challenges in practical applications. For example, drones are restricted by factors such as weather and environment and cannot be deployed 24 hours a day. Under adverse weather conditions, such as strong winds, heavy rains, smogs, etc., the flight safety and data collection quality of drones will be seriously affected.

[0005] In view of the limitations of drones, researchers have begun to explore using vision sensors such as cameras and fisheye cameras to replace or supplement the functions of drones. These vision sensors are easier to be deployed on the infrastructure of intersections, such as traffic light poles, street light poles, etc., and are not restricted by weather conditions. By reasonably arranging multiple cameras and adopting image fusion technology, the global information of intersections can also be obtained. This solution not only has a lower cost, but also is more practical and reliable.

[0006] However, when using vision sensors, etc. for trajectory prediction, due to the complexity of intersections, there is still a problem of low prediction accuracy. Summary of the Invention

[0007] In view of this, the present invention provides a vehicle trajectory prediction method, a training method of a prediction model, and a roadside device, which can improve the accuracy of vehicle trajectory prediction.

[0008] A vehicle trajectory prediction method according to an embodiment of the present invention is used in an intersection with traffic lights, and includes: obtaining graph data; and predicting the driving trajectory of a vehicle through a vehicle trajectory prediction model according to the graph data; wherein, the graph data at least includes: a dynamic vector graph, and the dynamic vector graph includes: the position, state, controlled lanes, and remaining time of the traffic light; wherein, the vehicle trajectory prediction model is specifically configured to predict the driving trajectory of the vehicle by extracting and based on the information related to the traffic light in the dynamic vector graph.

[0009] In some embodiments, the graph data further includes at least one of the following: a global adjacency graph, a static vector graph, and a plurality of local adjacency graphs; the global adjacency graph is used to represent the adjacent relationships of all vehicles in the intersection; the local adjacency graph is used to represent the adjacent relationships of all vehicles in a local area of the intersection; the static vector graph is used to represent the adjacent relationships between the centerlines of lanes in the intersection.

[0010] In some embodiments, the step of predicting the driving trajectory of the vehicle through a vehicle trajectory prediction model according to the graph data includes: using the vehicle trajectory prediction model: extracting the temporal features of each local adjacency graph; extracting the local features of each local adjacency graph according to the temporal features; interacting the local features of each local adjacency graph with the static vector graph, and filling the first local features after interaction into the global adjacency graph; interacting the local features of each local adjacency graph with the dynamic vector graph, and filling the second local features after interaction into the global adjacency graph; extracting features from the global adjacency graph to obtain global features; sharing the global features to each node in the global adjacency graph; and predicting the driving trajectory of the vehicle according to the features of each node in the global adjacency graph.

[0011] In some embodiments, the vehicle trajectory prediction model includes: a first graph attention network, a first Transformer structure, a second Transformer structure, a first graph convolutional network, a second graph attention network, a second graph convolutional network, and a recurrent neural network connected in series in sequence; wherein, the local adjacency graph is used as the input of the first graph attention network, the static vector graph and the dynamic vector graph are used as the input of the second Transformer structure, the global adjacency graph is used as the input of the first graph convolutional network, and the output of the recurrent neural network is the predicted driving trajectory of the vehicle.

[0012] In some embodiments, the method further includes: generating graph data based on the image data of the intersection and the high-precision map data of the intersection.

[0013] A method for training a prediction model of a vehicle trajectory according to an embodiment of the present invention includes: collecting image data of an intersection with traffic lights, where the image data includes information of the traffic lights; annotating the image data according to the high-precision map data of the intersection to generate a data set; preprocessing the data set to generate formatted data; constructing graph data according to the formatted data, where the graph data at least includes a dynamic vector graph, and the dynamic vector graph includes the position, status, controlled lanes, and remaining time of the traffic lights; and training the constructed prediction model of the vehicle trajectory according to the graph data until a prediction model of the vehicle trajectory that meets the accuracy requirement is obtained.

[0014] In some embodiments, the graph data further includes at least one of the following: a global adjacency graph, a static vector graph, and a plurality of local adjacency graphs; the global adjacency graph is used to represent the adjacent relationship of all vehicles in the intersection; the local adjacency graph is used to represent the adjacent relationship of all vehicles in a local area of the intersection; and the static vector graph is used to represent the adjacent relationship between the centerlines of lanes in the intersection.

[0015] In some embodiments, the preprocessing includes: filling the information in the data set into a feature matrix; and removing abnormal and incomplete data in the feature matrix and supplementing missing data in the feature matrix to obtain the formatted data.

[0016] In some embodiments, the step of annotating the image data according to the high-precision map data of the intersection includes: annotating the information of traffic participants in the image data in the high-precision map; associating the lanes controlled by traffic lights with traffic signals; and generating the data set based on the annotation result.

[0017] In some embodiments, the constructed prediction model of the vehicle trajectory includes: a feature extraction network composed of a graph attention network, a graph convolutional network, and a Transformer structure, and an output network for trajectory prediction based on the features extracted by the feature extraction network.

[0018] In some embodiments, the feature extraction network includes: a first graph attention network, a first Transformer structure, a second Transformer structure, a first graph convolutional network, a second graph attention network, and a second graph convolutional network connected in series in sequence, and the output network includes a recurrent neural network.

[0019] In some embodiments, the local adjacency graph serves as the input to the first graph attention network, the static vector graph and the dynamic vector graph serve as the inputs to the second Transformer structure, the global adjacency graph serves as the input to the first graph convolutional network, and the output of the recurrent neural network is the predicted driving trajectory of the vehicle.

[0020] An example of a roadside device according to an embodiment of the present invention is deployed at an intersection with traffic lights and includes: a receiving module configured to receive image data of the intersection collected by a vision sensor; a graph feature construction module configured to construct at least one graph data according to the received image data and high-precision map data of the intersection; a processing module configured to execute the above prediction method to predict the driving trajectory of a vehicle according to the graph data and obtain a prediction result; and a vehicle-road cooperation module configured to send the prediction result to a target vehicle.

[0021] An example of a computer device / apparatus / system according to an embodiment of the present invention includes a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the method according to an embodiment of the present invention.

[0022] An example of a computer-readable storage medium according to an embodiment of the present invention has a computer program / instructions stored thereon, and when the computer program / instructions are executed by a processor, the method according to an embodiment of the present invention is implemented.

[0023] An example of a computer program product according to an embodiment of the present invention includes a computer program / instructions, and when the computer program / instructions are executed by a processor, the method according to an embodiment of the present invention is implemented.

[0024] Advantages of the embodiments of the present invention:

[0025] In the embodiments of the present invention, for an intersection, the feature of the traffic light is considered, and information about the traffic light (such as position, status, controlled lanes, remaining time, etc.) is fused into the vehicle trajectory prediction model in the form of dynamic graph features, thereby improving the prediction accuracy of the vehicle trajectory. Description of the Drawings

[0026] Other details and advantages of the present invention will become apparent from the detailed description provided below. It should be understood that the following drawings are merely illustrative and thus should not be considered as limiting the present invention. The following detailed description will be made with reference to the drawings, where:

[0027] Figure 1 is a schematic diagram of an embodiment of an application scenario of the present invention;

[0028] Figure 2 is a schematic flowchart of an embodiment of the vehicle trajectory prediction method of the present invention;

[0029] Figure 3 is Figure 2 a schematic flowchart of an embodiment of step S22 in

[0030] Figure 4 a schematic structural diagram of an embodiment of the vehicle trajectory prediction model of the present invention;

[0031] Figure 5 a schematic structural diagram of an embodiment of the roadside device of the present invention;

[0032] Figure 6 a schematic flowchart of an embodiment of the training method of the prediction model of the vehicle trajectory of the present invention; and

[0033] Figure 7 a schematic structural diagram of an embodiment of the computer device / equipment / system of the present invention. Detailed Embodiments

[0034] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. Moreover, the terms "first", "second", etc. are applicable to distinguish similar objects and do not have to be used to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0036] In the embodiments of the present invention, for complex scenarios such as intersections, visual sensors, etc. can be used to collect data at intersections, and vehicle trajectory prediction can be performed based on the collected data. The solution of this embodiment can be deployed at the roadside, and the roadside device can perform vehicle trajectory prediction, and then provide the predicted vehicle trajectory to the target vehicle through vehicle-road cooperation and other technologies to support the target vehicle to achieve functions such as autonomous driving.

[0037] Specifically, in the embodiments of the present invention, when performing trajectory prediction, the influence of traffic lights on vehicle trajectories is considered. Traffic lights are one of the biggest differences between intersections and other road scenarios. They control the area outside the stop line at intersections, while vehicles within the area enclosed by the stop line inside the intersection continuously interact with traffic lights. Changes in traffic lights can lead to discontinuities in vehicle movement, such as stopping, starting, accelerating, decelerating, or changing direction. These discontinuities make trajectory prediction more complex and difficult because the movement of vehicles is no longer smooth and continuous, but is subject to discrete state changes controlled by traffic lights. Therefore, in developing a vehicle trajectory prediction scheme for intersections, the influence of traffic lights is fully considered in this embodiment. For example, the trajectory prediction algorithm of this embodiment can accurately identify changes in the state of traffic lights and predict the movement trend and trajectory of vehicles in the next period of time based on this, thereby improving the traffic efficiency and safety of autonomous vehicles at intersections and laying a solid foundation for the wide application of autonomous driving technology.

[0038] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] As Figure 1 shown, it is a schematic diagram of an embodiment of the application scenario of the present invention. As shown in the figure, the solution of this embodiment can be applied to intersections with traffic lights. Of course, other scenarios such as three-way intersections or roundabouts are also applicable to the solution of this embodiment, which will not be limited here. As shown in the figure, traffic lights 101 and visual sensors (such as cameras, etc.) 102 are arranged in all directions of the intersection. The solution of this embodiment can be deployed in the roadside device 103. The roadside device 103 obtains the image data of the intersection from the visual sensor 102 by means of wire or wireless, etc., and then based on the obtained image data, uses the deployed vehicle trajectory prediction model to predict the driving trajectory of vehicles within the intersection. Among them, the predicted driving trajectory can be used for vehicle-road cooperation or traffic management, etc., which is not limited here.

[0040] As Figure 2 shown, it is a schematic flowchart of an embodiment of the method for predicting the vehicle trajectory of the present invention. This method is used in intersections with traffic lights, such as Figure 1 the scenario shown. It includes:

[0041] Step 20: Obtain graph data.

[0042] Step 22: According to the graph data obtained in step S20, predict the driving trajectory of the vehicle through a vehicle trajectory prediction model.

[0043] In step S20, the map data can be generated based on the image data of the intersection and the high-precision map data of the intersection. Among them, the image data of the intersection can be collected in real time by visual sensors in different directions of the intersection, and the high-precision map data of the intersection can be obtained in advance.

[0044] In step S20, the obtained map data includes at least one of the following: global adjacency graph, local adjacency graph, static vector map, and dynamic vector map.

[0045] Among them, the global adjacency graph is used to represent the adjacent relationships of all vehicles in the intersection. In the global adjacency graph, the central vehicle in the scene is used as the root node, and the other vehicles in the scene are used as leaf nodes or intermediate nodes, and the nodes are connected based on the adjacent relationships between the vehicles. Among them, the central vehicle is the vehicle closest to the center of the scene, and the average value of the vehicle position coordinates can be used as the center of the scene. In addition, the size range of the scene can be determined based on the difference between the center coordinates of the scene and the coordinates of the edge vehicles, and the scene can be a circular scene or a non-circular scene, such as a scene with an irregular shape.

[0046] Among them, the local adjacency graph is used to represent the adjacent relationships of all vehicles in a local area of the intersection. Among them, the local area can be an area of interest. For example, several representative vehicles can be selected, such as vehicles with an open space around them, vehicles waiting for a red light at the intersection, etc. Taking them as the center, the area within a certain radius (which can be preset) is set as the area of interest. In addition, the scene can be divided into small areas according to a preset rule (such as, taking 3 to 5 vehicles as a group), and a series of local adjacency graphs are generated based on each small area. Among them, this series of local adjacency graphs can form the nodes of the global adjacency graph. That is to say, the local adjacency graph can be constructed first, and then the central vehicle is used as the root node, and the local adjacency graphs are connected to generate the global adjacency graph, thereby improving the efficiency of graph construction.

[0047] Among them, the static vector map is used to represent the adjacent relationships between the center lines of the lanes in the intersection, that is, the static vector map is also a kind of adjacency graph. Among them, in the static vector map, the root node is the central vehicle, and the intermediate nodes or leaf nodes are the center lines of the lanes in each direction. The relevant information of the lanes, etc., can be obtained from the high-precision map.

[0048] Among them, the dynamic vector map is used to represent the relevant information of traffic lights and controlled lanes in the intersection, and it is also a kind of adjacency graph. Specifically, the dynamic vector map can include information such as the position, status, controlled lanes, and remaining time of the traffic lights. In generating the dynamic vector map, the central vehicle is used as the root node, and based on the positions of the traffic lights in the scene, the traffic lights are used as intermediate nodes or leaf nodes. Among them, the characteristics of each intermediate node or leaf node include: the color of the traffic light, the controlled lanes, and the remaining time.

[0049] In the above-mentioned graph data, taking a vehicle as a traffic participant for example, it can be understood that traffic participants in reality can also include pedestrians, etc., and traffic participants such as pedestrians can also be added to the graph data.

[0050] In step S22, the vehicle trajectory prediction model is specifically used to predict the driving trajectory of the vehicle by extracting features in the graph data. For example, the driving trajectory of the vehicle is predicted by extracting graph features related to traffic lights in the dynamic vector graph. Since information about traffic lights (such as position, status, controlled lanes, remaining time, etc.) is fused into the vehicle trajectory prediction model in the form of dynamic graph features, the prediction accuracy of the vehicle trajectory is improved. The following will be combined with Figure 3 and 4 to specifically illustrate step S22.

[0051] As Figure 3 shown, it is a schematic flowchart of an embodiment of step S22, which includes performing the following operations using the vehicle trajectory prediction model:

[0052] Step S30: Extract the temporal features of each local adjacency graph.

[0053] Step S31: Based on the temporal features extracted in step S30, extract the local features of each local adjacency graph.

[0054] Step S32: Interact the local features of each local adjacency graph with the static vector graph, and fill the first local features after interaction into the global adjacency graph.

[0055] Step S33: Interact the local features of each local adjacency graph with the dynamic vector graph, and fill the second local features after interaction into the global adjacency graph.

[0056] Step S34: Extract features from the global adjacency graph to obtain global features.

[0057] Step S35: Share the global features to each node in the global adjacency graph, so that each node in the global adjacency graph will include: global features, interaction features with the dynamic vector graph, and interaction features with the static vector graph.

[0058] Step S36: Predict the driving trajectory of the vehicle according to the features of each node in the global adjacency graph.

[0059] As Figure 4As shown in the figure, it is a schematic structural diagram of an embodiment of the vehicle trajectory prediction model used in step S22, which includes: a first graph attention network 40, a first Transformer structure 41, a second Transformer structure 42, a first graph convolutional network 43, a second graph attention network 44, a second graph convolutional network 45, and a recurrent neural network 46 connected in series in sequence.

[0060] Among them, the first graph attention network 40 can be used to execute the above-mentioned step S30; the first Transformer structure 41 can be used to execute the above-mentioned step S31; the second Transformer structure 42 and the first graph convolutional network 43 can be used to execute the above-mentioned step S32 and step S33, such as interacting through the Transformer structure and then filling features through the graph convolutional network; the second graph attention network 44 can execute the above-mentioned step S34; the second graph convolutional network 45 can execute the above-mentioned step S35; and the recurrent neural network 46 can execute the above-mentioned step S36.

[0061] In the above, the first graph attention network 40 to the second graph convolutional network 45 are responsible for feature extraction, and the recurrent neural network 46 is used for trajectory prediction based on the extracted features.

[0062] In the above, the graph attention network, the Transformer structure, the graph convolutional network, and the recurrent neural network can adopt general network structures in the field, and their meanings and compositions are clear to those skilled in the art, so they will not be elaborated here.

[0063] In this embodiment, a vehicle trajectory prediction model is built based on the graph attention network, the Transformer structure, and the graph convolutional network, so as to be able to realize the trajectory prediction of multiple target vehicles at the roadside end. Moreover, in this embodiment, the state of the traffic light is embedded into the feature extraction module in the format of dynamic map features, realizing the multi-target roadside vehicle trajectory prediction that fuses traffic light information, and can improve the accuracy of trajectory prediction.

[0064] As Figure 5 shown, it is a schematic structural diagram of an embodiment of the roadside device 103 of the present invention. The roadside device 103 is deployed at an intersection with traffic lights, for example Figure 1 shown, and it can execute Figure 2 and 3 the steps in the related methods of the embodiment to realize the prediction of the vehicle driving trajectory. It includes:

[0065] A receiving module 501, which is used to receive the image data of the intersection collected by the visual sensor.

[0066] Among them, the image data of the intersection can be collected by visual sensors such as cameras (such as fisheye cameras or bullet cameras) installed at the intersection, and then received by the receiving module 501 via wired or wireless means.

[0067] A graph feature construction module 502 is configured to construct at least one graph data according to the high-precision map data of the intersection and the image data received by the receiving module 501.

[0068] Among them, at least one graph data may include at least one of the following: a global adjacency graph, a local adjacency graph, a static vector graph, and a dynamic vector graph. Their meanings are the same as those described in the Figure 2 embodiment and will not be elaborated here.

[0069] In this embodiment, the image data of the intersection can be detected to obtain information such as the position, speed, category, and size of traffic participants (such as vehicles) and the status of traffic lights. Then, the detected results are corresponded in the high-precision map to obtain fusion data, and then the above-mentioned graph data is constructed based on the fusion data.

[0070] A processing module 503 is configured to predict the driving trajectory of a vehicle according to at least one graph data to obtain a prediction result.

[0071] Among them, the processing module 503 can specifically be used to execute the steps in the above Figure 3 embodiment. Alternatively, the processing module 503 can deploy a prediction model for the vehicle trajectory with the structure shown in the above Figure 4 embodiment.

[0072] A vehicle-road cooperation module 504 is configured to send the prediction result to the target vehicle.

[0073] In this embodiment, the driving trajectory of the vehicle in the intersection is predicted at the road side, and then the prediction result is sent to the target vehicle through vehicle-road cooperation technology for the target vehicle to make decisions during autonomous driving, thereby improving the efficiency of autonomous driving. Of course, the solution of this embodiment can be used not only for vehicle-road cooperation but also for applications such as traffic management. For example, the predicted results can be reported to the traffic management platform to facilitate traffic management platforms for traffic flow prediction, intelligent traffic scheduling and management, etc.

[0074] In the above embodiment, a prediction model for the vehicle trajectory is used. This model can extract features from the graph data and predict the driving trajectory of the vehicle based on the extracted features. The training method of this model will be described below with reference to the accompanying drawings.

[0075] As Figure 6 shown, it is a schematic flowchart of an embodiment of the training method of the prediction model for the vehicle trajectory of the present invention, which includes:

[0076] Step S60: Collect image data of an intersection with traffic lights, where the image data includes information about the traffic lights.

[0077] Step S62: Based on the high-precision map data of the intersection, annotate the image data collected in Step S60 to generate a dataset.

[0078] In Step S60, tens of thousands of intersection image data with time lengths ranging from several seconds to dozens of seconds can be collected using sensors such as cameras at the intersection. Then, in Step S62, the information of the intersection image data is annotated in the high-precision map, and finally, the annotated data is cut into one scene data according to a preset duration to generate a dataset.

[0079] Among them, the annotation includes: corresponding the coordinates of traffic participants (such as vehicles, two-wheeled vehicles, and pedestrians, etc.) with the high-precision map, and annotating the information of each traffic participant, including but not limited to: position, speed, acceleration, category, size, and unique number, etc. Through this annotation, the trajectory of each traffic participant can be traced. In addition, the annotation also includes: associating the position of the traffic light with the lane controlled by the traffic light, and annotating the information of the traffic light, including but not limited to: color and remaining time, etc.

[0080] Step S64: Preprocess the dataset to generate formatted data.

[0081] Among them, the preprocessing includes: filling the information in the dataset into the feature matrix; and removing abnormal and incomplete data in the feature matrix, and supplementing the missing data in the feature matrix to obtain unified formatted data.

[0082] Among them, the feature matrix can be expressed as X = [A, T, F]. In the feature matrix X, A represents the number of vehicles (or traffic participants). T is the length of the time stamp, that is, the total number of time stamps. Taking 10-second scene data with a sampling frequency of 0.1 s as an example, T = 100. F is the feature vector with a predetermined length, which is used to encode the relevant information of vehicles and traffic lights, such as the position of the vehicle, the state of the traffic light, etc.

[0083] Among them, the abnormal and incomplete data in the feature matrix can, for example, refer to vehicle data with serious occlusion or insufficient data length. Generally, when a vehicle is occluded for more than 3 frames or the vehicle data is less than 100 frames, it can be considered abnormal and removed. The missing data in the feature matrix can, for example, refer to vehicle data that is briefly occluded, such as a vehicle occluded for 1 - 3 frames. For such a briefly occluded vehicle, dynamic features such as the position and speed of the vehicle can be supplemented by linear interpolation, while fixed features such as classification or size can directly inherit the data of the last frame before occlusion.

[0084] Step S66: Construct graph data based on the formatted data.

[0085] The graph data includes at least one of the following: a global adjacency graph, a static vector graph, a dynamic vector graph, and multiple local adjacency graphs.

[0086] Specifically, in step S66, for each scenario data, the vehicle position at the intermediate moment can be first extracted, the geometric center of the scenario can be calculated through the average value of all vehicle coordinates, and the scenario radius can be calculated based on the vehicle position and the geometric center of the scenario to delimit the scenario range. Specifically, the scenario radius can be determined based on the coordinates of the geometric center and the coordinates of the edge vehicles. In this embodiment, for each scenario data, the geometric center and the scenario radius are calculated with the intermediate moment as the standard, so that the data before this moment can be used as the training set, and the data after this moment can be used as the validation set for model training. For example, taking scenario data with a sampling frequency of 0.1s and a length of 10s as an example, with the 5th second as the segmentation, the trajectory of the vehicle in the latter 50 frames is predicted through the data of the first 50 frames, and the actual trajectory data of the vehicle in the latter 50 frames is used as the reference value to calculate the deviation between the predicted trajectory and the actual trajectory.

[0087] Then, the area within the scenario can be divided into small areas, and a series of local adjacency graphs can be constructed based on the small areas. The local adjacency graphs constructed based on these small areas are connected, and the vehicle closest to the geometric center is used as the root node to construct a global adjacency graph. The high-precision map uses the central vehicle as the root node, and all lane centerlines within the scenario radius are used as intermediate nodes or leaf nodes to construct an adjacency graph, which is called a static vector graph. Using the central vehicle as the root node, an adjacency graph is constructed based on the positions of traffic lights within the scenario, which is called a dynamic vector graph. Among them, the features of each intermediate node or leaf node in the dynamic vector graph include: traffic light color, controlled lane, and remaining time.

[0088] Step S68: Train the constructed vehicle trajectory prediction model according to the graph data constructed in step S66 until a vehicle trajectory prediction model that meets the accuracy requirements is obtained.

[0089] In step S68, the constructed vehicle trajectory prediction model includes: a feature extraction network composed of a graph attention network, a graph convolutional network, and a Transformer structure, and an output network for trajectory prediction based on the features extracted by the feature extraction network.

[0090] Among them, the feature extraction network includes: a first graph attention network, a first Transformer structure, a second Transformer structure, a first graph convolutional network, a second graph attention network, and a second graph convolutional network connected in series in sequence, and the output network includes: a recurrent neural network.

[0091] More specifically, the prediction model of the vehicle trajectory can refer to Figure 4 as shown

[0092] In this embodiment, in the training of the prediction model of the vehicle trajectory, information about traffic lights is introduced, so that the trained model can better predict the driving trajectory of the vehicle.

[0093] In addition, as Figure 7 shown, it is a schematic structural diagram of an embodiment of the computer device / equipment / system 7 of the present invention. The computer device / equipment / system 8 includes a memory 70, a processor 72, and a computer program stored on the memory 70. The processor 72 executes the computer program to implement the methods of the embodiments of the present invention, such as the prediction method and training method of the trajectory, etc.

[0094] In addition, an embodiment of the present invention also discloses a computer-readable storage medium, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the methods of the embodiments of the present invention are implemented, such as the prediction method and training method of the trajectory, etc.

[0095] In addition, an embodiment of the present invention also discloses a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the methods of the embodiments of the present invention are implemented, such as the prediction method and training method of the trajectory, etc.

[0096] The descriptions of the above device, storage medium, and program product embodiments are similar to the descriptions of the above method embodiments, and have beneficial effects similar to those of the method embodiments. For the technical details not disclosed in the embodiments of the device, storage medium, and program product of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0097] The above processor can be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, etc. It can be understood that other electronic devices for implementing the functions of the above processor are also possible, and the embodiments of the present application do not make specific limitations.

[0098] The above computer storage medium / memory can be a read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; it can also be various terminals including one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0099] It should be noted that the above description is only an example and not a limitation of the present invention. In other embodiments of the present invention, the method may have more, fewer, or different steps, and the relationships such as the order, inclusion, and functions between the steps may be different from those described and illustrated. For example, usually multiple steps can be combined into a single step, and a single step can also be split into multiple steps. For those of ordinary skill in the art, without creative efforts, the sequential changes of the steps are also within the protection scope of the present invention.

[0100] The technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.), or a processor, or a microcontroller to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0101] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments.

[0102] Although the present invention has been disclosed above in its preferred embodiments, the present invention is not limited thereto. Any person skilled in the art, without departing from the spirit and scope of the present invention, makes various changes and modifications, which should all be incorporated within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be subject to the scope defined by the claims.

Claims

1. A method for predicting a vehicle trajectory, characterized in that, For an intersection with traffic lights, it includes: Obtain graph data; and According to the graph data, predict the driving trajectory of a vehicle through a vehicle trajectory prediction model; Wherein, the graph data at least includes: a dynamic vector graph, and the dynamic vector graph includes: the position, status, controlled lanes and remaining time of the traffic lights; Wherein, the vehicle trajectory prediction model is specifically used to predict the driving trajectory of a vehicle by extracting graph features related to the traffic lights in the dynamic vector graph.

2. The method for predicting a vehicle trajectory according to claim 1, characterized in that, The graph data further includes at least one of the following: a global adjacency graph, a static vector graph, and multiple local adjacency graphs; The global adjacency graph is used to represent the adjacent relationships of all vehicles in the intersection; The local adjacency graph is used to represent the adjacent relationships of all vehicles in a local area of the intersection; The static vector graph is used to represent the adjacent relationships between the centerlines of the lanes in the intersection.

3. The vehicle trajectory prediction method according to claim 2, wherein The step of predicting the driving trajectory of a vehicle according to the graph data through a vehicle trajectory prediction model includes: Utilize the vehicle trajectory prediction model: Extract the temporal features of each local adjacency graph; According to the temporal features, extract the local features of each local adjacency graph; Interact the local features of each local adjacency graph with the static vector graph, and fill the first local features after interaction into the global adjacency graph; Interact the local features of each local adjacency graph with the dynamic vector graph, and fill the second local features after interaction into the global adjacency graph; Extract features from the global adjacency graph to obtain global features; Share the global features to each node in the global adjacency graph; and Predict the driving trajectory of a vehicle according to the features of each node in the global adjacency graph.

4. The method for predicting a vehicle trajectory according to claim 2, wherein, The vehicle trajectory prediction model includes: A first graph attention network, a first Transformer structure, a second Transformer structure, a first graph convolutional network, a second graph attention network, a second graph convolutional network, and a recurrent neural network connected in series in sequence; Wherein, the local adjacency graph is used as the input of the first graph attention network, the static vector graph and the dynamic vector graph are used as the input of the second Transformer structure, the global adjacency graph is used as the input of the first graph convolutional network, and the output of the recurrent neural network is the predicted driving trajectory of the vehicle.

5. The vehicle trajectory prediction method according to claim 1 or 2, characterized in that The method further includes: generating graph data based on the image data of the intersection and the high-precision map data of the intersection.

6. A training method for a prediction model of vehicle trajectories, characterized in that, It includes: Collect image data of an intersection with traffic lights, and the image data includes: information of the traffic lights; According to the high-precision map data of the intersection, annotate the image data to generate a data set; Preprocess the data set to generate formatted data; According to the formatted data, construct graph data, and the graph data at least includes: a dynamic vector graph, and the dynamic vector graph includes: the position, status, controlled lanes and remaining time of the traffic lights; and According to the graph data, train the built prediction model of the vehicle trajectory until a prediction model of the vehicle trajectory that meets the accuracy requirements is obtained.

7. The training method of the prediction model for vehicle trajectories according to claim 6, characterized in that, The graph data further includes at least one of the following: a global adjacency graph, a static vector graph, and a plurality of local adjacency graphs; The global adjacency graph is used to represent the adjacent relationship of all vehicles in the intersection; The local adjacency graph is used to represent the adjacent relationship of all vehicles in a local area of the intersection; The static vector graph is used to represent the adjacent relationship between the centerlines of lanes in the intersection.

8. The training method of the prediction model of the vehicle trajectory according to claim 6, characterized in that, The preprocessing includes: Filling the information in the dataset into the feature matrix; and Removing abnormal and incomplete data in the feature matrix, and supplementing missing data in the feature matrix to obtain the formatted data.

9. The training method of the prediction model for vehicle trajectories according to claim 6, wherein The step of annotating the image data according to the high-precision map data of the intersection includes: Annotating the information of traffic participants in the image data in the high-precision map; Associating the lanes controlled by traffic lights with traffic signals; and Generating the dataset based on the annotation results.

10. The training method of the prediction model for vehicle trajectories according to claim 6, characterized in that, The built vehicle trajectory prediction model includes: The built vehicle trajectory prediction model includes a feature extraction network composed of a graph attention network, a graph convolutional network, and a Transformer structure, and an output network for trajectory prediction based on the features extracted by the feature extraction network.

11. The training method of the prediction model for vehicle trajectories according to claim 10, characterized in that, The feature extraction network includes a first graph attention network, a first Transformer structure, a second Transformer structure, a first graph convolutional network, a second graph attention network, and a second graph convolutional network connected in series in sequence. The output network includes a recurrent neural network; Wherein, the local adjacency graph is used as the input of the first graph attention network, the static vector graph and the dynamic vector graph are used as the input of the second Transformer structure, the global adjacency graph is used as the input of the first graph convolutional network, and the output of the recurrent neural network is the predicted driving trajectory of the vehicle.

12. A roadside device is deployed at an intersection with traffic lights, characterized in that, It includes: A receiving module, configured to receive the image data of the intersection collected by a vision sensor; A graph feature construction module, configured to construct at least one graph data according to the received image data and the high-precision map data of the intersection; A processing module, configured to execute the method according to any one of claims 1 to 5 to predict the driving trajectory of a vehicle according to the graph data and obtain a prediction result; And A vehicle-road cooperation module, configured to send the prediction result to the target vehicle.

13. A computer device / equipment / system, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having computer programs / instructions stored thereon, characterized in that When the computer program / instructions are executed by the processor, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the method according to any one of claims 1 to 11 is implemented.

Images