A vehicle trajectory prediction method, electronic equipment and storage medium

Abstract

The application discloses a vehicle trajectory prediction method, an electronic device and a storage medium. The method comprises the following steps: acquiring first vehicle features of a target vehicle and surrounding vehicles of the target vehicle, and acquiring first lane segment features of lane segments of a target candidate lane; interacting the first vehicle features of any surrounding vehicle with those of other surrounding vehicles to obtain second vehicle features of the corresponding surrounding vehicle; and interacting the first lane segment features of any lane segment of the target candidate lane with those of other lane segments to obtain second lane segment features of the corresponding lane segment; fusing the second lane segment features of the lane segments on the same target candidate lane to obtain first lane features of the target candidate lane; interacting the second vehicle features, the first lane features and the first vehicle features of the target vehicle to obtain fusion features; and predicting based on the fusion features to obtain a predicted trajectory of the target vehicle. In this way, the accuracy of vehicle trajectory prediction can be improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a vehicle trajectory prediction method, electronic device, and storage medium. Background Technology

[0002] During operation, intelligent driving vehicles use multiple sensors to perceive surrounding environmental information, including road conditions and nearby road users. These vehicles need to predict the movement trajectories of nearby road users within a certain timeframe, then generate their own trajectory planning scheme to achieve automatic self-control. This helps them avoid collisions with nearby road users and ensures safety, stability, and comfort during driving.

[0003] However, the accuracy of vehicle trajectory prediction in existing technologies is poor, which makes it impossible to achieve precise vehicle control and causes collisions between vehicles or between vehicles and objects. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a vehicle trajectory prediction method, electronic device, and storage medium that can improve the accuracy of vehicle trajectory prediction.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a vehicle trajectory prediction method. The method includes: acquiring first vehicle features of a target vehicle and surrounding vehicles, and acquiring first lane segment features of a lane segment of a target candidate lane; wherein the target candidate lane is a lane that the target vehicle can reach by going straight, turning, changing lanes, or making a U-turn; interacting the first vehicle features of any surrounding vehicle with those of other surrounding vehicles to obtain second vehicle features of the corresponding surrounding vehicle; interacting the first lane segment features of any lane segment of the target candidate lane with those of other lane segments to obtain second lane segment features of the corresponding lane segment; fusing the second lane segment features of lane segments on the same target candidate lane to obtain first lane features of the target candidate lane; interacting the second vehicle features of surrounding vehicles, the first lane features of the target candidate lane, and the first vehicle features of the target vehicle to obtain fused features; and making a prediction based on the fused features to obtain the predicted trajectory of the target vehicle.

[0006] All interactions are based on multi-head attention.

[0007] The process of interacting with the first vehicle features of any surrounding vehicle to obtain the second vehicle features of the corresponding surrounding vehicle includes: selecting one surrounding vehicle as the first surrounding vehicle and designating the other surrounding vehicles as the second surrounding vehicles; performing multi-head interactive attention processing using the first vehicle features of the second surrounding vehicles as key elements and value elements, and the first vehicle features of the first surrounding vehicles as query elements, to obtain the third vehicle features of the first surrounding vehicles; and fusing the third vehicle features of the first surrounding vehicles with the first vehicle features to obtain the second vehicle features of the first surrounding vehicles.

[0008] The process of interacting with the first lane segment features of any lane segment of a target candidate lane to obtain the second lane segment feature of the corresponding lane segment includes: dividing each target candidate lane into a first preset number of lane segments to obtain a lane segment set; taking any lane segment in the lane segment set as the first lane segment and taking the lane segments other than the first lane segment as the second lane segment; performing multi-head interactive attention processing with the first lane segment feature of the second lane segment as the key element and value element, and the first lane segment feature of the first lane segment as the query element, to obtain the third lane segment feature of the first lane segment; and fusing the first lane segment feature and the third lane segment feature of the first lane segment to obtain the second lane segment feature of the corresponding first lane segment.

[0009] Before interacting with the first lane segment features of any lane segment of the target candidate lane with other lane segments, the method further includes: obtaining a segment set; wherein the segment set includes multiple first preset quantities;

[0010] The process involves interacting the first lane segment features of any lane segment of a target candidate lane with the first lane segment features of other lane segments to obtain the second lane segment features of the corresponding lane segment. This includes: for each first preset number of segments in the segment set, performing the following steps: dividing each target candidate lane into a first preset number of lane segments to obtain a lane segment set; taking any lane segment in the lane segment set as the first lane segment, and taking the lane segments other than the first lane segment as the second lane segment; performing multi-head interactive attention processing using the first lane segment features of the second lane segment as the key element and value element, and the first lane segment features of the first lane segment as the query element, respectively, to obtain the third lane segment features of the corresponding first lane segment; fusing the first lane segment features and the third lane segment features of the first lane segment to obtain multiple sets of second lane segment features corresponding to the first lane segment; wherein each set of second lane segment features corresponding to the first lane segment is obtained by dividing each target candidate lane into the same first preset number of lane segments.

[0011] The second lane segment features of lane segments on the same target candidate lane are fused to obtain the first lane feature of the target candidate lane. This includes: splicing the second lane segment features of lane segments on the same target candidate lane in each group of second lane segment features to obtain the second lane feature of the target candidate lane; and adding the second lane features of the same target candidate lane in multiple groups of second lane segment features to obtain the first lane feature of the target candidate lane.

[0012] Specifically, the fusion features are obtained by interacting with the second vehicle features of surrounding vehicles, the first lane features of the target candidate lane, and the first vehicle features of the target vehicle. This includes: performing multi-head interactive attention processing on the first vehicle features of the target vehicle and the first lane features of the target candidate lane to obtain the second vehicle features of the target vehicle; and performing multi-head interactive attention processing on the second vehicle features of the target vehicle and the second vehicle features of surrounding vehicles to obtain the fusion features.

[0013] Specifically, multi-head interactive attention processing is performed using the first vehicle feature of the target vehicle and the first lane feature of the target candidate lane to obtain the second vehicle feature of the target vehicle. This includes: using the first vehicle feature of the target vehicle as the query element and the first lane feature of the target candidate lane as the key element and value element, multi-head interactive attention processing is performed to obtain the second vehicle feature of the target vehicle.

[0014] And / or, using the second vehicle features of the target vehicle and the second vehicle features of surrounding vehicles to perform multi-head interactive attention processing to obtain fused features, including: using the second vehicle features of the target vehicle as query elements and the second vehicle features of surrounding vehicles as key elements and value elements, performing multi-head interactive attention processing to obtain fused features.

[0015] Before acquiring the first vehicle features of the target vehicle and surrounding vehicles, and acquiring the first lane segment features of the lane segment of the target candidate lane, the vehicle trajectory prediction method further includes: acquiring the historical trajectories of the target vehicle and surrounding vehicles, and acquiring the lane point set of the target candidate lane; wherein each lane point in the lane point set is located on the centerline of the target candidate lane.

[0016] Obtain the first vehicle features of the target vehicle and the surrounding vehicles, including: extracting features based on the historical trajectories of the target vehicle and the surrounding vehicles to obtain the first vehicle features of the target vehicle and the surrounding vehicles.

[0017] Obtaining the first lane segment feature of the lane segment of the target candidate lane includes: extracting features based on each lane point in the lane point set to obtain the first lane segment feature of the lane segment of the target candidate lane. Specifically, extracting features based on each lane point in the lane point set to obtain the first lane segment feature of the lane segment of the target candidate lane includes: classifying each lane point in the lane point set to obtain the lane points belonging to each lane segment; and extracting features based on the lane points of each lane segment to obtain the first lane segment feature of each lane segment.

[0018] Before obtaining the set of lane points for the target candidate lane, the vehicle trajectory prediction method also includes: selecting the lane point closest to the target vehicle as the starting lane point; using a depth-first traversal algorithm to traverse the lane points adjacent to the starting lane point to determine the initial candidate lane; and determining the target candidate lane based on the initial target candidate lane.

[0019] The process of determining a target candidate lane based on an initial target candidate lane includes: determining whether the number of lane points contained in the initial candidate lane reaches a second preset number; in response to the number of lane points contained in the initial candidate lane reaching the second preset number, using the initial candidate lane as a target candidate lane; in response to the number of lane points contained in the initial candidate lane not reaching the second preset number, supplementing the lane points of the initial candidate lane with zeros to obtain the target candidate lane; the number of lane points contained in the target candidate lane is equal to the second preset number.

[0020] To address the aforementioned technical problems, a second aspect of this application provides an electronic device comprising a memory and a processor coupled to each other; the memory stores program instructions; and the processor executes the program instructions stored in the memory to implement the method provided in the first aspect.

[0021] To address the aforementioned technical problems, a third aspect of this application provides a computer-readable storage medium for storing program instructions that can be executed by a processor to implement the method provided in the first aspect.

[0022] The beneficial effects of this application are as follows: Unlike existing technologies, after obtaining the first vehicle features of the target vehicle and its surrounding vehicles, as well as the first lane segment features of the lane segment of the target candidate lane, this application firstly interacts the first vehicle features of any surrounding vehicle with those of other surrounding vehicles to obtain the second vehicle features corresponding to that vehicle, ensuring that the second vehicle features of that vehicle include the features of other surrounding vehicles. Then, it interacts the first lane segment features of any lane segment of the target candidate lane with those of other lane segments to obtain the second lane segment features corresponding to that lane segment, ensuring that the second lane segment features of that lane segment include the features of other lane segments. Secondly, it fuses the second lane segment features of lane segments on the same target candidate lane to obtain the first lane feature of the target candidate lane, ensuring that the first lane feature of the target candidate lane also includes the features of other target candidate lanes. Furthermore, it interacts with the second vehicle features of surrounding vehicles, the first lane feature of the target candidate lane, and the first vehicle feature of the target vehicle to obtain fused features, which then include the features of the target vehicle, all surrounding vehicles, and all target candidate lanes. Finally, it performs prediction based on the fused features, thereby improving the accuracy of vehicle trajectory prediction. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the first embodiment of the vehicle trajectory prediction method provided in this application;

[0024] Figure 2 This is a flowchart illustrating an embodiment of the vehicle trajectory prediction method provided in this application for obtaining the second vehicle features of the corresponding surrounding vehicles in the first embodiment.

[0025] Figure 3 This is a flowchart illustrating an implementation of the vehicle trajectory prediction method provided in this application for obtaining the second lane segment features of the corresponding lane segment in the first embodiment.

[0026] Figure 4 This is a flowchart illustrating the process of obtaining fused features in the first embodiment of the vehicle trajectory prediction method provided in this application.

[0027] Figure 5 This is a flowchart illustrating the second embodiment of the vehicle trajectory prediction method provided in this application;

[0028] Figure 6 This is a schematic diagram of the framework of one embodiment of the electronic device provided in this application;

[0029] Figure 7 This is a schematic diagram of a framework of one embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0031] It should be noted that the embodiments of this application contain descriptions involving "first," "second," etc., which are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0032] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0033] Please see Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the vehicle trajectory prediction method provided in this application. The method includes:

[0034] S11: Obtain the first vehicle features of the target vehicle and the surrounding vehicles, and obtain the first lane segment features of the lane segment of the target candidate lane.

[0035] In one implementation, the target vehicle is the vehicle to be predicted. The target vehicle is surrounded by corresponding road conditions and traffic participants. The algorithm predicts the future trajectory of the target vehicle based on its historical trajectory and surrounding information. Vehicles within a given range around the target vehicle are generally referred to as surrounding vehicles. This given range is set by the user, for example, a circle with a radius of 50 meters centered on the target vehicle. Surrounding vehicles may be located in target candidate lanes, which are lanes that the target vehicle can reach by going straight, turning, changing lanes, or making a U-turn.

[0036] The first vehicle features of the target vehicle and the first vehicle features of surrounding vehicles can be obtained based on the historical trajectories of the target vehicle and the surrounding vehicles, respectively. Specifically, a feature extraction network can be used to extract features based on the historical trajectories of the target vehicle and the surrounding vehicles to obtain the first vehicle features of the target vehicle and the first vehicle features of the surrounding vehicles. The first lane segment features of the lane segment of the target candidate lane can be obtained by using a feature extraction network based on the information of the lane points contained in the lane segment. The feature extraction network can be, but is not limited to, LSTM (Long Short-Term Memory). The target candidate lane can be divided into one or more lane segments as needed. In one specific implementation, if the target candidate lane is divided into multiple lane segments, each lane segment contains the same number of lane points to ensure that the feature dimension of each lane segment is the same. A lane point is a point located on the centerline of the target candidate lane.

[0037] S12: Interact with the first vehicle features of any surrounding vehicle to obtain the second vehicle features of the corresponding surrounding vehicle; and interact with the first lane segment features of any lane segment of the target candidate lane to obtain the second lane segment features of the corresponding lane segment.

[0038] In one embodiment, any surrounding vehicle is selected, and its first vehicle feature is interacted with the first vehicle features of each of the other surrounding vehicles to obtain a second vehicle feature corresponding to that surrounding vehicle. This second vehicle feature then includes the features of the other surrounding vehicles; wherein, the other surrounding vehicles are any surrounding vehicle other than the selected vehicle. By performing the above operation on each surrounding vehicle, the second vehicle feature corresponding to each surrounding vehicle can be obtained, meaning each surrounding vehicle corresponds to one first vehicle feature.

[0039] Similarly, for each lane segment of the target candidate lane, any lane segment is selected, and its first lane segment feature is interacted with the first lane segment features of other lane segments to obtain the second lane segment feature of the corresponding lane segment. Other lane segments can be any lane segment of the target candidate lane other than the selected lane segment, that is, other lane segments and the selected lane segment can belong to the same target candidate lane; other lane segments can also be any lane segment of other target candidate lanes, that is, other lane segments and the selected lane segment belong to different target candidate lanes.

[0040] S13: Fuse the second lane segment features of the lane segments on the same target candidate lane to obtain the first lane features of the target candidate lane.

[0041] In one implementation, the lane segmentation operation can be performed only once for each target candidate lane. For example, each target candidate lane can be divided into 3 segments, and each lane segment can contain the same number of lane points. After obtaining the second lane segment features corresponding to each lane segment, the second lane segment features of lane segments belonging to the same target candidate lane can be concatenated in sequence to obtain the first lane feature of that target candidate lane. The sequence refers to the arrangement order of the lane segments within the lane.

[0042] In another embodiment, the lane segmentation operation can be performed multiple times for each target candidate lane. The number of lane segments obtained for the same target candidate lane each time can be different. For example, the first target candidate lane can be divided into 3 lane segments in the first instance, and into 5 lane segments in the second instance. During the same lane segmentation, each target candidate lane can be divided into the same number of lane segments. For example, each target candidate lane can be divided into 3 lane segments in the first instance, and into 5 lane segments in the second instance.

[0043] After performing lane segmentation and feature interaction on each target candidate lane, a set of second lane segment features corresponding to that lane segment is obtained. That is, the number of times lane segmentation is performed on the same target candidate lane is the same as the number of sets of second lane segment features obtained. When multiple sets of second lane segment features are obtained, the second lane segment features belonging to the same target candidate lane in each set are concatenated sequentially to obtain the second lane feature of that target candidate lane. Then, the second lane features belonging to the same target candidate lane in multiple sets of second lane segment features are added together to obtain the first lane feature of the target candidate lane.

[0044] S14: The second vehicle features of surrounding vehicles, the first lane features of the target candidate lane, and the first vehicle features of the target vehicle are interacted to obtain fused features.

[0045] In one embodiment, the target vehicle's first vehicle feature can be interacted with the first lane features of all target candidate lanes to obtain the target vehicle's second vehicle feature; then, the target vehicle's second vehicle feature can be interacted with the second vehicle features of all surrounding vehicles to obtain the fused feature.

[0046] In another embodiment, the target vehicle's first vehicle features and all surrounding vehicles' second vehicle features can be interacted first, and then the interacted features can be interacted with the first lane features of all target candidate lanes to obtain fused features.

[0047] S15: Based on the fusion features, a prediction trajectory of the target vehicle is obtained.

[0048] In one implementation, the fused features can be input into a multi-layer perceptron (MLP). The MLP outputs a feature vector obtained by extracting features from the fused features, and the predicted trajectory of the target vehicle can be obtained based on the feature vector. For example, the MLP can output a one-dimensional vector with dimension K*f*2, which can be reconstructed into a three-dimensional vector with dimension [K,f,2], where K is the number of modalities, i.e., the number of predicted trajectories generated, f is the number of prediction times, and 2 is the coordinate dimension. The predicted trajectory of the target vehicle can be determined using this three-dimensional vector.

[0049] In one embodiment, the aforementioned interaction can be based on multi-head attention, that is, multi-head interactive attention processing is performed using the first vehicle features of surrounding vehicles and other surrounding vehicles to obtain the second vehicle features of the corresponding surrounding vehicles; and / or, multi-head interactive attention processing is performed using the first lane segment features of the lane segment of the target candidate lane to obtain the second lane segment features of the lane segment; and / or, multi-head interactive attention processing is performed using the second vehicle features of surrounding vehicles, the first lane features of the target candidate lane, and the first vehicle features of the target vehicle to obtain fused features.

[0050] In other implementations, the aforementioned interaction can be performed using a neural network, such as a graphical neural network or a convolutional neural network, that is, using a neural network to interact with features.

[0051] The above method, after obtaining the first vehicle features of the target vehicle and its surrounding vehicles, as well as the first lane segment features of the lane segments of the target candidate lane, firstly, interacts the first vehicle features of any surrounding vehicle with those of other surrounding vehicles to obtain the second vehicle features corresponding to that vehicle, ensuring that the second vehicle features of that vehicle include the features of other surrounding vehicles; then, interacts the first lane segment features of any lane segment of the target candidate lane with those of other lane segments to obtain the second lane segment features corresponding to that lane segment, ensuring that the second lane segment features of that lane segment include the features of other lane segments; secondly, the second lane segment features of lane segments on the same target candidate lane are fused to obtain the first lane features of the target candidate lane, ensuring that the first lane features of the target candidate lane also include the features of other target candidate lanes; furthermore, the second vehicle features of surrounding vehicles, the first lane features of the target candidate lane, and the first vehicle features of the target vehicle are interacted to obtain fused features, which at this point include the features of the target vehicle, all surrounding vehicles, and all target candidate lanes; finally, prediction is performed based on the fused features, thereby improving the accuracy of vehicle trajectory prediction.

[0052] Please see Figure 2 , Figure 2This is a flowchart illustrating an implementation of the vehicle trajectory prediction method provided in this application for obtaining the second vehicle features of corresponding surrounding vehicles in the first embodiment. The method involves interacting the first vehicle features of any surrounding vehicle with the first vehicle features of other surrounding vehicles to obtain the second vehicle features of the corresponding surrounding vehicle, including:

[0053] S221: Select one surrounding vehicle as the first surrounding vehicle, and designate the other surrounding vehicles as the second surrounding vehicles respectively.

[0054] In one embodiment, if there are multiple surrounding vehicles around the target vehicle, any one of the surrounding vehicles can be selected as the first surrounding vehicle, and the remaining surrounding vehicles can be selected as the second surrounding vehicles.

[0055] S222: Using the first vehicle features of the second surrounding vehicle as the key element and value element, and the first vehicle features of the first surrounding vehicle as the query element, perform multi-head interactive attention processing to obtain the third vehicle features of the first surrounding vehicle.

[0056] Specifically, a feature vector is obtained by performing multi-head interactive attention processing on the first vehicle feature of the first surrounding vehicle as the query element and the first vehicle feature of the first second surrounding vehicle as the key and value element. This feature vector can be a one-dimensional vector. Then, a new feature vector is obtained by performing multi-head interactive attention processing on the first vehicle feature of the second surrounding vehicle as the query element and the first vehicle feature of the second second surrounding vehicle as the key and value element. By taking turns using the first vehicle features of each second surrounding vehicle as the key and value element for feature interaction, the third vehicle feature of the first surrounding vehicle can be obtained.

[0057] S223: The third vehicle features of the first surrounding vehicle and the first vehicle features are fused to obtain the second vehicle features of the first surrounding vehicle.

[0058] In one embodiment, fusing the third vehicle feature and the first vehicle feature of the first surrounding vehicle can be achieved by adding the third vehicle feature of the first surrounding vehicle to the first vehicle feature of the first surrounding vehicle to obtain the second vehicle feature of the first surrounding vehicle. It is understood that in other embodiments, other methods can also be used to fuse the two features. In this case, the second vehicle feature of the first surrounding vehicle includes the features of all surrounding vehicles.

[0059] Please see Figure 3 , Figure 3 This is a flowchart illustrating an implementation of the vehicle trajectory prediction method provided in this application for obtaining the second lane segment feature of a corresponding lane segment in the first embodiment. The method involves interacting any lane segment of the target candidate lane with the first lane segment features of other lane segments to obtain the second lane segment feature of the corresponding lane segment, including:

[0060] S321: Divide each target candidate lane into a first preset number of lane segments to obtain a set of lane segments.

[0061] In one embodiment, considering that the influence of each lane segment of the target candidate lane on the prediction of the target vehicle trajectory varies, the target candidate lane is divided into a first preset number of lane segments. The first preset number is set by the user, for example, it is 1, 3, or 5. Assuming the first preset number is 3 and there are also 3 target candidate lanes, dividing each target candidate lane into the first preset number of lane segments will result in 9 lane segments. Each lane segment contains the same number of lane points.

[0062] S322: Take any lane segment in the set of lane segments as the first lane segment, and take the lane segments other than the first lane segment as the second lane segment.

[0063] Specifically, for any lane segment in the set of lane segments, it is designated as the first lane segment, and the lane segments other than the first lane segment are designated as the second lane segments. The second lane segment may belong to the same target candidate lane as the first lane segment, or it may belong to a different target candidate lane from the first lane segment.

[0064] S323: Using the first lane segment feature of the second lane segment as the key element and value element, and the first lane segment feature of the first lane segment as the query element, perform multi-head interactive attention processing to obtain the third lane segment feature of the first lane segment.

[0065] Specifically, using the first lane segment feature of the first lane segment as the query element, and the first lane segment feature of the first second lane segment as the key and value element, a feature vector is obtained after multi-head interactive attention processing. This feature vector can be a one-dimensional vector. Then, using this feature vector as the query element, and the first lane segment feature of the second second lane segment as the key and value element, a new feature vector is obtained after multi-head interactive attention processing. By taking turns using the first lane segment features of each second lane segment as the key and value element for feature interaction, the third lane segment feature of the first lane segment can be obtained.

[0066] S324: The first lane segment features and the third lane segment features of the first lane segment are fused to obtain the second lane segment features corresponding to the first lane segment.

[0067] In one embodiment, fusing the first lane segment features and the third lane segment features of the first lane segment can be achieved by adding the first lane segment features and the third lane segment features of the first lane segment to obtain the second lane segment features corresponding to the first lane segment. It is understood that in other embodiments, other methods can also be used to fuse the two features. In this case, the second lane segment features of the first lane segment obtained include the features of all lane segments. By performing the above operation on each lane segment, the second lane segment features corresponding to each first lane segment can be obtained.

[0068] In this embodiment, each target candidate lane can be segmented only once. After obtaining the second lane segment features of each first lane segment, the second lane segment features of the first lane segments belonging to the same target candidate lane are spliced ​​together according to the corresponding lane segment arrangement order to obtain the first lane features of each target candidate lane.

[0069] In other implementations, each target candidate lane can be segmented multiple times to fully consider the impact of the local and global features of each target candidate lane on the prediction of the target vehicle trajectory.

[0070] In one embodiment, before interacting with the first lane segment features of any lane segment of the target candidate lane to other lane segments, the method further includes obtaining a segment set; wherein the segment set includes multiple first preset quantities; the multiple first preset quantities in the segment set can be set by the user. When multiple first preset quantities are set, multiple sets of second lane segment features need to be obtained accordingly. At this point, the first lane segment features of any lane segment of the target candidate lane are interacted with the first lane segment features of other lane segments to obtain the second lane segment features of the corresponding lane segment. This includes: for each first preset number in the segment set, the following steps are performed: each target candidate lane is divided into a first preset number of lane segments to obtain a lane segment set; any lane segment in the lane segment set is taken as the first lane segment, and the lane segments other than the first lane segment are taken as the second lane segments; multi-head interactive attention processing is performed using the first lane segment features of the second lane segment as the key element and value element, and the first lane segment features of the first lane segment as the query element, to obtain the third lane segment features of the corresponding first lane segment; the first lane segment features and the third lane segment features of the first lane segment are fused to obtain multiple sets of second lane segment features corresponding to the first lane segment; wherein, each set of second lane segment features corresponding to the first lane segment is obtained by dividing each target candidate lane into the same first preset number of lane segments.

[0071] After obtaining multiple sets of second lane segment features corresponding to the first lane segment, the second lane segment features of lane segments on the same target candidate lane are fused to obtain the first lane feature of the target candidate lane. Specifically, the second lane segment features of lane segments on the same target candidate lane in each set of second lane segment features are concatenated to obtain the second lane feature of the target candidate lane; the second lane features belonging to the same target candidate lane in multiple sets of second lane segment features are added together to obtain the first lane feature of the target candidate lane.

[0072] For ease of understanding, consider the following example: Assume the segment set contains two first preset quantities, 3 and 5 respectively; the number of target candidate lanes is 3. When the first preset quantity is 3, each target candidate lane is divided into 3 lane segments, resulting in 9 lane segments. For each lane segment, the first lane segment feature of that lane segment is used as the query element, and the first lane segment features of other lane segments are used as the key and value elements in turn for multi-head interactive attention processing to obtain the third lane segment features corresponding to each lane segment. The third lane segment features and the first lane segment features corresponding to each lane segment are added together to obtain the second lane segment features corresponding to each lane segment. The second lane segment features corresponding to each lane segment are summarized into a set to obtain the first set of second lane segment features. When the first preset quantity is 5, each target candidate lane is divided into 5 lane segments, resulting in 15 lane segments; the second set of second lane segment features is obtained in the same way. For the first group of second lane segment features, the second lane segment features of lane segments belonging to the same target candidate lane are concatenated to obtain the first second lane feature of the three target candidate lanes; similarly, for the second group of second lane segment features, the second lane segment features of lane segments belonging to the same target candidate lane are concatenated to obtain the second second lane feature of the three target candidate lanes; the first second lane feature of the three target candidate lanes and the second second lane feature of the two second lane features belonging to the same target candidate lane are added together to obtain the first lane feature of the target candidate lane.

[0073] This implementation divides the target candidate lane into multiple lane segments, which can fully consider the local and global features of the target candidate lane during the trajectory prediction process of the target vehicle, thereby improving the accuracy of the target vehicle trajectory prediction.

[0074] Please see Figure 4 , Figure 4 This is a flowchart illustrating the first embodiment of the vehicle trajectory prediction method provided in this application, which obtains fused features by interacting with the second vehicle features of surrounding vehicles, the first lane features of the target candidate lane, and the first vehicle features of the target vehicle. The fused features include:

[0075] S441: Use the first vehicle features of the target vehicle and the first lane features of the target candidate lane to perform multi-head interactive attention processing to obtain the second vehicle features of the target vehicle.

[0076] In one embodiment, multi-head interactive attention processing is performed using the first vehicle feature of the target vehicle as the query element and the first lane features of all target candidate lanes as the key and value elements to obtain the second vehicle feature of the target vehicle. The second vehicle feature of the target vehicle integrates the features of each target candidate lane and the target vehicle.

[0077] S442: Multi-head interactive attention processing is performed using the second vehicle features of the target vehicle and the second vehicle features of surrounding vehicles to obtain fused features;

[0078] Specifically, using the second vehicle feature of the target vehicle as the query element and the second vehicle features of all surrounding vehicles as the key and value elements, multi-head interactive attention processing is performed to obtain fused features, which include features of each lane, each surrounding vehicle, and the target vehicle.

[0079] In this implementation, the features between surrounding vehicles and between lane segments are first interacted to obtain second vehicle features for the corresponding surrounding vehicles and second lane segment features for the corresponding lane segments. The second vehicle features of the surrounding vehicles include the features of all other surrounding vehicles, and the second lane segment features of the lane segments include the features of all other lane segments. The second lane segment features of each lane segment are then used to obtain the first lane features of the target candidate lane, ensuring that the first lane features of the target candidate lane include the features of other lanes. Finally, the first lane features of the target candidate lane, the first vehicle features of the target vehicle, and the second vehicle features of the surrounding vehicles are used to obtain a fused feature, which includes the features of each lane, each surrounding vehicle, and the target vehicle. By using the fused feature for prediction, a highly accurate predicted trajectory of the target vehicle can be obtained.

[0080] Please see Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the vehicle trajectory prediction method provided in this application. The method includes:

[0081] S51: Obtain the historical trajectory of the target vehicle and surrounding vehicles, and obtain the set of lane points for the target candidate lane.

[0082] In this context, each lane point in the lane point set is located on the centerline of the target candidate lane.

[0083] In one implementation, the historical trajectories of all vehicles and target candidate lanes can be predetermined. All vehicles include the target vehicle and all surrounding vehicles. The method for determining the target candidate lanes is as follows: First, select the lane point closest to the target vehicle and designate it as the starting lane point. Second, use a depth-first search algorithm to traverse the lane points adjacent to the starting lane point to determine the initial candidate lanes. Specifically, assuming the current candidate lane set is L1, add the starting lane point to L1. Select a lane point Pnext connected to the starting lane point. Lane point Pnext is the lane point that the target vehicle can reach through operations such as going straight, turning, changing lanes, or making a U-turn. If there are multiple lane points Pnext, add one of them to L1 and use it as the new starting lane point. Continue searching for lane points connected to the new starting lane point, repeating this process until there are no more lane points connected to the new starting lane point. Traversing all lane points Pnext connected to the starting lane point yields the initial candidate lane set. The number of lane points contained in each initial candidate lane in the initial candidate lane set can be the same or different. Finally, based on the initial candidate lanes, determine the target candidate lanes. In one embodiment, it can be determined whether the number of lane points contained in the initial candidate lane reaches a second preset number. If the number of lane points in the initial candidate lane reaches the second preset number, the initial candidate lane is designated as the target candidate lane. If the number of lane points in the initial candidate lane does not reach the second preset number, lane points in the initial candidate lane are padded with zeros to make the number of lane points in the target candidate lane equal to the second preset number, thus obtaining the target candidate lane. Specifically, if the number of lane points in the initial candidate lane is greater than the second preset number, then the second preset number of lane points are selected as lane points of the target candidate lane according to the order in which the lane points were added. After determining the target candidate lane, the set of lane points for the target candidate lane is obtained.

[0084] The historical trajectory of a vehicle is determined as follows: Consider the current time T0 and the previous h times {T} h For the i-th surrounding vehicle, its position coordinates in the current local coordinate system at time t are Vi, T(h1), ..., T2, T1}. i t Then the historical trajectory V of the vehicle i ={V i h V i (h-1) ,…,V i 1 V i 0}; For the target vehicle V0 and n surrounding vehicles {V1,V2,…,V...} nThe historical trajectories of all vehicles are V = {V0, V1, ..., V}. n}

[0085] After determining the historical trajectories of all vehicles and the set of lane points for the target candidate lane, the system can respond to user commands by acquiring the historical trajectories of all vehicles and the set of lane points for the target candidate lane. The system then inputs the historical trajectories of all vehicles and the set of lane points for the target candidate lane into a neural network, enabling the neural network to extract features based on the historical trajectories of the target vehicle and surrounding vehicles to obtain the first vehicle features of the target vehicle and surrounding vehicles. Furthermore, the system extracts features based on each lane point in the set of lane points to obtain the first lane segment features of the lane segment of the target candidate lane.

[0086] S52: Based on the historical trajectories of the target vehicle and surrounding vehicles, feature extraction is performed to obtain the first vehicle features of the target vehicle and surrounding vehicles; and based on each lane point in the lane point set, feature extraction is performed to obtain the first lane segment features of the lane segment of the target candidate lane.

[0087] In one embodiment, step S52 can be performed by a neural network; in other embodiments, step S52 can also be performed by a feature extraction model. No specific limitation is made here.

[0088] In one specific implementation, the vehicle's historical trajectory is defined by dimensions [+1, d]. v The array description of ], where d v These are the characteristic dimensions of a vehicle at a given moment, including coordinates (two dimensions), velocity (two dimensions), acceleration (two dimensions), heading angle (one dimension), etc., and can be arbitrarily expanded as needed. The lane points of the target candidate lane are defined by dimensions [l, d]. l The array description of ], where d l These are the characteristic dimensions of a lane point, including coordinates (two dimensions), whether the lane point has obstacles (one dimension), whether the lane point is on the stop line (one dimension), whether the lane point is on the pedestrian crossing (one dimension), etc., which can be expanded arbitrarily according to needs.

[0089] In one implementation, feature extraction can be performed directly based on each lane point of the target candidate lane to obtain the lane features of the target candidate lane; alternatively, the target candidate lane can be divided into several lane segments. Specifically, each lane point in the set of lane points of the target candidate lane is classified to obtain the lane points belonging to each lane segment. Feature extraction is then performed based on the lane points corresponding to each lane segment to obtain the first lane segment feature of each lane segment. Assuming the target candidate lane is divided into Sj lane segments, and the target candidate lane contains a total of l lane points, then each lane segment contains Sj / l lane points, and the dimension of the lane points is... After feature extraction from lane points based on lane segments, the output dimension can be [dimension not specified]. Given a one-dimensional vector, for the target candidate lane, we can obtain Sj dimensions. A one-dimensional vector.

[0090] S53: Interact with the first vehicle features of any surrounding vehicle to obtain the second vehicle features of the corresponding surrounding vehicle; and interact with the first lane segment features of any lane segment of the target candidate lane to obtain the second lane segment features of the corresponding lane segment.

[0091] In one implementation, if the target candidate lanes are segmented, the features of each lane segment can be interacted. If the target candidate lanes are not segmented, the features of each target candidate lane can be interacted, or no interaction can be performed. The specific choice can be made as needed.

[0092] S54: Fuse the second lane segment features of the lane segments on the same target candidate lane to obtain the first lane features of the target candidate lane.

[0093] S55: The fused features are obtained by interacting with the second vehicle features of surrounding vehicles, the first lane features of the target candidate lane, and the first vehicle features of the target vehicle.

[0094] S56: Based on the fusion features, a prediction trajectory of the target vehicle is obtained.

[0095] The detailed implementation of steps S53-S56 can be found in steps S12-S15 of the first embodiment of the vehicle trajectory prediction method provided in this application, and will not be repeated here.

[0096] In this implementation, the set of lane points for each target candidate lane and the historical trajectory of each vehicle can be determined based on the original data. The original data includes the world coordinates of all target candidate lanes near the target vehicle and all surrounding vehicles at the current time and several previous times. The target candidate lanes are divided into different segments to extract lane features of different lengths, thereby improving the prediction accuracy of vehicle trajectories.

[0097] Please see Figure 6 , Figure 6 This is a schematic diagram of one embodiment of the electronic device provided in this application.

[0098] The electronic device 60 includes a memory 61 and a processor 62 coupled to each other. The memory 61 stores program instructions, and the processor 62 executes the program instructions to implement the steps in any of the above method embodiments. Specifically, the electronic device 60 may include, but is not limited to, desktop computers, laptops, servers, mobile phones, tablets, etc., and is not limited thereto.

[0099] Specifically, processor 62 controls itself and memory 61 to implement the steps in any of the above method embodiments. Processor 62 may also be referred to as a CPU (Central Processing Unit). Processor 62 may be an integrated circuit chip with signal processing capabilities. Processor 62 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 62 may be implemented using integrated circuit chips.

[0100] Please see Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 70 stores program instructions 71, which, when executed by a processor, are used to implement the steps in any of the above method embodiments.

[0101] The computer-readable storage medium 70 can specifically be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium that can store computer programs. Alternatively, it can be a server that stores the computer program, which can send the stored computer program to other devices for execution or can also run the stored computer program itself.

[0102] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0103] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A vehicle trajectory prediction method, characterized by, The method comprises the following steps: obtaining first vehicle features of a target vehicle and surrounding vehicles of the target vehicle, and obtaining first lane segment features of lane segments of a target candidate lane; wherein the target candidate lane is a lane that the target vehicle can reach through straight driving, turning, lane changing or U-turning; interacting any of the surrounding vehicles with the first vehicle features of other surrounding vehicles to obtain second vehicle features corresponding to the surrounding vehicles; and interacting any of the lane segments of the target candidate lane with the first lane segment features of other lane segments to obtain second lane segment features corresponding to the lane segments; fusing the second lane segment features of the lane segments on the same target candidate lane to obtain first lane features of the target candidate lane; interacting the second vehicle features of the surrounding vehicles, the first lane features of the target candidate lane and the first vehicle features of the target vehicle to obtain fusion features; predicting based on the fusion features to obtain a predicted trajectory of the target vehicle; all the interactions are based on multi-head attention interaction; the interaction of any of the surrounding vehicles with the first vehicle features of other surrounding vehicles to obtain second vehicle features corresponding to the surrounding vehicles comprises: selecting a surrounding vehicle as a first surrounding vehicle, selecting other surrounding vehicles as second surrounding vehicles, performing multi-head interaction attention processing on the first vehicle features of the first surrounding vehicle as a query element, the first vehicle features of the first second surrounding vehicle as key elements and value elements to obtain a feature vector, performing multi-head interaction attention processing on the feature vector as a query element, the first vehicle features of the second second surrounding vehicle as key elements and value elements to obtain a new feature vector, performing feature interaction on the first vehicle features of each second surrounding vehicle as key elements and value elements in turn to obtain third vehicle features of the first surrounding vehicle, fusing the third vehicle features of the first surrounding vehicle and the first vehicle features to obtain second vehicle features of the first surrounding vehicle.

2. The method of claim 1, wherein, the interaction of any of the lane segments of the target candidate lane with the first lane segment features of other lane segments to obtain second lane segment features corresponding to the lane segments comprises: dividing each of the target candidate lanes into a first preset number of lane segments to obtain a lane segment set; taking any of the lane segments in the lane segment set as a first lane segment, and taking the lane segments other than the first lane segment as second lane segments; performing multi-head interaction attention processing on the first lane segment features of the second lane segments as key elements and value elements, and the first lane segment feature of the first lane segment as a query element to obtain third lane segment features of the first lane segment; fusing the first lane segment feature of the first lane segment and the third lane segment feature to obtain second lane segment features corresponding to the first lane segment.

3. The method of claim 1, wherein, before the interaction of any of the lane segments of the target candidate lane with the first lane segment features of other lane segments, the method further comprises: Obtain a segment set, wherein the segment set comprises a plurality of first preset quantities; The interaction of any of the lane segments of the target candidate lane with the first lane segment features of other lane segments obtains second lane segment features corresponding to the lane segments, comprising: For each of the first preset quantities in the segment set, the following steps are performed: Divide each of the target candidate lanes into a first preset number of lane segments to obtain a lane segment set; Any of the lane segments in the lane segment set is taken as a first lane segment, and the lane segments other than the first lane segment are taken as second lane segments respectively; Respectively taking the first lane segment features of the second lane segments as key elements and value elements and the first lane segment features of the first lane segments as query elements, multi-head interaction attention processing is performed to obtain third lane segment features corresponding to the first lane segments; Fusing the first lane segment features of the first lane segments and the third lane segment features obtains a plurality of groups of second lane segment features corresponding to the first lane segments; wherein each group of second lane segment features corresponding to the first lane segments is obtained after each of the target candidate lanes is divided into the same first preset number of lane segments; The second lane segment features of the lane segments on the same target candidate lane are fused to obtain the first lane feature of the target candidate lane, comprising: Splicing the second lane segment features of the lane segments on the same target candidate lane in each group of second lane segment features obtains the second lane feature of the target candidate lane; Adding the second lane features belonging to the same target candidate lane in a plurality of groups of second lane segment features obtains the first lane feature of the target candidate lane.

4. The method of claim 1, wherein, The interaction of the second vehicle features of the surrounding vehicles, the first lane feature of the target candidate lane, and the first vehicle feature of the target vehicle obtains a fusion feature, comprising: Multi-head interaction attention processing is performed on the first vehicle feature of the target vehicle and the first lane feature of the target candidate lane to obtain the second vehicle feature of the target vehicle; Multi-head interaction attention processing is performed on the second vehicle feature of the target vehicle and the second vehicle feature of the surrounding vehicles to obtain the fusion feature.

5. The method of claim 4, wherein, The multi-head interaction attention processing on the first vehicle feature of the target vehicle and the first lane feature of the target candidate lane to obtain the second vehicle feature of the target vehicle, comprising: Taking the first vehicle feature of the target vehicle as a query element, the first lane feature of the target candidate lane as a key element and a value element, and performing multi-head interaction attention processing obtains the second vehicle feature of the target vehicle; And / or, the multi-head interaction attention processing on the second vehicle feature of the target vehicle and the second vehicle feature of the surrounding vehicles to obtain the fusion feature, comprising: Taking the second vehicle feature of the target vehicle as a query element, the second vehicle feature of the surrounding vehicles as a key element and a value element, and performing multi-head interaction attention processing obtains the fusion feature.

6. The method of claim 1, wherein, Before the method of acquiring the first vehicle features of the target vehicle and surrounding vehicles of the target vehicle, and acquiring the first lane segment features of the lane segments of the target candidate lane, the method further comprises: acquiring historical trajectories of the target vehicle and the surrounding vehicles, and acquiring a set of lane points of the target candidate lane; each lane point in the set of lane points is located on a center line of the target candidate lane; the method of acquiring the first vehicle features of the target vehicle and surrounding vehicles of the target vehicle comprises: performing feature extraction based on the historical trajectories of the target vehicle and the surrounding vehicles to obtain the first vehicle features of the target vehicle and the surrounding vehicles; the method of acquiring the first lane segment features of the lane segments of the target candidate lane comprises: performing feature extraction based on each lane point in the set of lane points to obtain the first lane segment features of the lane segments of the target candidate lane.

7. The method of claim 6, wherein, the method of performing feature extraction based on each lane point in the set of lane points to obtain the first lane segment features of the lane segments of the target candidate lane comprises: classifying each lane point in the set of lane points to obtain lane points belonging to each lane segment; performing feature extraction based on the lane points of each lane segment to obtain the first lane segment features of each lane segment.

8. The method of claim 6, wherein, Before the method of acquiring the set of lane points of the target candidate lane, the method further comprises: selecting a lane point closest to the target vehicle as a starting lane point; using a depth-first traversal algorithm to traverse lane points adjacent to the starting lane point to determine an initial candidate lane; determining a target candidate lane based on the initial candidate lane.

9. The method of claim 8, wherein, the method of determining a target candidate lane based on the initial candidate lane comprises: determining whether the number of lane points contained in the initial candidate lane reaches a second preset number; in response to the number of lane points contained in the initial candidate lane reaching the second preset number, taking the initial candidate lane as the target candidate lane; in response to the number of lane points contained in the initial candidate lane not reaching the second preset number, supplementing the lane points of the initial candidate lane in a zero padding manner to obtain a target candidate lane; the target candidate lane contains a number of lane points equal to the second preset number.

10. An electronic device, comprising: The device comprises a memory and a processor coupled to each other; the memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the method of any one of claims 1-9.

11. A computer readable storage medium, characterized in that, The computer readable storage medium is used to store program instructions, which can be executed by the processor to implement the method of any one of claims 1-9. The computer readable storage medium is used to store program instructions, which can be executed by the processor to implement the method of any one of claims 1-9.

Images