Traffic participant trajectory prediction method, intelligent vehicle and storage medium

Abstract

The application discloses a traffic participant trajectory prediction method, an intelligent automobile and a storage medium. The traffic participant trajectory prediction method comprises the following steps: acquiring 2D historical trajectory grid information, 3D historical trajectory grid information and 2D traffic grid information of all traffic participants; respectively performing feature coding on the 2D historical trajectory grid information, the 3D historical trajectory grid information and the 2D traffic grid information, so as to correspondingly generate a plurality of 2D trajectory features, a plurality of 3D trajectory features and a plurality of 2D traffic map features; splicing the plurality of 2D trajectory features, the plurality of 3D trajectory features and the plurality of 2D traffic map features in corresponding scales, and performing multi-scale fusion after splicing to generate fusion features; performing feature decoding on the fusion features to generate decoding features, and using the decoding features to predict the trajectories of all traffic participants. In the above manner, the prediction effect of the trajectories of traffic participants can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent vehicles, in particular to a traffic participant trajectory prediction method, an intelligent vehicle and a storage medium. BACKGROUND

[0002] With the rapid development of science and technology, various intelligent products are becoming more and more popular in people's lives. In the field of automobiles, intelligent vehicles are developing rapidly. Intelligent vehicles can have automatic driving or assisted driving functions, which can reduce the burden of drivers. In a traffic scene, an intelligent vehicle needs to predict the trajectories of traffic participants within a certain range to plan the driving route of the intelligent vehicle.

[0003] In the prior art, the method for predicting the trajectory of a traffic participant often only predicts the trajectory of a single vehicle, does not predict all traffic participants within a certain range, and loses the interaction information between traffic participants, so the prediction effect needs to be improved. SUMMARY

[0004] The technical problem solved by the present application is to provide a traffic participant trajectory prediction method, an intelligent vehicle and a storage medium, which can improve the prediction effect of the trajectory of a traffic participant.

[0005] To solve the above technical problems, the first technical solution adopted by the present application is to provide a traffic participant trajectory prediction method, which comprises: acquiring 2d historical trajectory grid information, 3d historical trajectory grid information of all traffic participants within a preset map range, and 2d traffic grid information within the preset map range. The 2d historical trajectory grid information, the 3d historical trajectory grid information and the 2d traffic grid information are respectively encoded to correspondingly generate a plurality of 2d trajectory features corresponding to a plurality of scales, a plurality of 3d trajectory features corresponding to a plurality of scales and a plurality of 2d traffic map features corresponding to a plurality of scales. The plurality of 2d trajectory features, the plurality of 3d trajectory features and the plurality of 2d traffic map features are spliced in corresponding scales, and after splicing, a plurality of scales are fused to generate a fusion feature. The fusion feature is decoded to generate a decoding feature, and the decoding feature is used to predict the trajectory of all traffic participants on the corresponding map of the preset map range.

[0006] In some embodiments, the 2d historical trajectory grid map information, the 3d historical trajectory grid map information and the 2d traffic grid map information are respectively feature encoded to correspondingly generate a plurality of 2d trajectory features corresponding to a plurality of scales, a plurality of 3d trajectory features corresponding to a plurality of scales and a plurality of 2d traffic map features corresponding to a plurality of scales, including: feature encoding the 2d historical trajectory grid map information by using a first encoding network to generate the plurality of 2d trajectory features. Feature encoding the 3d historical trajectory grid map information by using a second encoding network to generate the plurality of 3d trajectory features. The second encoding network is different from the first encoding network. Feature encoding the 2d traffic grid map information by using a third encoding network to generate the plurality of 2d traffic map features. The third encoding network is different from the second encoding network and the first encoding network.

[0007] In some embodiments, the first encoding network is an HRNet-Transformer encoding network obtained by at least fusing an HRNet model and a Transformer model.

[0008] In some embodiments, the feature encoding of the 2d historical trajectory grid map information by using the first encoding network to generate the plurality of 2d trajectory features includes: integrating the 2d historical trajectory grid map information in a preset number of different Patch sizes to obtain a preset number of Patch block features corresponding to different sizes. Sending the preset number of Patch block features to corresponding Transformer Blocks respectively for generation processing to generate a preset number of 2d trajectory features. Performing a preset number of cascading processing on the preset number of 2d trajectory features to obtain a plurality of 2d trajectory features corresponding to a plurality of scales, and the number of the plurality of 2d trajectory features is equal to the sum of the preset number and the preset number of times.

[0009] In some embodiments, the integration of the 2d historical trajectory grid map information in a preset number of different Patch sizes includes: performing integration processing on the 2d historical trajectory grid map information by using two Patch Embedding layers generated by different Patches.

[0010] In some embodiments, the preset number of 2d trajectory features are subjected to a preset number of cascading processing, including: downsampling the smallest one of the 2d trajectory features in the current level in a Patch Merging manner to generate a spatial size feature. Sending the 2d trajectory features in the current level and the spatial size feature to corresponding Transformer Blocks respectively for generation processing to generate 2d trajectory features in the next level, and the number of the 2d trajectory features in the next level is one more than the number of the 2d trajectory features in the current level.

[0011] In some embodiments, the preset number of patch block features are respectively sent to corresponding Transformer Blocks for generation processing to generate a preset number of 2d trajectory features, including: processing the inputted preset number of patch block features by using three fully connected layers to generate a Query matrix, a Key matrix and a Value matrix. The Query matrix, the Key matrix and the Value matrix are processed by using a Swin-Transformer attention mechanism to obtain generated features, and the generated features are subjected to shape transformation operation to generate a preset number of initial 2d trajectory features. Each initial 2d trajectory feature is subjected to down-sampling or up-sampling, and the initial 2d trajectory feature subjected to down-sampling or up-sampling is converted into one-dimensional data through flattening processing. Each initial 2d trajectory feature is fused with one-dimensional data converted from the remaining initial 2d trajectory features subjected to down-sampling or up-sampling to generate a 2d trajectory feature corresponding to each initial 2d trajectory feature, thereby obtaining a preset number of 2d trajectory features.

[0012] In some embodiments, the second encoding network is a 3d convolution network including a BottleNeck having a 3d convolution layer, a batch processing layer and an activation function. The 3d historical trajectory grid map information is subjected to feature encoding by using the second encoding network to generate a plurality of 3d trajectory features, including: performing down-sampling on the 3d historical trajectory grid map information in the spatial dimension by using a single 3d convolution layer to generate a first 3d trajectory feature. The first 3d trajectory feature is subjected to preset number of cascading processing to obtain a plurality of 3d trajectory features.

[0013] In some embodiments, the first 3d trajectory feature is subjected to preset number of cascading processing to obtain a plurality of 3d trajectory features, including: alternately performing preset number of cascading processing on the first 3d trajectory feature in the time dimension and the spatial dimension to obtain a plurality of 3d trajectory features.

[0014] In some embodiments, the first 3d trajectory feature is subjected to preset number of cascading processing to obtain a plurality of 3d trajectory features, including: alternately performing preset number of cascading processing on the first 3d trajectory feature in the time dimension and the spatial dimension to obtain a plurality of 3d trajectory features.

[0015] In some embodiments, the third encoding network is a 2d convolution network including a BottleNeck with a 2d convolution layer, a batch layer and an activation function. The 2d traffic grid map information is encoded by the third encoding network to generate a plurality of 2d traffic map features, including: a first 2d traffic map feature is generated by using a single 2d convolution layer to down-sample the 2d traffic grid map information in spatial dimension and feature dimension. The first 2d traffic map feature is processed in cascade for a preset number of times to obtain a plurality of 2d traffic map features.

[0016] In some embodiments, the first 2d traffic map feature is processed in cascade for a preset number of times to obtain a plurality of 2d traffic map features, including: a next 2d traffic map feature is generated by using a BottleNeck to down-sample the first 2d traffic map feature.

[0017] In some embodiments, the plurality of 2d trajectory features, the plurality of 3d trajectory features and the plurality of 2d traffic map features are spliced in corresponding scales, and after splicing, a plurality of scale fusion features are fused to generate fusion features, including: a plurality of scale synthesis features are obtained by splicing the plurality of 2d trajectory features, the plurality of 3d trajectory features and the plurality of 2d traffic map features in corresponding sizes. A fusion scale synthesis feature is obtained by using a BiFPN multi-scale feature fusion network to fuse and process the plurality of scale synthesis features. The plurality of scale synthesis features and the fusion scale synthesis feature constitute the fusion features.

[0018] In some embodiments, the fusion features are decoded to generate decoding features, and the decoding features are used to predict the trajectories of all traffic participants on the map corresponding to the preset map range, including: the fusion features are decoded by using a 3d convolution network to generate decoding features. The decoding features are sent to a shared layer for prediction processing to realize trajectory prediction of all traffic participants on the map corresponding to the preset map range.

[0019] In some embodiments, the fusion features are decoded by using a 3d convolution network to generate decoding features, including: the fusion features are unfolded in time dimension by using a 3d convolution network, and time features of a prediction time step are output. The decoding features are sent to a shared layer for prediction processing, including: the time features of each step in the prediction time step are sent to the shared layer for prediction processing step by step according to the time dimension index.

[0020] In some embodiments, before acquiring the 2d historical trajectory raster map information of all the traffic participants in the preset map range, the 3d historical trajectory raster map information, and the 2d traffic raster map information in the preset map range, the method further comprises: acquiring the historical trajectory raster map information of all the traffic participants in the preset map range, the bird's eye view in the preset map range, and the traffic feature information in the preset map range. The historical trajectory raster map information comprises the contour size, the heading angle, the historical trajectory, and the current location of the traffic participants. The traffic feature information comprises the road information and the traffic signal information. The speed of all the traffic participants is calculated according to the historical trajectory raster map information and the time interval. The splicing operation is performed on all the traffic participants in the time dimension to generate the 3d historical trajectory raster map information comprising the length, the width, the time dimension, and the feature dimension. The time dimension of the 3d historical trajectory raster map information is compressed to the feature dimension to generate the 2d historical trajectory raster map information. The bird's eye view and the traffic feature information are spliced in the feature dimension to generate the 2d traffic raster map information.

[0021] To solve the above technical problems, a second technical solution adopted by the present application is to provide an intelligent automobile, which comprises a processor, a memory, and a detection component. The detection component is used to detect the historical trajectory raster map information of all the traffic participants in a preset map range, the bird's eye view in the preset map range, and the traffic feature information in the preset map range. The processor is coupled with the detection component. The memory stores a computer program. The processor is used to execute the computer program to implement the method provided by the first technical solution.

[0022] To solve the above technical problems, a third technical solution adopted by the present application is to provide a computer-readable storage medium, which stores a computer program. The computer program can be executed by a processor to implement the method provided by the first technical solution.

[0023] The beneficial effects of this application are as follows: Unlike existing technologies, by acquiring 2D historical trajectory raster image information, 3D historical trajectory raster image information, and 2D traffic raster image information of all traffic participants within a preset map area, feature encoding is performed on each of these information to generate multiple 2D trajectory features, multiple 3D trajectory features, and multiple 2D traffic map features corresponding to multiple scales. These multiple 2D trajectory features, multiple 3D trajectory features, and multiple 2D traffic map features are then mapped to each other. The system stitches together multiple scales and then fuses them to generate fused features. These fused features are then decoded to generate decoded features. The decoded features are then used to predict the trajectories of all traffic participants on a map corresponding to a preset map range. The feature encoding stage obtains multiple 2D trajectory features, multiple 3D trajectory features, and multiple 2D traffic map features corresponding to multiple scales. Multi-scale feature fusion is then performed to obtain features with different receptive field sizes. Furthermore, by considering the interaction features between all traffic participants within the preset map range, the prediction effect of predicting the trajectories of all traffic participants within the preset map range is improved. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating an embodiment of the method for predicting the trajectory of traffic participants in this application;

[0025] Figure 2 This is a schematic diagram of the network structure of the method for predicting the trajectory of traffic participants in this application;

[0026] Figure 3 This is a schematic diagram of the structure of the first coding network;

[0027] Figure 4 for Figure 3 An enlarged schematic diagram of the Stage 3 structure shown;

[0028] Figure 5 for Figure 3 An enlarged schematic diagram of the Stage 4 structure shown;

[0029] Figure 6 This is a schematic diagram of the structure of the second coding network;

[0030] Figure 7 This is a schematic diagram of the third coding network;

[0031] Figure 8 This is a schematic block diagram of an embodiment of the intelligent vehicle of this application;

[0032] Figure 9 This is a schematic block diagram of an embodiment of a computer-readable storage medium according to this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0034] Through long-term research, the inventors have discovered that in traffic scenarios, intelligent vehicles need to predict the trajectories of traffic participants within a certain range in order to plan the vehicle's driving route. Existing methods for predicting the trajectories of traffic participants often only predict the trajectory of a single vehicle, failing to predict the trajectory of all traffic participants within a certain range and losing interaction information between them, thus the prediction accuracy needs improvement. To address the above technical problems, this application provides the following embodiments.

[0035] like Figure 1 and Figure 2 As shown in the embodiment of the traffic participant trajectory prediction method of this application, the traffic participant trajectory prediction method includes: S100: acquiring 2D historical trajectory raster map information, 3D historical trajectory raster map information, and 2D traffic raster map information within the preset map range for all traffic participants. S200: performing feature encoding on the 2D historical trajectory raster map information, 3D historical trajectory raster map information, and 2D traffic raster map information respectively to generate multiple 2D trajectory features, multiple 3D trajectory features, and multiple 2D traffic map features corresponding to multiple scales. S300: stitching the multiple 2D trajectory features, multiple 3D trajectory features, and multiple 2D traffic map features at corresponding scales, and then fusing them at multiple scales to generate fused features. S400: decoding the fused features to generate decoded features, and using the decoded features to predict the trajectories of all traffic participants on the map corresponding to the preset map range.

[0036] Traffic participants may include vehicles, pedestrians, and non-motorized vehicles.

[0037] In the driving scene of an intelligent vehicle, various traffic participants and road facilities need to be identified, and the volumes and sizes of the various traffic participants and road facilities are often different. When performing feature extraction, the feature map at each position obtains input feature information with the same receptive field. However, for objects of different sizes and volumes, the receptive field requirements are often different. For example, when extracting the features of a target, the receptive field of a high-level network is relatively large, and the semantic information representation capability is strong, but the resolution of the feature map is low, and the geometric information representation capability is weak. The receptive field of a low-level network is relatively small, and the geometric detail information representation capability is strong, although the resolution is high, but the semantic information representation capability is weak. As the model depth continues to increase, the information related to the input features is lost layer by layer, which can cause small targets to be lost.

[0038] Therefore, the multiple 2D trajectory features, the multiple 3D trajectory features, and the multiple 2D traffic map features corresponding to multiple scales can be obtained through the feature encoding stage respectively, and then multi-scale feature fusion is performed to obtain features of different receptive field sizes, thereby improving the reliability of feature extraction of various traffic participants and road facilities.

[0039] By obtaining 2D historical trajectory grid map information, 3D historical trajectory grid map information of all traffic participants in a preset map range, and 2D traffic grid map information in the preset map range, the 2D historical trajectory grid map information, the 3D historical trajectory grid map information, and the 2D traffic grid map information are respectively feature-encoded to correspondingly generate multiple 2D trajectory features corresponding to multiple scales, multiple 3D trajectory features corresponding to multiple scales, and multiple 2D traffic map features corresponding to multiple scales. The multiple 2D trajectory features, the multiple 3D trajectory features, and the multiple 2D traffic map features are spliced according to the corresponding scales, and after splicing, multi-scale fusion is performed to generate fusion features. The fusion features are feature-decoded to generate decoding features, and the decoding features are used to predict the trajectories of all traffic participants on the corresponding map of the preset map range. Through the feature encoding stage, the multiple 2D trajectory features, the multiple 3D trajectory features, and the multiple 2D traffic map features corresponding to multiple scales are obtained, and then multi-scale feature fusion is performed to obtain features of different receptive field sizes. By considering the interaction features between all traffic participants in the preset map range, the prediction effect when predicting the trajectories of all traffic participants in the preset map range is improved.

[0040] The following describes in detail the embodiments of the traffic participant trajectory prediction method of the present application.

[0041] S100: Obtain 2D historical trajectory grid map information, 3D historical trajectory grid map information of all traffic participants in a preset map range, and 2D traffic grid map information in the preset map range.

[0042] By obtaining the 2D historical trajectory grid information, the 3D historical trajectory grid information and the 2D traffic grid information of the traffic participants, multiple orientation information supports can be provided for predicting the trajectories of the traffic participants, so as to improve the prediction effect.

[0043] By obtaining the grid information of all traffic participants in the preset map range, the interaction information between vehicles can be supported by integrating the interaction feature information between all traffic participants, so as to predict multiple traffic participants in the preset map range at the same time, and improve the prediction effect. For example, by obtaining the grid information of all vehicles in the preset map range, the interaction information between vehicles can be supported, so as to predict multiple vehicles in the preset map range at the same time.

[0044] Optionally, how to obtain the 2D historical trajectory grid information, the 3D historical trajectory grid information of all traffic participants in the preset map range and the 2D traffic grid information in the preset map range can refer to the following steps included in S100:

[0045] S101: Obtain the historical trajectory grid information of all traffic participants in the preset map range, the bird's eye view in the preset map range and the traffic feature information in the preset map range. The historical trajectory grid information includes the contour size, heading angle, historical trajectory and current position of the traffic participants, and the traffic feature information includes road information and traffic signal information.

[0046] By obtaining the historical trajectory grid information of the traffic participants, the bird's eye view in the preset map range and the traffic feature information in the preset map range, the basis for predicting the trajectories of the traffic participants can be provided, so as to improve the prediction effect.

[0047] By obtaining the contour size of the traffic participants, the occupied range of the vehicle volume can be considered when predicting the trajectories of the traffic participants, so as to avoid the collision of the traffic participants, for example, to avoid the occurrence of vehicle collision.

[0048] Optionally, the traffic signal information can include road indication signboards, traffic signal lights and the like.

[0049] Optionally, the intelligent automobile can include a laser radar, and the laser radar is used for the intelligent automobile to obtain the historical trajectory grid information of all traffic participants in the preset map range, the bird's eye view in the preset map range and the traffic feature information in the preset map range.

[0050] Optionally, the intelligent automobile can include a camera, and the camera is used for the intelligent automobile to obtain the historical trajectory grid information of all traffic participants in the preset map range, the bird's eye view in the preset map range and the traffic feature information in the preset map range.

[0051] S102: Calculate the speed of all traffic participants based on historical trajectory raster information and time intervals.

[0052] Specifically, the speed of all traffic participants is calculated using their historical trajectories and time intervals, such as vehicle speed.

[0053] S103: Perform a stitching operation on all traffic participants in the time dimension to generate a 3D historical trajectory raster map information containing length, width, time dimension, and feature dimension.

[0054] Furthermore, the 3D historical trajectory raster map information includes the speed of traffic participants.

[0055] S104: Compress the time dimension of the 3D historical trajectory raster image information to the feature dimension and stitch them together to generate 2D historical trajectory raster image information.

[0056] Specifically, the 2D historical trajectory raster image information includes length, width, and feature dimensions.

[0057] S105: Generate 2D traffic raster map information by stitching together bird's-eye view and traffic feature information in the feature dimension.

[0058] During the operation of intelligent vehicles, bird's-eye view and traffic feature information can provide traffic information beyond the traffic participants.

[0059] S200: Perform feature encoding on 2D historical trajectory raster image information, 3D historical trajectory raster image information and 2D traffic raster image information respectively to generate multiple 2D trajectory features corresponding to multiple scales, multiple 3D trajectory features corresponding to multiple scales and multiple 2D traffic map features corresponding to multiple scales.

[0060] like Figures 3 to 7 As shown, the number of generated 2D trajectory features, 3D trajectory features, and 2D traffic map features can be equal. For example, feature encoding of 2D historical trajectory raster information outputs five 2D trajectory features (C1, C2, C3, C4, and C5) arranged by size; feature encoding of 3D historical trajectory raster information outputs five 3D trajectory features (F1, F2, F3, F4, and F5) arranged by size; and feature encoding of 2D traffic raster information outputs five 2D traffic map features (A1, A2, A3, A4, and A5) arranged by size. C1, F1, and A1 correspond to scales, C2, F2, and A2 correspond to scales, and so on.

[0061] Further, C1, F1 and A1 have the same scale, C2, F2 and A2 have the same scale, and so on, facilitating subsequent splicing of corresponding scales.

[0062] By obtaining the plurality of 2d trajectory features corresponding to the plurality of scales, the plurality of 3d trajectory features corresponding to the plurality of scales, and the plurality of 2d traffic map features corresponding to the plurality of scales through the feature encoding stage, features of different receptive field sizes can be obtained, the reliability of feature extraction in the subsequent stage is improved, and the prediction effect of the trajectory of each traffic participant is further improved.

[0063] Optionally, as shown in Figures 3 to 5 How to respectively perform feature encoding on the 2d historical trajectory grid map information, the 3d historical trajectory grid map information and the 2d traffic grid map information to correspondingly generate the plurality of 2d trajectory features corresponding to the plurality of scales, the plurality of 3d trajectory features corresponding to the plurality of scales and the plurality of 2d traffic map features corresponding to the plurality of scales can be specifically referred to the following steps included in S200:

[0064] S210: performing feature encoding on the 2d historical trajectory grid map information by using a first encoding network to generate the plurality of 2d trajectory features.

[0065] The plurality of 2d trajectory features generated in this step have different scales.

[0066] Optionally, the first encoding network is an HRNet-Transformer encoding network obtained by fusing at least an HRNet model and a Transformer model.

[0067] By fusing the HRNet model and the Transformer model to obtain the HRNet-Transformer encoding network, the multi-scale feature fusion characteristics of the HRNet model and the multi-receptive field interaction feature extraction characteristics of the Transformer can be fused to form an encoding structure having the multi-scale feature fusion characteristics and the multi-receptive field interaction feature extraction characteristics.

[0068] In the 2d trajectory features, the time sequence features can be fused into the feature dimension. By fusing the 2d trajectory features of the plurality of scales, sufficient rich time sequence feature information can be captured, and the time sequence of the predicted traffic participant is prolonged, and the trajectory of the traffic participant is predicted for a longer time.

[0069] The HRNet-Transformer encoding network can adopt a hierarchical design. In some embodiments, the HRNet-Transformer encoding network can include a plurality of stages.

[0070] Optionally, the resolution of the input feature map can be reduced through the plurality of stages, and the receptive field can be expanded layer by layer.

[0071] Optionally, the Transformer model comprises a plurality of stacked encoders and decoders.

[0072] Further, the Transformer model comprises a Swin-Transformer model, for example, a Swin-Transformer V2.

[0073] Optionally, the 2d historical trajectory raster map information is feature-encoded by using a first encoding network to generate a plurality of 2d trajectory features. For details, refer to the following steps included in S210:

[0074] S211: The 2d historical trajectory raster map information is integrated in a preset number of different Patch sizes to obtain a preset number of Patch block features corresponding to different sizes.

[0075] Each Patch block feature can correspond to a 2d trajectory feature. Accordingly, the preset number of Patch block features can correspond to a plurality of 2d trajectory features corresponding to a plurality of scales.

[0076] Optionally, when the preset number of Patch block features is two, how to integrate the 2d historical trajectory raster map information in a preset number of different Patch sizes can refer to the following steps included in S211:

[0077] S2111: The 2d historical trajectory raster map information is integrated by using Patch Embedding layers generated by two different Patches.

[0078] The Patch Embedding layer can process the 2d historical trajectory raster map information as input information. The integration processing of the 2d historical trajectory raster map information by the Patch Embedding layers generated by two different Patches can generate two Patch block features.

[0079] S212: The preset number of Patch block features are respectively sent to the corresponding Transformer Block for generation processing to generate a preset number of 2d trajectory features.

[0080] For example, when the preset number is two, the two Patch block features are respectively sent to the corresponding Transformer Block for generation processing, and two 2d trajectory features can be generated.

[0081] For example, when the preset number is two, the two Patch block features are respectively sent to the corresponding Transformer Block for generation processing, and two 2d trajectory features can be generated. Figures 3 to 5As shown, specifically, each Stage of the HRNet-Transformer encoding network can include a plurality of Patch Mergings and a plurality of Transformer Blocks. The first Stage in the plurality of Stages can be denoted as Stage1. Stage1 can include a one-to-one correspondence of two Patch Mergings and two Transformer Blocks. Two 2d trajectory features can be outputted by inputting two patch block features into the two Patch Mergings and the two Transformer Blocks.

[0082] Optionally, the preset number of patch block features are respectively sent to the corresponding Transformer Blocks for generation processing to generate the preset number of 2d trajectory features. For details, refer to the following steps included in S212:

[0083] S2121: The preset number of patch block features are processed by using three fully connected layers to generate a Query matrix, a Key matrix, and a Value matrix.

[0084] S2122: The Query matrix, the Key matrix, and the Value matrix are processed by using a Swin-Transformer attention mechanism to obtain generated features, and the generated features are subjected to a shape transformation operation to generate the preset number of initial 2d trajectory features.

[0085] Specifically, the Swin-Transformer attention mechanism can be expressed in a formula form as follows:

[0086]

[0087] Wherein, B is a relative position offset, Q, K, and V are the Query matrix, the Key matrix, and the Value matrix respectively, and d is the feature dimension number of the Query matrix and the Key matrix.

[0088] S2123: Each initial 2d trajectory feature is subjected to down-sampling or up-sampling, and the initial 2d trajectory feature subjected to the down-sampling or the up-sampling is converted into one-dimensional data by flattening processing.

[0089] Specifically, the initial 2d trajectory feature is subjected to a Reshape operation to be converted into two-dimensional data, and then subjected to down-sampling convolution and bilinear interpolation up-sampling operations, and finally converted into one-dimensional data by flattening processing.

[0090] For example, if the preset quantity is two, there are two initial 2D trajectory features, denoted as B1 and B2. The initial 2D trajectory feature B1 can be reshaped to convert it into two-dimensional data, then downsampled by a factor of 2 and flattened to convert it into one-dimensional data. Similarly, the initial 2D trajectory feature B2 can be reshaped to convert it into two-dimensional data, then upsampled by a factor of 2 and flattened to convert it into one-dimensional data.

[0091] S2124: Fuse each initial 2D trajectory feature with the one-dimensional data converted from the remaining initial 2D trajectory features after downsampling or upsampling to generate 2D trajectory features that correspond one-to-one with each initial 2D trajectory feature, thereby obtaining a preset number of 2D trajectory features.

[0092] For example, there are two initial 2D trajectory features, denoted as B1 and B2. The one-dimensional data converted from the initial 2D trajectory feature B2 can be fused with the initial 2D trajectory feature B1 to generate a 2D trajectory feature corresponding to the initial 2D trajectory feature B1. Alternatively, the one-dimensional data converted from the initial 2D trajectory feature B1 can be fused with the initial 2D trajectory feature B2 to generate a 2D trajectory feature corresponding to the initial 2D trajectory feature B2.

[0093] S213: Perform a preset number of cascaded processing on a preset number of 2D trajectory features to obtain multiple 2D trajectory features corresponding to multiple scales. The number of multiple 2D trajectory features is equal to the sum of the preset number and the preset number of processing times.

[0094] Based on a preset number of 2D trajectory features, a further number of 2D trajectory features can be obtained through a preset number of cascaded processing steps.

[0095] For example, such as Figures 3 to 5 As shown, the preset quantity is two, the preset number of iterations is three, and the HRNet-Transformer coding network has four stages, denoted as Stage1, Stage2, Stage3, and Stage4 respectively. Stage2 yields two 2D trajectory features. Stage2 can include three Transformer Blocks, yielding three 2D trajectory features. Stage3 can include four Transformer Blocks, yielding four 2D trajectory features. Stage4 can include five Transformer Blocks, yielding five 2D trajectory features.

[0096] Optionally, a preset number of 2D trajectory features can be cascaded a preset number of times, as detailed in the following steps included in S213:

[0097] S2131: Downsample the smallest 2D trajectory feature in the current level using Patch Merging to generate spatial size features.

[0098] The spatial size feature is different from the size of the 2D trajectory features of the current stage. For example, if the current stage is Stage 1, the smallest of the two 2D trajectory features obtained from Stage 1 can be downsampled using Patch Merging to generate a 1 / 2 spatial size feature. The 1 / 2 spatial size feature is different from the size of the two 2D trajectory features obtained from Stage 1. As another example, if the current stage is Stage 2, the smallest of the three 2D trajectory features obtained from Stage 2 can be downsampled using Patch Merging to generate a 1 / 2 spatial size feature.

[0099] S2132: Send the current level's 2D trajectory features and spatial size features to the corresponding TransformerBlock for generation processing to generate the next level's 2D trajectory features, ensuring that the number of 2D trajectory features in the next level is one more than the number of 2D trajectory features in the current level.

[0100] For example, such as Figures 3 to 5 As shown, when the current stage is Stage 1, the two 2D trajectory features and the half-space size feature obtained from Stage 1 can be sent to Stage 2, generating three 2D trajectory features. Specifically, Stage 2 includes three different Transformer Blocks. The two 2D trajectory features and the half-space size feature obtained from Stage 1 can be sent to one Transformer Block respectively, resulting in three initial 2D trajectory features. Then, each initial 2D trajectory feature is fused with the one-dimensional data converted from the remaining downsampled or upsampled initial 2D trajectory features to generate 2D trajectory features that correspond one-to-one with the three initial 2D trajectory features, thus obtaining three 2D trajectory features.

[0101] Optionally, the Transformer Block processes the 2D trajectory features and half-space size features as follows: It uses three fully connected layers to process the input 2D trajectory features and half-space size features, generating a Query matrix, a Key matrix, and a Value matrix. The Swin-Transformer attention mechanism is then used to process the Query matrix, Key matrix, and Value matrix to obtain generated features. Finally, a shape transformation operation is performed on the generated features to generate the initial 2D trajectory features.

[0102] Further, when the current stage is Stage2, the 3 2d trajectory features obtained from Stage2 and the 1 / 2 spatial size feature can be sent to Stage3, and 4 2d trajectory features are generated in Stage3. When the current stage is Stage3, the 4 2d trajectory features obtained from Stage3 and the 1 / 2 spatial size feature can be sent to Stage4, and 5 2d trajectory features are generated in Stage4.

[0103] As shown in Figures 3 to 5 Stage1, two initial 2d trajectory features are fused with one-dimensional data converted from the initial 2d trajectory features after 2 times down-sampling or 2 times up-sampling, to generate 2d trajectory features corresponding to the two initial 2d trajectory features respectively. In Stage2, 3 initial 2d trajectory features are fused with one-dimensional data converted from the initial 2d trajectory features after 2 times down-sampling, 2 times up-sampling, 4 times down-sampling or 4 times up-sampling, to generate 2d trajectory features corresponding to the 3 initial 2d trajectory features respectively. In Stage3, 4 initial 2d trajectory features are fused with one-dimensional data converted from the initial 2d trajectory features after 2 times down-sampling, 2 times up-sampling, 4 times down-sampling, 4 times up-sampling, 8 times down-sampling or 8 times up-sampling, to generate 2d trajectory features corresponding to the 4 initial 2d trajectory features respectively. In Stage4, 5 initial 2d trajectory features are fused with one-dimensional data converted from the initial 2d trajectory features after 2 times down-sampling, 2 times up-sampling, 4 times down-sampling, 4 times up-sampling, 8 times down-sampling, 8 times up-sampling, 16 times down-sampling or 16 times up-sampling, to generate 2d trajectory features corresponding to the 5 initial 2d trajectory features respectively.

[0104] Wherein, 2 times down-sampling, 2 times up-sampling, 4 times down-sampling, 4 times up-sampling, 8 times down-sampling, 8 times up-sampling, 16 times down-sampling and 16 times up-sampling can be denoted as d2, u2, d4, u4, d8, u8, d16 and u16 respectively.

[0105] S220: Feature encoding of the 3d historical trajectory grid map information is performed by using a second encoding network to generate a plurality of 3d trajectory features. The second encoding network is different from the first encoding network.

[0106] The plurality of 3d trajectory features generated in this step are different in scale.

[0107] Optionally, as shown in Figure 6 The second encoding network is a 3d convolution network, which includes a BottleNeck with a 3d convolution layer, a batch processing layer and an activation function. How to use the second encoding network to perform feature encoding on the 3d historical trajectory grid map information to generate a plurality of 3d trajectory features can be referred to the following steps included in S220:

[0108] S221: Down-sampling the 3D historical trajectory grid map information in the spatial dimension by using a single 3D convolutional layer to generate a first 3D trajectory feature.

[0109] The first 3D trajectory feature can be denoted as F1.

[0110] S222: Performing a preset number of cascading processes on the first 3D trajectory feature to obtain a plurality of 3D trajectory features.

[0111] For example, the preset number is 4, and the first 3D trajectory feature is processed 4 times in a cascading manner to obtain 4 3D trajectory features, which can be denoted as F2, F3, F4 and F5.

[0112] Optionally, the first 3D trajectory feature is processed in a preset number of cascading processes to obtain a plurality of 3D trajectory features, which can be specifically referred to the following steps included in S222:

[0113] S2221: Alternately performing a preset number of cascading processes on the first 3D trajectory feature in the time dimension and the spatial dimension to obtain a plurality of 3D trajectory features.

[0114] For example, the preset number is 4, and the first 3D trajectory feature F1 is down-sampled in the spatial dimension to generate a 3D trajectory feature F2, the 3D trajectory feature F2 is down-sampled in the time dimension to generate a 3D trajectory feature F3, the 3D trajectory feature F3 is down-sampled in the spatial dimension to generate a 3D trajectory feature F4, and the 3D trajectory feature F4 is down-sampled in the time dimension to generate a 3D trajectory feature F5.

[0115] Optionally, as shown in Figure 6 the first 3D trajectory feature is alternately processed in a preset number of cascading processes in the time dimension and the spatial dimension to obtain a plurality of 3D trajectory features, which can be specifically referred to the following steps included in S2221:

[0116] S2222: Down-sampling the first 3D trajectory feature in the time dimension or the spatial dimension by using a BottleNeck to generate a next 3D trajectory feature.

[0117] For example, the first 3D trajectory feature F1 is down-sampled in the spatial dimension by using a BottleNeck to generate a 3D trajectory feature F2.

[0118] S2223: Down-sampling the current 3d trajectory feature in time dimension or space dimension by BottleNeck to generate the next 3d trajectory feature, until a plurality of 3d trajectory features are generated. Wherein, the dimension of the current 3d trajectory feature for down-sampling is different from the dimension of its previous 3d trajectory feature and its next 3d trajectory feature for down-sampling.

[0119] For example, the number of 3d trajectory features is 5, the 3d trajectory feature F2 is down-sampled in time dimension by BottleNeck to generate the 3d trajectory feature F3, the 3d trajectory feature F3 is down-sampled in space dimension by BottleNeck to generate the 3d trajectory feature F4, the 3d trajectory feature F4 is down-sampled in time dimension by BottleNeck to generate the 3d trajectory feature F5, thereby generating 5 3d trajectory features.

[0120] S230: Feature encoding of the 2d traffic grid map information by the third encoding network to generate a plurality of 2d traffic map features. The third encoding network is different from the second encoding network and the first encoding network.

[0121] The plurality of 2d traffic map features generated in this step are different in scale.

[0122] Optionally, as shown in Figure 7 The third encoding network is a 2d convolution network, including BottleNeck with 2d convolution layer, batch processing layer and activation function. How to use the third encoding network to encode the 2d traffic grid map information to generate a plurality of 2d traffic map features can be referred to the following steps included in S230:

[0123] S231: Down-sampling the 2d traffic grid map information in space dimension and feature dimension by a single 2d convolution layer to generate the first 2d traffic map feature.

[0124] The first 2d traffic map feature can be denoted as A1.

[0125] S232: The first 2d traffic map feature is processed by a preset number of cascading to obtain a plurality of 2d traffic map features.

[0126] For example, the preset number is 4, the 2d traffic map feature A2 is obtained from the first 2d traffic map feature A1, the 2d traffic map feature A3 is obtained from the 2d traffic map feature A2, the 2d traffic map feature A4 is obtained from the 2d traffic map feature A3, and the 2d traffic map feature A5 is obtained from the 2d traffic map feature A4.

[0127] Optionally, the BottleNeck has a plurality of 2d convolution layers.

[0128] Optionally, the first 2d traffic map feature is subjected to preset number of cascading processing to obtain a plurality of 2d traffic map features, which can refer to the following steps included in S232 for details:

[0129] S2321: The first 2d traffic map feature is down-sampled by BottleNeck to generate the next 2d traffic map feature.

[0130] For example, 2d traffic map feature A2 can be obtained from the first 2d traffic map feature A1 by BottleNeck, 2d traffic map feature A3 can be obtained from 2d traffic map feature A2 by BottleNeck, 2d traffic map feature A4 can be obtained from 2d traffic map feature A3 by BottleNeck, and 2d traffic map feature A5 can be obtained from 2d traffic map feature A4 by BottleNeck.

[0131] S300: The plurality of 2d trajectory features, the plurality of 3d trajectory features, and the plurality of 2d traffic map features are spliced in corresponding scales, and after splicing, multi-scale fusion is performed to generate a fusion feature.

[0132] The fusion feature can be used for decoding to obtain prediction information of the trajectory of the traffic participant, and decoding the fusion feature can improve the prediction effect of the trajectory of the traffic participant.

[0133] Optionally, how to splice the plurality of 2d trajectory features, the plurality of 3d trajectory features, and the plurality of 2d traffic map features in corresponding scales, and after splicing, multi-scale fusion is performed to generate a fusion feature, which can refer to the following steps included in S300 for details:

[0134] S310: The plurality of 2d trajectory features, the plurality of 3d trajectory features, and the plurality of 2d traffic map features are spliced in corresponding sizes to obtain a plurality of scale synthesis features.

[0135] For example, the 2d trajectory features can be C1, C2, C3, C4, and C5, the 3d trajectory features can be F1, F2, F3, and F4, and the 2d traffic map features can be F5, A1, A2, A3, A4, and A5. Among them, C1, F1, and A1 correspond in scale, and C1, F1, and A1 can be spliced. C2, F2, and A2 correspond in scale, and C2, F2, and A2 can be spliced. C3, F3, and A3 correspond in scale, and C3, F3, and A3 can be spliced. C4, F4, and A4 correspond in scale, and C4, F4, and A4 can be spliced. C5, F5, and A5 correspond in scale, and C5, F5, and A5 can be spliced. Splicing each scale can obtain a scale synthesis feature, thereby obtaining five scale synthesis features.

[0136] Optionally, when the scales of the corresponding features are different, for example, the scales of C1, F1 and A1 are different, a difference algorithm can be used to keep the features of different scales at one scale to facilitate the splicing of the corresponding scale.

[0137] S320: The multiple scale synthesis features are fused by using the BiFPN multi-scale feature fusion network to generate processing, to obtain a fused scale synthesis feature. The multiple scale synthesis features and the fused scale synthesis feature constitute the fused feature.

[0138] For example, the five scale synthesis features are fused by using the BiFPN multi-scale feature fusion network to generate processing, to obtain a fused scale synthesis feature. The five scale synthesis features and the fused scale synthesis feature can constitute the fused feature.

[0139] S400: The fused feature is decoded to generate a decoded feature, and the decoded feature is used to predict the trajectories of all traffic participants on the map corresponding to the preset map range.

[0140] The fused feature contains information of all traffic participants in multiple scales and multiple directions in the preset map range. Decoding the fused feature can improve the effect of predicting the trajectories of all traffic participants on the map corresponding to the preset map range.

[0141] Optionally, the fused feature is decoded to generate a decoded feature, and the decoded feature is used to predict the trajectories of all traffic participants on the map corresponding to the preset map range. For details, refer to the following steps included in S400:

[0142] S410: The fused feature is decoded by using a 3d convolution network to generate a decoded feature.

[0143] Optionally, the fused feature is decoded by using a 3d convolution network to generate a decoded feature. For details, refer to the following steps included in S410:

[0144] S411: The fused feature is unfolded in the time dimension by using the 3d convolution network, and a time feature of a predicted time step is output.

[0145] Specifically, the multiple scale synthesis features and the fused scale synthesis feature are sent to a decoder composed of a 3d convolution network, the features processed by the 3d convolution layer are fused in a multi-scale feature fusion manner, and the time dimension of the generated features is unfolded step by step.

[0146] Optionally, S420: The decoded feature is sent to a shared layer for prediction processing to realize trajectory prediction of all traffic participants on the map corresponding to the preset map range.

[0147] Specifically, the decoded features are sent to the shared layer, and the future occupancy map of all traffic participants and the position offset of the occupancy position at each time step are output, and the prediction of the future occupancy of all traffic participants is completed, that is, the prediction of the future trajectory of all traffic participants is completed. For example, the future occupancy map of the vehicle and the position offset of the occupancy position at each time step are output, and the prediction of the future occupancy of the vehicle is completed, that is, the prediction of the future trajectory of the vehicle is completed.

[0148] Optionally, the decoded features are sent to the shared layer for prediction processing. For details, refer to the following steps included in S420:

[0149] S421: The time features of each step in the prediction time step are sent to the shared layer for prediction processing in a time dimension index.

[0150] By sending the time features of each step in the prediction time step to the shared layer for prediction processing in a time dimension index, the prediction of the future trajectory of the traffic participant can be realized.

[0151] As shown in Figure 8 The intelligent vehicle 100 described in the intelligent vehicle embodiment of the present application includes a processor 110, a memory 120, and a detection component 130. The detection component 130 is configured to detect historical trajectory grid information of all traffic participants within a preset map range, an aerial view within the preset map range, and traffic feature information within the preset map range. The processor 110 is coupled with the detection component 130. The memory 120 stores a computer program. The processor 110 is configured to execute the computer program to implement the above method.

[0152] The processor 110 can be configured to control the operation of the intelligent vehicle.

[0153] Optionally, the detection component 130 can include a laser radar and a camera.

[0154] The memory 120 can be configured to store the computer program, which can be RAM, ROM, or other types of storage devices. Specifically, the memory 120 can include one or more computer-readable storage media, which can be non-transitory. The memory 120 can also include a high-speed random access memory and a non-volatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0155] As shown in Figure 9 The computer-readable storage medium 200 described in the computer-readable storage medium embodiment of the present application stores a computer program, which can be executed by a processor to implement the above method.

[0156] If the above method is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions / computer programs for making a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media and electronic devices with the above storage medium, such as computers, mobile phones, notebook computers, tablet computers, cameras, etc.

[0157] The description of the execution process of the program data in the computer-readable storage medium 200 can refer to the description in the above-mentioned traffic participant trajectory prediction method embodiment, which will not be repeated here.

[0158] In summary, in the 2d trajectory feature, the timing feature can be fused into the feature dimension in the present embodiment. By fusing multiple scales of 2d trajectory features, sufficient rich timing feature information can be captured, and the time sequence of the predicted traffic participant can be extended. Multiple traffic participants are predicted simultaneously within a preset map range, and the prediction effect is improved. By obtaining the contour size of the traffic participant, the vehicle occupancy range can be integrated into the present embodiment when predicting the trajectory of the traffic participant, so as to avoid the collision of the traffic participants, such as the occurrence of vehicle collision.

[0159] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for predicting trajectories of traffic participants, characterized in that, The method comprises the following steps: acquiring 2D historical trajectory grid information, 3D historical trajectory grid information of all traffic participants in a preset map range, and 2D traffic grid information in the preset map range, the 2D historical trajectory grid information and the 3D historical trajectory grid information being obtained based on historical trajectory grid information, the historical trajectory grid information comprising the contour size, heading angle, historical trajectory and current position of the traffic participant, the traffic participant comprising a vehicle, a pedestrian and a non-motor vehicle; respectively performing multi-scale feature coding on the 2D historical trajectory grid information, the 3D historical trajectory grid information and the 2D traffic grid information to correspondingly generate a plurality of 2D trajectory features corresponding to a plurality of scales, a plurality of 3D trajectory features corresponding to the plurality of scales and a plurality of 2D traffic map features corresponding to the plurality of scales; splicing the plurality of 2D trajectory features, the plurality of 3D trajectory features and the plurality of 2D traffic map features in corresponding scales, and performing fusion of the plurality of scales through a BIFPN multi-scale feature fusion network after splicing to generate fusion features; performing feature decoding on the fusion features to generate decoding features, and using the decoding features to perform trajectory prediction of the all traffic participants on a corresponding map of the preset map range; respectively performing feature coding on the 2D historical trajectory grid information, the 3D historical trajectory grid information and the 2D traffic grid information to correspondingly generate a plurality of 2D trajectory features corresponding to a plurality of scales, a plurality of 3D trajectory features corresponding to the plurality of scales and a plurality of 2D traffic map features corresponding to the plurality of scales, comprising: performing feature coding on the 2D historical trajectory grid information by using a first coding network to generate the plurality of 2D trajectory features, the first coding network being an HRNet-Transformer coding network obtained by fusing at least an HRNet model and a Transformer model; performing feature coding on the 3D historical trajectory grid information by using a second coding network to generate the plurality of 3D trajectory features, comprising: performing spatial dimension downsampling on the 3D historical trajectory grid information by using a single 3D convolution layer to generate a first 3D trajectory feature, and performing cascade processing on the first 3D trajectory feature for a preset number of times to obtain the plurality of 3D trajectory features, comprising: alternately performing cascade processing on the first 3D trajectory feature for a preset number of times in the time dimension and the space dimension to obtain the plurality of 3D trajectory features; the second coding network is different from the first coding network, and the second coding network is a 3D convolution network; performing feature coding on the 2D traffic grid information by using a third coding network to generate the plurality of 2D traffic map features; the third coding network is different from the second coding network and the first coding network, and the third coding network is a 2D convolution network.

2. The method according to claim 1, characterized in that, The first encoding network is used to encode the 2D historical trajectory grid information to generate the plurality of 2D trajectory features, including: The 2D historical trajectory grid information is integrated in a preset number of different Patch sizes to obtain a plurality of Patch block features corresponding to the different sizes; The plurality of Patch block features are sent to corresponding Transformer Blocks for generation processing to generate the plurality of 2D trajectory features. The plurality of 2D trajectory features are processed in a preset number of cascading processes to obtain the plurality of 2D trajectory features corresponding to the plurality of scales, and the number of the plurality of 2D trajectory features is equal to the sum of the preset number and the preset number of times.

3. The method of claim 2, wherein the integrating the 2D historical trajectory grid information in a preset number of different Patch sizes comprises: using two Patch Embedding layers generated by different Patches to integrate the 2D historical trajectory grid information.

4. The method of claim 2, wherein the processing the plurality of 2D trajectory features in a preset number of cascading processes comprises: down-sampling the smallest one of the 2D trajectory features in the current stage in a Patch Merging manner to generate a spatial size feature; sending the 2D trajectory features in the current stage and the spatial size feature to corresponding Transformer Blocks for the generation processing to generate the 2D trajectory features in the next stage, and making the number of the 2D trajectory features in the next stage one more than the number of the 2D trajectory features in the current stage.

5. The method of claim 2, wherein the sending the plurality of Patch block features to corresponding Transformer Blocks for generation processing to generate the plurality of 2D trajectory features comprises: processing the plurality of Patch block features using three fully connected layers to generate a Query matrix, a Key matrix and a Value matrix; processing the Query matrix, the Key matrix and the Value matrix using a Swin-Transformer attention mechanism to obtain generated features, and performing shape transformation on the generated features to generate a plurality of initial 2D trajectory features; down-sampling or up-sampling each of the initial 2D trajectory features, and converting the initial 2D trajectory features subjected to the down-sampling or up-sampling into one-dimensional data through flattening processing; fusing each of the initial 2D trajectory features with the one-dimensional data converted from the remaining initial 2D trajectory features subjected to the down-sampling or up-sampling to generate the 2D trajectory features corresponding to each of the initial 2D trajectory features, thereby obtaining the plurality of 2D trajectory features. ​ ​ ​ 6. The method of claim 1, wherein, the second encoding network comprises a BottleNeck having a 3d convolution layer, a batch layer, and an activation function.

7. The method of claim 1, wherein, the cascading processing of the first 3d trajectory feature in the time dimension and the space dimension for a preset number of times to obtain the plurality of 3d trajectory features comprises: using a BottleNeck to down-sample the first 3d trajectory feature in the time dimension or the space dimension to generate a next 3d trajectory feature; using the BottleNeck to down-sample the current 3d trajectory feature in the time dimension or the space dimension to generate a next 3d trajectory feature until the plurality of 3d trajectory features are generated; wherein the dimension of the current 3d trajectory feature for down-sampling is different from the dimensions of the previous 3d trajectory feature and the next 3d trajectory feature for down-sampling.

8. The method of claim 1, wherein, the third encoding network is a 2d convolution network comprising a BottleNeck having a 2d convolution layer, a batch layer, and an activation function; the feature encoding of the 2d traffic grid map information using the third encoding network to generate the plurality of 2d traffic map features comprises: using a single 2d convolution layer to down-sample the 2d traffic grid map information in the spatial dimension and the feature dimension to generate a first 2d traffic map feature; cascading processing of the first 2d traffic map feature for a preset number of times to obtain the plurality of 2d traffic map features.

9. The method of claim 8, wherein, the cascading processing of the first 2d traffic map feature for a preset number of times to obtain the plurality of 2d traffic map features comprises: using the BottleNeck to down-sample the first 2d traffic map feature to generate a next 2d traffic map feature.

10. The method of claim 1, wherein, the splicing of the plurality of 2d trajectory features, the plurality of 3d trajectory features, and the plurality of 2d traffic map features in corresponding scales, and the fusion of the plurality of scales after splicing to generate a fusion feature comprises: splicing the plurality of 2d trajectory features, the plurality of 3d trajectory features, and the plurality of 2d traffic map features in corresponding sizes to obtain a plurality of scale synthesis features; using a BiFPN multi-scale feature fusion network to fuse the plurality of scale synthesis features to generate a fusion scale synthesis feature; wherein the plurality of scale synthesis features and the fusion scale synthesis feature constitute the fusion feature.

11. The method of claim 1, wherein, the feature decoding of the fusion feature to generate a decoded feature, and the trajectory prediction of all traffic participants on the corresponding map of the preset map range using the decoded feature comprises: perform feature decoding on the fusion feature by using a 3D convolution network to generate the decoded feature; send the decoded feature to a shared layer for prediction processing to realize trajectory prediction of all the traffic participants on a map corresponding to the preset map range.

12. The method of claim 11, wherein the feature decoding on the fusion feature by using a 3D convolution network to generate the decoded feature comprises: expanding the fusion feature in a time dimension by using the 3D convolution network and outputting time features of a prediction time step; the sending of the decoded feature to a shared layer for prediction processing comprises: gradually sending the time features of each of the prediction time steps in a time dimension index to the shared layer for prediction processing.

13. The method of claim 1, wherein, before the obtaining of 2D historical trajectory raster map information, 3D historical trajectory raster map information of all the traffic participants in a preset map range, and 2D traffic raster map information in the preset map range, comprising: obtaining historical trajectory raster map information of all the traffic participants in the preset map range, an aerial view in the preset map range, and traffic feature information in the preset map range; the traffic feature information comprises road information and traffic signal information; calculating speeds of all the traffic participants according to the historical trajectory raster map information and a time interval; performing a splicing operation on all the traffic participants in a time dimension to generate 3D historical trajectory raster map information comprising length, width, time dimension, and feature dimension; compressing a time dimension of the 3D historical trajectory raster map information to a feature dimension to splice and generate the 2D historical trajectory raster map information; splicing the aerial view and the traffic feature information in a feature dimension to generate the 2D traffic raster map information.

14. An intelligent automobile, characterized by a detection component for detecting historical trajectory raster map information of all the traffic participants in a preset map range, an aerial view in the preset map range, and traffic feature information in the preset map range, the processor being coupled with the detection component, the memory storing a computer program, and the processor being configured to execute the computer program to implement the method of any one of claims 1-13.

15. A computer readable storage medium, characterized in that, a computer program stored in the memory and executable by the processor to implement the method of any one of claims 1-13.

Images