Trajectory prediction system based on convolution and self-attention
By using a trajectory prediction system based on convolution and self-attention, we have solved the common-sense errors and stability issues in trajectory prediction of end-to-end models, and achieved more reasonable and safer trajectory prediction, which can meet the actual needs of autonomous driving scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG POLICE COLLEGE (GUANGDONG PROVINCIAL PUBLIC SECURITY JUDICIAL MANAGEMENT CADRE COLLEGE)
- Filing Date
- 2026-01-28
- Publication Date
- 2026-05-01
AI Technical Summary
Existing end-to-end trajectory prediction systems are prone to making basic errors when faced with simple traffic rule tasks. They lack explicit correlation with the basic constraints of the road environment, resulting in insufficient rationality and safety of trajectory prediction. The training datasets are of single dimension and have insufficient scene coverage, leading to poor model training stability and failure to effectively adapt to vehicle kinematic characteristics, resulting in large trajectory prediction errors and abnormal fluctuations.
A trajectory prediction system based on convolution and self-attention is adopted. Through a network architecture consisting of DenseNet layers, self-attention coding layers, fully connected coding layers, and decoding output layers, and trained on a simulation dataset, the system uses polar coordinate trajectories to describe vehicle motion. A specific loss function and optimizer are designed to optimize feature extraction and fusion, ensuring model stability and generalization ability.
It improves the rationality and safety of trajectory prediction, reduces the average error, meets real-time requirements, and the model shows good generalization ability and prediction accuracy in multiple scenarios, adapting to the actual needs of autonomous driving scenarios.
Smart Images
Figure CN121963146A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of trajectory prediction technology, specifically relating to a trajectory prediction system based on convolution and self-attention. Background Technology
[0002] Autonomous driving technology, a product of the integration of artificial intelligence and the automotive industry, is gradually moving from the laboratory to real life, leading the transformation of future transportation. Its basic principle is to enable vehicles to drive safely and efficiently without human driver intervention by integrating multiple sensors, high-performance computing platforms, and advanced artificial intelligence algorithms. Key technologies include sensor fusion, path planning, and decision control. With improvements in sensor technology, algorithm optimization, and data processing capabilities, the performance and safety of autonomous driving systems are continuously improving.
[0003] Existing autonomous driving systems can be mainly divided into three major technical paradigms: modular system architecture, multi-task learning architecture, and end-to-end system architecture. Currently, the mainstream architecture in academia is the end-to-end system. This method borrows the "behavioral reflex" characteristics of human driving, eliminating the need for manually predefined rules and explicit interfaces between modules. It treats the pipeline processes of perception, decision-making, and control as a holistic learning task for optimization, directly mapping trajectory points or control commands from onboard sensor signals. The biggest advantage of the end-to-end model is that it eliminates the performance bottlenecks caused by manual predefinition and can uncover a large amount of hidden information. However, it is worth noting that while end-to-end models can often complete highly complex driving tasks through training, they make common-sense, basic errors when faced with simple tasks like "stop at a red light and go at a green light." This is one of the biggest obstacles limiting the engineering application of the end-to-end method.
[0004] The application of autonomous driving technology in the transportation industry has reduced many traffic accidents caused by human error, but it has also increased risks due to the immaturity and unpredictability of the technology. Technically, autonomous driving technology still has significant room for improvement and is not yet sufficient to support accurate judgments and evasive maneuvers. Furthermore, the algorithm design of autonomous driving often lacks an understanding of traffic rules, sometimes ignoring traffic rules and outputting unreasonable driving behaviors in order to reach the destination, leading to traffic disruptions, violations, and even accidents. In particular, the end-to-end methods popular in recent years, represented by imitation learning and reinforcement learning, suffer from weak interpretability and unreliable performance limits due to their "black box" architecture, potentially leading to common-sense errors and compromising safety.
[0005] To enhance the interpretability of end-to-end models and help humans understand the mechanisms behind model decisions, end-to-end models also incorporate intermediate auxiliary outputs, including attention visualization, semantic segmentation, occupancy prediction, cost graphs, text explanations, and interpretable feature maps. These outputs provide auxiliary tasks for the end-to-end architecture, helping to explain model behavior and assisting in troubleshooting failures. Complementing modular and end-to-end methods—that is, leveraging the strong interpretability of modular methods and combining them with end-to-end architectures to create a modular end-to-end learning framework—is one of the new approaches in the development of autonomous driving technology in recent years.
[0006] However, (1) there is a lack of explicit correlation with the basic constraints of the road environment. The input data does not systematically integrate key traffic information such as collision boxes and road markings, which makes it difficult for the model to form an effective understanding of the logic of traffic scenarios. It is easy to make common-sense errors such as ignoring traffic rules and collision risks. The rationality and safety of trajectory prediction are insufficient. (2) The Cartesian coordinate system is used to directly output trajectory points, which does not fit the time continuity characteristics of speed and steering in vehicle kinematics. Furthermore, the range of trajectory parameter values is not effectively limited, which can easily generate infeasible trajectories or abnormal speed fluctuations, and cannot adapt to the actual driving needs of autonomous driving. (3) The training dataset has a single dimension and insufficient scene coverage. It is not trained with high-quality data from multiple scenarios and multiple frames. Moreover, the optimizer and loss function design lacks specificity. It does not apply special penalties to situations where the steering angle and speed change are large, resulting in poor model training stability, weak generalization ability, and difficulty in controlling the prediction error at a low level.
[0007] To address these issues, we propose a trajectory prediction system based on convolution and self-attention. Summary of the Invention
[0008] The purpose of this invention is to provide a trajectory prediction system based on convolution and self-attention to solve the problems mentioned in the background art.
[0009] To achieve the above objectives, the present invention provides the following technical solution: a trajectory prediction system based on convolution and self-attention, comprising: an overall network architecture, a training method, and an inference output algorithm; The overall network architecture takes the road grid map, vehicle motion state and target information as input, and processes them sequentially through the DenseNet layer, self-attention coding layer, fully connected coding layer and decoding output layer to output the predicted trajectory; The training method trains the network model based on a simulation dataset, calculates the loss value through a specific loss function, and updates the network weight parameters through backpropagation. The inference output algorithm obtains the final trajectory output through polar coordinate trajectory difference calculation, coordinate transformation and multi-step cyclic prediction. The module is used for driving trajectory prediction in autonomous driving scenarios.
[0010] Preferably, the road grid map is a three-channel bird's-eye view grid map with a resolution of 256×256. The three channels are the collision boxes of surrounding traffic participants, road markings, and road surface maps, respectively. Each channel corresponds to a different numerical label: the collision box value of the main vehicle is 0.4, and the collision box value of other traffic participants is 1; the solid line marking value is 0.6, and the dashed line and pedestrian crossing marking value is 0.4; the road surface value within the intersection is 0.4, and the road surface value of other surfaces is 1.
[0011] Preferably, the vehicle motion state includes the current vehicle speed and the polar coordinate driving trajectory information of the previous 6 frames. The target information includes the target vehicle speed and the relative coordinates of the target point. The current vehicle speed, the target vehicle speed, and the relative coordinates of the target point form a four-dimensional vector. The vehicle speed is in m / s and the coordinate length is in m. The target speed is set to 0 when encountering a red light.
[0012] Preferably, the first 6 frames of polar coordinate driving trajectory information form a 6×3 matrix, which includes three elements: steering angle, driving distance, and orientation angle. The unit quantity of steering angle is 15°, the unit quantity of driving distance is 1.5m, and the unit quantity of orientation angle is 30°. The trajectory information value is controlled within the range of [0,1], and when the driving distance is less than 0.1m, the values of steering angle and orientation angle are 0.
[0013] Preferably, the DenseNet layer replaces the pooling layers in the original structure with linear mappings of 1×1 convolutional kernels, and replaces all ReLU+BatchNorm layers with GELU+LayerNorm layers; The DenseNet layer receives two types of grid image inputs: a 6-frame continuous collision box grid image with 6 channels and a current frame collision box grid image, and a road marking and road surface merged grid image with 3 channels. After processing by the DenseNet network with the same structure, the two types of inputs are spliced into a tensor of size b×128×512, where b is the batch processing size.
[0014] Preferably, the self-attention encoding layer adopts a 6-layer stacked Transformer Encoder architecture, with both input and output being tensors of size b×128×512, and its positional encoding is generated by the following formula: , Where pos is the sequence position of the feature vector, and i is the dimensional position of the feature vector.
[0015] Preferably, the fully connected coding layer transposes the grid pattern encoding output by the self-attention coding layer, reduces the dimensionality through one-dimensional convolution, and calculates a feature vector of size b×768 through MLP; the historical trajectory information is processed by MLP layer, one-dimensional convolution and GELU layer to obtain a feature vector of size b×256; the two feature vectors are concatenated to obtain a coding vector of size b×1024.
[0016] Preferably, the decoding output layer combines the vehicle trajectory information from the first 6 frames and the predicted trajectory with all zeros replaced in the last 3 frames to form a b×9×3 historical trajectory tensor. This tensor is then processed by an MLP layer and a one-dimensional convolution to obtain a b×32 vector. This vector is concatenated with the current frame's velocity information and the target position dimension to obtain a b×36 feature vector. After dimensionality increase and LayerNorm processing by an MLP layer, this feature vector is used as the input to the GRU. The output vector of the fully connected coding layer is processed by LayerNorm and used as the hidden layer of the GRU. The GRU output is processed by the Head layer to calculate the trajectory difference. This process is repeated 4 times to obtain the final trajectory output of size b×4×3. The calculation formula for each step in the loop is as follows:
[0017] The design layer has an input of A and an output of O; where A is a tensor of size b×3×4.
[0018] Preferably, the loss function calculation process includes: converting the normalized trajectory difference matrix into polar coordinate trajectory information.
[0019] Then, convert it to the coordinates of the trajectory point in the Cartesian coordinate system using the following formula:
[0020] The loss value is finally calculated using the following formula:
[0021] in Standard deviation The increment factor is added to apply a larger penalty when the turning angle and speed increase, so that the predicted trajectory obtained by training is smooth and stable. After the calculation is completed, backpropagation is performed based on the loss value and the learning rate to update the weight parameters of the network.
[0022] Preferably, the simulation dataset is collected through the Carla simulation simulator and includes a collision box grid map, a road marking grid map, a road grid map, target speed, current speed, target position, current position, and vehicle orientation angle. The interval between consecutive frames in the acquisition scene is 0.5 seconds, and during training, information from 6 consecutive frames is extracted and processed into polar coordinate driving trajectory information.
[0023] Compared with the prior art, the beneficial effects of the present invention are: (1) By taking the road grid map containing information such as collision boxes and road markings and the vehicle motion state as input, and combining the self-attention mechanism with road environment constraints, the polar coordinate trajectory description method that conforms to the vehicle kinematics characteristics is adopted to limit the range of trajectory parameter values, effectively avoid common sense errors and abnormal trajectories, meet traffic rules and driving safety requirements, and form a trajectory prediction with outstanding rationality and safety. (2) Based on 16,000 frames of multi-dimensional training data (including collision box, velocity, position and other information, with a continuous frame interval of 0.5 seconds) collected by the Carla simulation simulator, the average error was reduced in the validation set after optimized training, and the error in the intersection turning scene was reduced; a lightweight architecture design was adopted, with an average calculation time of 0.1095 seconds per frame and a video memory usage of about 2G, which meets the requirements of real-time performance and engineering embedding. (3) Through the collaborative design of DenseNet layer, 6 stacked TransformerEncoder layers, fully connected encoding layer and GRU decoding layer, the feature extraction and fusion effect is optimized; during training, information from 6 consecutive frames is extracted and processed into polar coordinate driving trajectory information, and combined with Adam optimizer and adaptive learning rate adjustment strategy, combined with loss function containing incremental factor, to ensure stable model training and strong generalization ability. Attached Figure Description
[0024] Figure 1 This diagram illustrates the changes in position and orientation of the vehicle as it travels according to the invention. Figure 2 This is a schematic diagram of the overall logical framework of the prediction network of this invention; Figure 3 This is a schematic diagram of the DenseNet layer of the present invention; Figure 4 This is a schematic diagram of the architecture of the self-attention coding layer of the present invention; Figure 5 This is a schematic diagram of the fully connected coding layer of the present invention; Figure 6 This is a schematic diagram of the decoding output layer of the present invention; Figure 7 This is a diagram of the three-channel merged grid of the present invention; Figure 8 The grille of the present invention Figure 3 Channel splitting diagram; Figure 9 The following are schematic diagrams of the collision box of the present invention; Figure 10 This is a schematic diagram of the road markings and road surface of the present invention; Figure 11 This is a schematic diagram of the training loss variation curve of the present invention; Figure 12This is a schematic diagram of the average error curve for the final verification of this invention; Figure 13 This is a schematic diagram illustrating a prediction example of the present invention. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] Example: Please refer to Figure 1-13 This invention provides a technical solution: a trajectory prediction system based on convolution and self-attention, comprising: Overall network architecture: The overall network architecture takes the road grid map, vehicle motion state and target information as input, and processes them sequentially through the DenseNet layer, self-attention coding layer, fully connected coding layer and decoding output layer to output the predicted trajectory. Training methods: The training method trains the network model based on a simulation dataset, calculates the loss value through a specific loss function, and updates the network weight parameters through backpropagation. Inference output algorithm: The inference output algorithm obtains the final trajectory output through polar coordinate trajectory difference calculation, coordinate transformation and multi-step cyclic prediction. The module is used for driving trajectory prediction in autonomous driving scenarios.
[0027] The designed network model is trained by using road grid maps, vehicle motion states, and target information, and the trained model is used to predict driving trajectories. Traditional trajectory prediction networks typically output coordinates in a Cartesian coordinate system. However, this design considers the vehicle's kinematic characteristics and uses polar coordinate trajectories to describe the vehicle's motion. The following definitions are given: (1) Polar coordinate driving trajectory information: indicates the changes in position and orientation caused by the vehicle's movement (e.g., Figure 1 As shown), with the vehicle's position and orientation in the previous frame as the center reference, the polar coordinate driving trajectory in each frame contains three elements: steering angle. Driving distance and orientation angle .
[0028] (2) Polar coordinate driving trajectory difference: The difference between the polar coordinate driving trajectory information of the current frame and the polar coordinate driving trajectory information of the previous frame, which includes three elements: the change in steering angle. Change in travel distance Change in orientation angle .
[0029] Since the speed and steering of a vehicle are continuous over time, using polar coordinates centered on the vehicle to describe its motion changes is more consistent with the description of vehicle motion control.
[0030] Before network training, a simulation training dataset was first collected using a simulation simulator, including the collision box grid pattern, road marking grid pattern, road grid pattern, target speed, current speed, target position, current position, and vehicle heading angle. The collected grid patterns are shown below. Figure 2 As shown in the figure, during each training session, the current position and vehicle orientation angle information of 6 consecutive frames are extracted from the dataset and processed into 6 frames of polar coordinate driving trajectory information centered on the current frame position and orientation of the vehicle.
[0031] The overall logical framework of the prediction network is as follows: Figure 2 (As shown).
[0032] Specific steps: Input data format, the input grid map information resolution is a three-channel bird's-eye view grid map of [256×256], where the three channels are the collision boxes of surrounding traffic participants, road markings and road surface map, respectively, and the different values of each channel are specified in Table 1 below.
[0033] Table 1 shows the specifications for different values under each channel.
[0034] The speed and target information consist of a four-dimensional vector of information: the vehicle's current speed, the vehicle's target speed, and the target point's relative coordinates (Cartesian coordinate system). The speed is measured in m / s, and the coordinate length is measured in meters. Furthermore, it is stipulated that the target speed must be set to 0 when encountering a red light, in order to train the vehicle's trajectory generation when encountering a red light at an intersection.
[0035] The historical trajectory uses the polar coordinate driving trajectory information of the previous 6 frames to form a matrix of size [6×3]. The unit value of the steering angle is 15°, the unit value of the driving distance is 1.5m, and the unit value of the orientation angle is 30°. This unit value controls the value of the trajectory information within the range of [0,1]. At the same time, in order to prevent the angle values of the driving trajectory information from fluctuating greatly when the driving distance is close to 0, the value of the steering angle and orientation angle is set to 0 when the driving distance is less than 0.1m.
[0036] DenseNet layer: The structure of the DenseNet layer is as follows (see below) Figure 3As shown in the diagram, compared to the original structure, the final pooling layer is changed to a linear mapping of a 1×1 convolutional kernel. This layer mainly performs word embedding for the Transformer encoding layer. At the same time, all ReLU+BatchNorm layers in the network structure are replaced with GELU+LayerNorm layers to avoid unstable training results in sequence prediction training. The grid image input is divided into two categories: 6 consecutive frames of collision box grid image input (6 channels), and the collision box, road marking and road surface grid images of the current frame (merged into 3 channels). The two types of grid image data are respectively input into two DenseNet networks with the same structure. The output results of the two networks will be concatenated into a tensor of size [b×128×512], where b represents the batch size.
[0037] Self-attention encoding layer: The architecture of the self-attention encoding layer is as follows (see below) Figure 4 As shown), it uses the classic Transformer Encoder architecture, stacking a total of 6 encoder layers. Both the input and output are tensors of size [b×128×512]. That is, it performs self-attention encoding calculation on a batch of 128 feature vector sequences of dimension 512. The positional encoding is generated by the following formula:
[0038] Where pos is the sequence position of the feature vector, and i is the dimensional position of the feature vector.
[0039] Fully connected coding layer: The architecture of the fully connected coding layer is as follows: Figure 5 As shown, the grid image output from the self-attention encoding layer is first transposed, then input into a one-dimensional convolution to reduce its dimensionality, and then processed through an MLP layer to obtain a feature vector of size [b×768]. At the same time, the historical trajectory information is processed similarly to obtain a feature vector of size [b×256]. Finally, the two feature vectors are concatenated to obtain an encoding vector of size [b×1024] as the output.
[0040] Decoding output layer: The structure of the decoding output layer is as follows: Figure 6As shown, the vehicle trajectory information of the first 6 frames and the vehicle trajectory of the last 3 frames are first combined to form a historical trajectory information tensor of size [b×9×3]. Since the trajectory of the last 3 frames needs to be predicted, a tensor of all zeros is used to replace it. The historical trajectory tensor is then increased to size [b×9×16] through an MLP layer, compressed to size [b×128] through one-dimensional convolution, and then reduced to size [b×32] through an MLP layer again. The velocity information of the current frame (including the current velocity and the target velocity), the target position, and the processed historical trajectory vector are concatenated to obtain a feature vector of size [b×36]. The input tensor is first divided into [b×512] vectors by an MLP layer, and then passed through LayerNorm as input to the GRU. The output vector of the aforementioned fully connected coding layer is then passed through LayerNorm as a hidden layer of the GRU. After one step of computation, the GRU obtains a new hidden layer output, which is then passed to the Head layer to obtain the trajectory difference output. The output computation layer in the Head layer first divides the input tensor from [b×12] to [b×3×4], and then obtains the trajectory difference output of size [b×3] through matrix operations. The input of the computation layer is designed as follows: The output is Where A is a tensor of size b×3×4, and the calculation formula for each step in the loop is as follows:
[0041] The formula maps the element values of the obtained trajectory difference matrix to the range of [-1, 1]. The output result is then added to the vehicle trajectory information of the previous frame to obtain the predicted vehicle trajectory for the next frame. The historical trajectory tensor is then refreshed. This process is repeated for a total of 4 steps (or 4 times, depending on the situation) to obtain the 4-step predicted trajectory difference. The 4-step predicted values are then combined into a tensor of size [b×4×3], which is the final output of the network.
[0042] Loss calculation: After a series of calculations, the network model will return a normalized trajectory difference matrix. This matrix needs to be converted into polar coordinate trajectory information using the following formula:
[0043] Then, convert it to the coordinates of the trajectory point in the Cartesian coordinate system using the following formula:
[0044] The loss value is finally calculated using the following formula:
[0045] in Standard deviation The increment factor is added mainly to apply a larger penalty when the turning angle and speed change significantly, so that the predicted trajectory obtained from training is smoother and more stable. After the calculation is completed, backpropagation is performed based on the loss value and the learning rate to update the weight parameters of the network.
[0046] Experimental verification: Training environment, hardware environment: The graphics card used is an NVIDIA GeForce RTX 3090, and the CPU is an AMD EPYC 7H12 64-Core Processor.
[0047] Software environment: The operating system is Ubuntu 22.04, and the network framework is built and trained using Python 3.7.13 + PyTorch 1.11.
[0048] Training data preparation: The training data was collected using the Carla simulation simulator, and the results of the grid pattern data collection are shown below. Figure 7-10 (As shown).
[0049] 2014 frames were run continuously in the same scene (since training requires information from historical and future frames, some redundant frames need to be collected, of which 2000 frames are available for training), with an interval of 0.5 seconds between each frame, and data from a total of 8 scenes were collected.
[0050] Training process: Training was performed using 2000 frames of simulated data collected continuously over 8 rounds. The initial learning rate was set to 0.0001, with 2000 training iterations per round and a batch size of 10. The learning rate was reduced to 0.5 times after every 4 rounds of training. The Adam optimizer was used. The training loss variation curve is shown below. Figure 11 (As shown).
[0051] Effect verification: The verification was performed using a simulation dataset of 4000 frames continuously acquired in another scenario. The error of each frame was calculated using the root mean square error algorithm, as follows:
[0052] Where x and y represent the coordinates of the trajectory point in the Cartesian coordinate system, the algorithm's error indicator is the average error distance (in meters) between the predicted trajectory point and the labeled trajectory point. Simultaneously, the output trajectory point position obtained by the model is limited. During calculation, the predicted steering angle for each frame ranges from [-30°, 30°], the driving distance ranges from [0, 5m], and the orientation angle ranges from [-60°, 60°] to avoid outlier accumulation. The final verified average error curve is shown below. Figure 12 (As shown).
[0053] The final lowest average error distance was approximately 0.5091 m.
[0054] (Down Figure 13 The prediction example shown is a visualization of the prediction output of the model trained for the 50th round at an intersection turning scenario, where (below) Figure 13 (As shown) The left image shows the future driving trajectory line. The blue line is the line connecting the predicted trajectory points, and the green line is the line connecting the labeled trajectory points. The average error in this scenario reached 0.3489m; (Below) Figure 13 The right image (shown) is the grid map input for this scenario, showing the distribution of collision boxes, road markings, and road surfaces including traffic participants. The main vehicle is located in the center of the image (marked with a semi-transparent light red). It can be seen that the model has good target trajectory prediction capabilities in this scenario.
[0055] Performance: In the verification, the model's memory usage was approximately 2GB, and the average calculation time per frame was 0.1095 seconds, which meets the real-time requirements.
[0056] The model was trained for 50 rounds using a simulation training dataset of 16,000 frames. Finally, in a simulation verification dataset with 4,000 consecutive frames, the lowest average error distance of 0.5091m was obtained, and the average calculation time per frame reached 0.1095 seconds. The results show that the proposed model can learn and make planning predictions for the vehicle trajectory in the next 4 steps based on the provided grid pattern, historical trajectory and target information. The predicted points can be used as a reference for vehicle control and have good accuracy and real-time performance.
[0057] References (e.g., patents / papers / journals) Reference 1: Chen Yanyan, Tian Daxin, Lin Chunmian, et al., A review of research on end-to-end automated driving systems [J], Journal of Image and Graphics, 2024, 29(11):3216-3237.
[0058] Reference 2: Feng Yanrui, Ethical risks and soft law governance of autonomous driving algorithms [J], Industrial Innovation Research, 2024, (22):34-36.
[0059] Reference 3: Chu Duanfeng, Wang Rukang, Wang Jingyi, et al. Research progress and challenges of end-to-end autonomous driving [J], China Journal of Highway and Transport, 2024, 37(10):209-232. DOI:10.19721 / j.cnki.1001-7372.2024.10.019.
[0060] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A trajectory prediction system based on convolution and self-attention, characterized in that, include: Overall network architecture: The overall network architecture takes the road grid map, vehicle motion state and target information as input, and processes them sequentially through the DenseNet layer, self-attention coding layer, fully connected coding layer and decoding output layer to output the predicted trajectory; Training methods: The training method trains the network model based on a simulation dataset, calculates the loss value through a specific loss function, and updates the network weight parameters through backpropagation. Inference output algorithm: The inference output algorithm obtains the final trajectory output through polar coordinate trajectory difference calculation, coordinate transformation and multi-step cyclic prediction. The module is used for driving trajectory prediction in autonomous driving scenarios.
2. The trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The road grid map is a 256×256 resolution three-channel bird's-eye view grid map. The three channels are the collision boxes of surrounding traffic participants, road markings, and road surface map, respectively. Each channel corresponds to a different numerical label: the collision box value for the main vehicle is 0.4, and the collision box value for other traffic participants is 1; the solid line marking value is 0.6, and the dashed line and pedestrian crossing marking value is 0.4; the road surface value within the intersection is 0.4, and the road surface value for other surfaces is 1.
3. The trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The vehicle motion status includes the current vehicle speed and the polar coordinate driving trajectory information of the previous 6 frames. The target information includes the target vehicle speed and the relative coordinates of the target point. The current vehicle speed, the target vehicle speed, and the relative coordinates of the target point form a four-dimensional vector. The vehicle speed is in m / s and the coordinate length is in m. When encountering a red light, the target speed is set to 0.
4. The trajectory prediction system based on convolution and self-attention according to claim 3, characterized in that: The first 6 frames of polar coordinate driving trajectory information form a 6×3 matrix, which includes three elements: steering angle, driving distance, and orientation angle. The unit of steering angle is 15°, the unit of driving distance is 1.5m, and the unit of orientation angle is 30°. The trajectory information value is controlled within the range of [0,1], and when the driving distance is less than 0.1m, the values of steering angle and orientation angle are 0.
5. The trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The DenseNet layer replaces the pooling layers in the original structure with linear mappings of 1×1 convolutional kernels, and replaces all ReLU+BatchNorm layers with GELU+LayerNorm layers. The DenseNet layer receives two types of grid image inputs: a 6-frame continuous collision box grid image with 6 channels and a current frame collision box grid image, and a road marking and road surface merged grid image with 3 channels. After processing by the DenseNet network with the same structure, the two types of inputs are spliced into a tensor of size b×128×512, where b is the batch processing size.
6. The trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The self-attention encoding layer adopts a 6-layer stacked Transformer Encoder architecture, with both input and output being tensors of size b×128×512. Its positional encoding is generated using the following formula: , Where pos is the sequence position of the feature vector, and i is the dimensional position of the feature vector.
7. The trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The fully connected coding layer transposes the grid pattern encoding output by the self-attention coding layer, reduces the dimensionality through one-dimensional convolution, and calculates a feature vector of size b×768 through MLP; the historical trajectory information is processed by MLP layer, one-dimensional convolution and GELU layer to obtain a feature vector of size b×256; the two feature vectors are concatenated to obtain a coding vector of size b×1024.
8. The trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The decoding output layer takes the vehicle trajectory information from the first 6 frames and the predicted trajectory with all zeros replaced in the last 3 frames to form a historical trajectory tensor of size b×9×3. After passing through an MLP layer and one-dimensional convolution, a vector of size b×32 is obtained. This vector is concatenated with the current frame's velocity information and target position to obtain a feature vector of size b×36. After being upsized by an MLP layer and processed by LayerNorm, this feature vector is used as the input to the GRU. The output vector of the fully connected coding layer is processed by LayerNorm and used as the hidden layer of the GRU. The GRU output is processed by the Head layer to calculate the trajectory difference. After 4 iterations, a final trajectory output of size b×4×3 is obtained. The calculation formula for each step in the loop is as follows: ， The design layer has an input of A and an output of O; where A is a tensor of size b×3×4.
9. A trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The loss function calculation process includes: converting the normalized trajectory difference matrix into polar coordinate trajectory information. , Then, convert it to the coordinates of the trajectory point in the Cartesian coordinate system using the following formula: , The loss value is finally calculated using the following formula: , in Standard deviation The increment factor is added to apply a larger penalty when the turning angle and speed increase, so that the predicted trajectory obtained by training is smooth and stable. After the calculation is completed, backpropagation is performed based on the loss value and the learning rate to update the weight parameters of the network.
10. A trajectory prediction system based on convolution and self-attention according to claim 1, characterized in that: The simulation dataset was collected using the Carla simulation simulator and includes a collision box grid map, a road marking grid map, a road grid map, target speed, current speed, target position, current position, and vehicle orientation angle. The interval between consecutive frames in the acquisition scene is 0.5 seconds. During training, information from 6 consecutive frames is extracted and processed into polar coordinate driving trajectory information.