Multi-scale fusion intersection trajectory prediction method based on symmetric cross attention

Abstract

The invention discloses a multi-scale fusion intersection trajectory prediction method based on symmetric cross attention, and relates to the technical field of automatic driving. Dividing the motion characteristics of the target vehicle at the intersection at the plurality of observation moments, the relative motion characteristics of the target vehicle relative to the surrounding vehicles, and the green light remaining time corresponding to the plurality of observation moments into a plurality of time scales; for any time scale, extracting embedded representation vectors of the motion features and the relative motion features, and traffic signal control information features of residual green light time corresponding to a plurality of observation moments; based on the embedded representation vector and the traffic signal control information feature, obtaining a track feature; and splicing the trajectory features of different scales, and obtaining global features of the multi-scale trajectory information according to the spliced trajectory features, so as to determine a trajectory prediction result of the target vehicle at the future moment. The method can accurately predict the vehicle track of the intersection.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a multi-scale fusion intersection trajectory prediction method based on symmetric cross attention. Background Art

[0002] Trajectory prediction has become an indispensable task in the field of autonomous driving. Faced with complex road traffic environments, autonomous vehicles (AVs) are not only affected by road conditions and traffic facilities but also require accurate perception of the current state of surrounding vehicles to accurately predict their future trajectories. Accurate trajectory prediction can help AVs anticipate possible future traffic conditions and plan their future maneuvers, thereby minimizing traffic conflicts and improving traffic efficiency. The vehicle whose trajectory is being predicted is called the target vehicle (TV), while other vehicles within a certain distance of the TV are called surrounding vehicles (SVs).

[0003] Traffic signal-controlled intersections are a common and complex traffic scenario in cities, and predicting vehicle trajectories is becoming increasingly difficult. At intersections, target vehicles (TVs) are not only affected by surrounding vehicles (SVs), road geometry, and pedestrians, but are also strongly influenced by signal phase and timing (SPaT) information. For example, the duration of a traffic signal can significantly influence driving behavior, prompting vehicles to accelerate, decelerate, or change direction. Currently, existing trajectory prediction research focuses on linear scenarios such as highways, while limited research is focused on predicting vehicle trajectories at intersections.

[0004] Therefore, the existing technology urgently needs a method that can predict vehicle trajectories at intersections. Summary of the Invention

[0005] Based on this, it is necessary to provide a multi-scale fusion intersection trajectory prediction method based on symmetric cross attention to address the above technical problems, which can accurately predict vehicle trajectories at intersections.

[0006] The present invention adopts the following technical solutions: The present invention provides a multi-scale fusion intersection trajectory prediction method based on symmetric cross attention, comprising: Obtain the motion characteristics of the target vehicle at the intersection at multiple observation times, the relative motion characteristics of the target vehicle relative to the surrounding vehicles, and the remaining green light time corresponding to the multiple observation times; motion features of the target vehicle at multiple observation time points, relative motion features of the target vehicle relative to surrounding vehicles, and traffic signal control information features of green light remaining time corresponding to the multiple observation time points are divided into multiple time scales; For any time scale, embedding representation vectors of the motion features and the relative motion features are extracted, and traffic signal control information features of the green light remaining time corresponding to the multiple observation time points are extracted; Hidden state features of the embedding representation vectors and the traffic signal control information features are extracted, and attention features of the motion features, the relative motion features and the green light remaining time are extracted respectively, to obtain first attention features between the target vehicle and the surrounding vehicles, and second attention features between the target vehicle and the traffic signal; First interaction features between the first attention features and hidden state features of the target vehicle are extracted, and second interaction features between the second attention features and the hidden state features of the target vehicle are extracted; and the hidden state features of the target vehicle, the first interaction features and the second interaction features are spliced to obtain trajectory features; Trajectory features of different scales are spliced, fusion weights of each trajectory feature are generated according to the spliced trajectory features, and different scale trajectory features are fused through the fusion weights of the trajectory features of each scale to obtain global features of multi-scale trajectory information; According to the global features, a future trajectory of the target vehicle is predicted to obtain a trajectory prediction result of the target vehicle at a future time.

[0007] Optionally, the trajectory prediction result is obtained through a trajectory prediction model; the trajectory prediction model comprises an encoder, a driving intention feature fusion module and a future trajectory decoder; The encoder comprises a multi-scale fusion module, a symmetric cross-attention module, a first splicing unit and a dynamic convolution layer connected in sequence; the multi-scale fusion module comprises three encoding units connected in parallel, an addition unit and three long short-term memory neural networks connected in parallel, the output ends of the three encoding units are connected to the input end of the addition unit, and the output end of the addition unit is connected to the input ends of the three long short-term memory neural networks respectively; The symmetric cross-attention module comprises three multi-layer perceptrons connected in parallel, two completely symmetric cross-attention units, and a second splicing unit; the cross-attention unit comprises a multiplication unit, a scaling unit, a mask unit, an activation function, a multiplication unit, a third splicing unit, a gated linear unit, an addition unit and a layer normalization connected in sequence; The dynamic convolution layer consists of two branches. One branch includes an average pooling layer, a multi-layer perceptron, an activation function ReLU, a multi-layer perceptron, and an activation function softmax connected in series. The other branch includes a convolution unit. The activation function softmax of the first branch is connected to the convolution unit of the second branch through an addition unit. The output of the convolution unit of the second branch is the output of the dynamic convolution layer. The future trajectory decoder consists of an addition unit, a multi-layer perceptron, a long short-term memory neural network decoder layer and a multi-layer perceptron connected in series.

[0008] Optionally, the multi-scale fusion module includes a first encoding unit, a second encoding unit, and a third encoding unit, the first encoding unit includes a position encoding unit and a multi-layer perceptron connected in series, and the second encoding unit and the third encoding unit each include a convolution filter and a multi-layer perceptron connected in series; extracting embedded representation vectors of motion features and relative motion features, as well as traffic signal control information features of the remaining green light time corresponding to multiple observation moments, includes: The motion features, relative motion features, and traffic signal control information features are input into the multi-scale fusion module. In the first encoding unit, the remaining green light time corresponding to multiple observation moments is converted into an embedded representation using the sine-cosine function of the position encoding unit. The embedded representation is then converted into traffic signal control information features using a multi-layer perceptron. In the second coding unit, the target features of the motion features at each observation moment are extracted by a convolution filter, and the target features are converted into embedded representation vectors by a multi-layer perceptron; in the third coding unit, the relative features of the relative motion features at each observation moment are extracted by a convolution filter, and the relative features are converted into embedded representation vectors respectively by a multi-layer perceptron.

[0009] Optionally, extracting hidden state features of the embedding representation vector and traffic signal control information features includes: The embedding representation vectors and traffic signal control information features of all scales are added together through the addition unit, and the hidden state features of the embedding representation vectors and traffic signal control information features are extracted through the long short-term memory neural network of the corresponding scale.

[0010] Optionally, attention extraction is performed on the motion feature, the relative motion feature, and the remaining time of the green light, respectively, to obtain a first attention feature between the target vehicle and surrounding vehicles, and a second attention feature between the target vehicle and the traffic signal, including: The motion features, relative motion features, and remaining green time are input into the symmetric cross attention module. Three parallel multi-layer perceptrons are used to convert the motion features into query vectors, the relative motion features into key vectors and value vectors, and the remaining green time into key vectors and value vectors. The query vector, the key vector and the value vector of the relative motion feature are concatenated to obtain the first attention feature between the target vehicle and the surrounding vehicles; The query vector, the key vector, and the value vector of the remaining green light time are concatenated to obtain the second attention feature between the target vehicle and the traffic signal.

[0011] Optionally, extracting a first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and extracting a second interaction feature between the second attention feature and the hidden state feature of the target vehicle, and concatenating the hidden state feature of the target vehicle, the first interaction feature, and the second interaction feature to obtain a trajectory feature, including: In the symmetric cross attention module, two completely symmetric cross attention units are used to extract the first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and the second interaction feature between the second attention feature and the hidden state feature of the target vehicle; The hidden state feature, the first interaction feature and the second interaction feature of the target vehicle are spliced ​​together by the second splicing unit to obtain a trajectory feature.

[0012] Optionally, trajectory features of different scales are concatenated, including: The trajectory features of different scales are spliced ​​together through the first splicing unit.

[0013] Optionally, based on the spliced ​​trajectory features, a fusion weight of each trajectory feature is generated, and trajectory features of different scales are fused using the fusion weights of trajectory features of each scale to obtain a global feature of multi-scale trajectory information, including: The concatenated trajectory features are input into the dynamic convolution layer, and the fusion weight of each trajectory feature is generated through the first branch of the dynamic convolution layer; The trajectory features are convolved through the convolution unit in the second branch to obtain the dynamic convolution result; The dynamic convolution results of trajectory features at different scales are weightedly fused by fusion weights to obtain the global features of multi-scale trajectory information.

[0014] Optionally, the future trajectory of the target vehicle is predicted based on the global features to obtain a trajectory prediction result of the target vehicle at the future moment, including: In the driving intention feature fusion module, the probabilities of lateral and longitudinal maneuvering classes, as well as the combined features of different timestamps, are determined based on the global features. In the future trajectory decoder, the probabilities of lateral and longitudinal maneuvering classes, as well as the combined features of different timestamps, are fused to obtain coded features. The coded features are decoded to obtain the position distribution of the target vehicle at the future moment, i.e., the trajectory prediction result.

[0015] The present invention provides a multi-scale fusion intersection trajectory prediction device based on symmetric cross attention, comprising: An acquisition module is used to obtain the motion characteristics of a target vehicle at an intersection at multiple observation times, the relative motion characteristics of the target vehicle relative to surrounding vehicles, and the remaining time of the green light corresponding to the multiple observation times; A division module is used to divide the motion characteristics of the target vehicle at multiple observation moments, the relative motion characteristics of the target vehicle with respect to surrounding vehicles, and the remaining time of the green light corresponding to the multiple observation moments into multiple time scales; An extraction module is used to extract, for any time scale, embedded representation vectors of motion features and relative motion features, as well as traffic signal control information features of the remaining green light time corresponding to multiple observation moments; extract hidden state features of the embedded representation vectors and traffic signal control information features, and perform attention extraction on the motion features, relative motion features, and remaining green light time, respectively, to obtain a first attention feature between the target vehicle and surrounding vehicles, and a second attention feature between the target vehicle and the traffic signal; extract a first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and extract a second interaction feature between the second attention feature and the hidden state feature of the target vehicle; and concatenate the hidden state feature, the first interaction feature, and the second interaction feature of the target vehicle to obtain a trajectory feature; The fusion module is used to splice trajectory features of different scales and generate the fusion weight of each trajectory feature based on the spliced ​​trajectory features. The trajectory features of different scales are fused using the fusion weights of the trajectory features of each scale to obtain the global features of the multi-scale trajectory information. The prediction module is used to predict the future trajectory of the target vehicle based on the global features and obtain the trajectory prediction result of the target vehicle at the future moment.

[0016] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned multi-scale fusion intersection trajectory prediction method based on symmetric cross attention.

[0017] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the multi-scale fusion intersection trajectory prediction method based on symmetric cross attention is implemented.

[0018] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects: In the present invention, the motion characteristics of the target vehicle are collected, and the relative motion characteristics between it and the surrounding vehicles are obtained at the same time, ensuring that the model takes into account the dynamic interaction between vehicles, and introduces the remaining green light time as a traffic signal control feature, which is directly related to the dynamic environmental data of the vehicle. The trajectory of the target vehicle is jointly predicted based on these three types of information. When predicting the trajectory, different time scales are divided, and the interaction characteristics between vehicles and the interaction characteristics between vehicles and signals at different time scales are obtained through the temporal dependencies at different time scales, so as to obtain the global characteristics of multi-scale trajectory information at different time scales, thereby avoiding the information distortion caused by one-size-fits-all fusion. As a result, this method captures the microscopic details of vehicle motion and considers the macroscopic rules of the traffic system through the layer-by-layer progressive design of "multi-source information input → multi-scale feature extraction → attention-enhanced key interaction → multi-dimensional feature fusion", thereby achieving accurate prediction of future trajectories. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 A schematic diagram of the impact of traffic signal state transition on target vehicle trajectory provided by the present invention; Figure 2 A schematic flow chart of a multi-scale fusion intersection trajectory prediction method based on symmetric cross attention provided by the present invention; Figure 3 A schematic diagram of a scene description of an intersection provided by the present invention; Figure 4 A data collection schematic diagram provided by the present invention; Figure 5 A schematic diagram of the structure of a trajectory prediction model provided by the present invention; Figure 6 A structural diagram of a dynamic convolutional layer provided by the present invention; Figure 7 A schematic diagram of a computer device for implementing a multi-scale fusion intersection trajectory prediction method based on symmetric cross attention provided by the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] There are three limitations in performing intersection trajectory prediction tasks: (1) Traffic signal control data and vehicle trajectory data have heterogeneous characteristics in terms of temporal resolution, spatial representation dimension, and data structure: i) Traffic signal control data mainly describes the state changes and timing of signals, which are periodic, global, and externally constrained, reflecting the rules guiding vehicle behavior under the road environment; ii) Vehicle trajectory data is the movement trajectory of each vehicle in the time and space dimensions, which is continuous, individual, and behaviorally indicative, and is the core basis for predicting future positions. Therefore, it is necessary to accurately couple vehicle trajectories with traffic signal control information; (2) Due to the influence of traffic signal control, vehicles frequently accelerate and decelerate at intersections, and traditional kinematic models perform poorly at intersections; (3) Trajectory prediction models require a large amount of real intersection data for training, and currently such high-quality datasets are relatively rare. Therefore, predicting vehicle trajectories at intersections is extremely challenging.

[0023] In order to meet the above challenges, early studies usually used traditional machine learning methods to input traffic signal control data together with surrounding vehicle features into the model to predict vehicle trajectories at intersections. Methods include: Bayesian networks, support vector machines, clustering, and dictionary learning. However, due to the complexity of various interaction data and the limitations of the model itself, traditional methods are difficult to effectively capture complex dynamic interactions, resulting in reduced prediction accuracy. In order to solve the above problems, with the development of deep learning, some studies have tried to use deep neural networks to model various dynamic interactions to accurately predict vehicle trajectories at intersections. Due to the temporal correlation of traffic signal control information, most studies use recurrent neural networks (RNNs) to capture the interaction of traffic signal control information on target vehicles (TVs). These methods convert discrete traffic states into high-dimensional spaces, which are further processed by RNNs. However, since traffic signal control data has characteristics such as periodicity and discreteness, while vehicle trajectory data is continuous, micro-behavior data based on individuals, there are significant differences between the two in terms of time granularity, semantic hierarchy, and data structure characteristics. For example, Figure 1As shown in the figure, when the traffic signal is about to change from green to red, the target vehicle may be traveling at a constant speed before the red light comes on. After the red light comes on, the driver needs 1-2 seconds to react to control the vehicle to slow down, and then the speed gradually decreases until it stops. The change in the traffic signal state is an instantaneous event, while the vehicle's response is a delayed and continuous process. It is difficult for RNNs to capture this causal dynamic within a fixed time step. In addition, traffic signals are high-level instructions, while vehicles are low-level states. RNNs have no advantage in learning complex, nonlinear, and context-dependent mapping relationships. Therefore, a single RNN cannot accurately simulate the dynamic interaction between traffic signals and target vehicles, resulting in insufficient information utilization for traffic signal control. These oversights may lead to inaccurate trajectory predictions at intersections.

[0024] Furthermore, trajectory prediction performance is closely related to the richness of features contained in historical trajectories. Specifically, short-term predictions (1-3 seconds) rely heavily on information from the most recent trajectory segments, known as late features. However, for long-term predictions (4-5 seconds), it is necessary to leverage information from the entire historical trajectory segment, known as global features. Research has shown that the more detailed data a model obtains from different trajectory stages, the richer the features it extracts, thereby deepening the model's understanding of long-term vehicle motion patterns and dynamic interactions between vehicles. Early research typically used traditional neural network methods to extract spatiotemporal features of trajectories, including Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN). However, these methods focus on local features and may overlook global features or longer-term temporal dependencies within the trajectory. With the advent of the Transformer, its core attention mechanism helps the model focus on the most relevant parts of the prediction target, performing a weighted combination of different features to extract more comprehensive trajectory features. However, the inherent encoder architecture of models like the Transformer fails to explicitly extract secondary features within the model, which may not be important in the short term but may become crucial in the long term. Ignoring these features can limit the model's predictive capabilities. To address this issue, some studies have used multi-scale fusion methods to extract features from trajectories of different scales and fuse them together to account for the lack of model representation of minor features, thereby improving model prediction performance. However, existing methods often use basic convolutional neural networks to fuse trajectory features of different scales or directly concatenate these features. This approach fails to fully consider the dynamic correlations between trajectory features of different scales, limiting the fusion effect and making it difficult for the model to effectively capture the key information between features of different scales. This, in turn, limits the model's adaptability to complex driving scenarios and its predictive accuracy.

[0025] In recent years, significant achievements have been made in the field of intersection trajectory prediction both domestically and internationally. However, there are still significant deficiencies in the research on the interaction between the heterogeneous characteristics of multi-source data input to the prediction model and dynamic traffic environments such as traffic signal control and the target vehicle trajectory. The primary technical problem to be addressed by the present invention is how to design a deep learning module to analyze the above interaction mechanism based on the heterogeneous characteristics of multi-source data input to the prediction model and the interaction between the dynamic traffic environment such as traffic signal control and the target vehicle (TV) trajectory, scientifically and rationally integrate the heterogeneous characteristics of data, and accurately characterize its impact on trajectory prediction.

[0026] Secondly, existing research often overlooks the problem of minor features in trajectory data being excluded from model representation when performing trajectory prediction tasks. This results in the model being unable to simultaneously meet both short-term and long-term prediction accuracy requirements, limiting its predictive power and applicability. Designing a rational encoder structure based on the characteristics of historical trajectory data at different scales to effectively extract trajectory features at different scales, fully tapping into the rich information contained in historical trajectories and addressing the issue of minor features being excluded from model representation, is key to improving the accuracy and applicability of intersection trajectory prediction and is a pressing technical issue addressed by this present invention.

[0027] Based on the above problems, how to more flexibly and accurately integrate trajectory features of different scales, enhance the model's ability to model vehicle behavior patterns in complex traffic scenarios, and solve the problem of not fully considering the dynamic correlation between trajectory features of different scales are also technical problems that need to be solved by the present invention.

[0028] This paper proposes a multi-scale fusion intersection trajectory prediction method based on symmetric cross-attention for predicting future vehicle trajectories in intersection scenarios. To fully utilize the rich information contained in historical trajectories and avoid the exclusion of minor features from model representation, inspired by existing research, this paper adopts a multi-scale feature fusion approach and designs encoder blocks of different scales to extract richer features from the representation subspace of trajectories at different scales, thereby improving prediction performance. Furthermore, to fully account for the dynamic correlations between trajectory features at different scales, this paper introduces a dynamic convolutional network (DCN) to adaptively adjust the fusion weights of features at different scales, achieving a more flexible and accurate feature fusion strategy. This enhances the model's ability to model vehicle behavior patterns in complex traffic scenarios and improves trajectory prediction accuracy. To overcome the underutilization of traffic signal control information, this paper designs a novel symmetric cross attention module (SCAM) as the core component of the multi-scale fusion encoder. This module captures the temporal interaction features between vehicles and traffic signal control information and models the dynamic interaction features between vehicles. Finally, a driving intention feature fusion module is used to generate different feature combinations for trajectory prediction. Taking into account the temporal correlation of future trajectories, a decoder layer based on a long short-term memory network (LSTM) is used to predict the multimodal trajectories of future vehicles.

[0029] The technical solutions provided by various embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0030] Figure 2 The flowchart of the multi-scale fusion intersection trajectory prediction method based on symmetric cross attention in the present invention specifically includes the following steps: S101, obtain the motion characteristics of the target vehicle at the intersection at multiple observation times, the relative motion characteristics of the target vehicle with respect to the surrounding vehicles, and the remaining green light time corresponding to the multiple observation times, and divide the motion characteristics of the target vehicle at the multiple observation times, the relative motion characteristics of the target vehicle with respect to the surrounding vehicles, and the remaining green light time corresponding to the multiple observation times into multiple time scales.

[0031] (1) Scenario description In order to further analyze the impact of traffic signals on trajectory prediction, this paper takes the scenario of the target vehicle (TV) passing through the intersection with different remaining time of green lights as an example to illustrate the important role of traffic signals in trajectory prediction. Figure 1The figure shows the scenes of vehicles passing through the intersection with different remaining time of green lights. The red vehicle is the target vehicle (TV), the surrounding blue vehicles are surrounding vehicles (SVs) that may affect the driving of the target vehicle, and the yellow vehicles are non-interfering vehicles (NIVs). Indicates the remaining time of the green light. Indicates the initial position of the TV, Indicates the stop line at the intersection. Indicates the location where the TV enters the conflict zone through the intersection. Figure 3 As shown, when the remaining time of the green light is short (i.e. ), TV will choose to accelerate quickly to reach and pass the intersection, or decelerate quickly and stop to wait. However, when the remaining time of the green light is long (i.e. ), the TV will run smoothly. In this case, in order to accurately predict the trajectory of the red TV, it is necessary to comprehensively consider the influence of blue SVs and the remaining time of the green light. It is worth noting that although the present invention uses the remaining time of the green light as the traffic signal control information for the trajectory prediction task, it also implicitly considers the influence of the red light, which is mainly reflected in the change of vehicle behavior when the remaining time of the green light is short: when the green light is about to end, some vehicles will choose to actively slow down and stop, waiting for the next green light stage to pass. Although this behavior occurs during the green light period, its decision basis is essentially the prediction of the upcoming red light. Therefore, this part of the trajectory already includes the response to the influence of the red light. In addition, during the red light period, most vehicles are already stationary or in a decelerating state, and the changes in their motion trajectories are very small. Even if there is speed, its fluctuation is very limited, and the trajectory length is not enough to support effective prediction.

[0032] (2) Intersection data collection and extraction: Cross-shaped intersections with wide views and moderate traffic flow within the city can be selected for data collection. The specific collection methods are as follows: Figure 4 shown.

[0033] Vehicle trajectory data: Use a drone to record fixed-point aerial photography of vehicle trajectory videos at intersections. Use the SURF-Flann algorithm to stabilize the aerial videos and eliminate drone jitter errors. Use the YOLOv10-BoTSORT deep learning algorithm to detect and track vehicles. Use the VisDrone2019 dataset to train the algorithm. The collected vehicle trajectory videos are input into the trained model to output high-precision vehicle trajectory data. The trajectory data is then subjected to coordinate transformation and data noise reduction.

[0034] Traffic signal control data uses high-definition cameras to synchronously capture traffic signal control phase and timing data. This data is timestamped and aligned with vehicle trajectory data to ensure that the signal state at each moment is correctly mapped to the trajectory data at that moment, initially integrating the heterogeneity between traffic signal control data and vehicle trajectory data.

[0035] (3) Problem description Based on the collected and extracted data, the present invention describes the vehicle trajectory prediction problem as predicting the future trajectory of the TV based on the historical trajectory information of the TV and SVs and the traffic signal control information of the intersection. At timestamp t, the trajectory of vehicle i is defined as follows:

[0036] (1) in, n Indicates the timestamp t The number of vehicles in the training set, and the trajectory of each vehicle contains f state variables. The state variables are expressed as:

[0037] (2) The status of each vehicle includes coordinates, speed, acceleration, type, lane ID, and vehicle size.

[0038] vehicle i The observation trajectory is expressed as: (3) in, Indicates the observation frame length (including t Frame), that is, the frame length of the historical trajectory, the maximum value of which cannot exceed t . is the trajectory of the target vehicle T.

[0039] Target vehicle T The predicted trajectory is expressed as: (4) in, Indicates the predicted frame length, represents the x and y coordinates of the f state variable. Represents a timestamp in the future.

[0040] The motion characteristics of the target vehicle T as the model input are expressed as: (5) Considering the vehicles in the current lane, left lane, and right lane, the relative positions and relative velocities of TV and SVs are extracted as the motion features of SV, which are expressed as: (6) At an intersection, the historical trajectory of a vehicle is not only the instantaneous state at the current moment, but is also usually closely related to the traffic signal changes at historical moments. (TTI stands for time to intersection) and remaining time on green light At each timestamp t Relative time. Extract timestamp t Previous historical moments (including t The relative time corresponding to the frame is used as the time feature of the traffic signal control information.

[0041] (7) The input feature of the scene context is represented as: (8) The goal of this invention is to predict the target vehicle T The future trajectory of x and y Coordinates. The probability distribution of future trajectories is In the present invention, The distribution is parameterized to have mean ,variance and correlation coefficient The present invention is achieved by taking the average value Considered as predicted position To evaluate the model of the present invention.

[0042] Therefore, based on the above analysis, the motion characteristics of the target vehicle at the intersection at multiple observation times are obtained. , the relative motion characteristics of the target vehicle relative to the surrounding vehicles , and the remaining time of the green light corresponding to multiple observation times (Signal characteristics).

[0043] Cut the features of multiple observation moments from the tail to the front into lengths of L 、 L / 2 and L The total time length of multiple observation moments is L.

[0044] In order to achieve accurate trajectory prediction of vehicles at intersections, it is necessary to consider the spatiotemporal interaction between the target vehicle and surrounding vehicles, as well as the influence of traffic signal control information. Therefore, in one embodiment, the trajectory prediction result is obtained through a trajectory prediction model; Figure 5 As shown in Figure 3, the trajectory prediction model includes an encoder, a driving intention feature fusion module, and a future trajectory decoder.

[0045] The encoder includes a multi-scale fusion module, a symmetric cross attention module (SCAM), a first splicing unit (C) and a dynamic convolutional layer connected in series; the multi-scale fusion module includes three parallel encoding units, an addition unit and three parallel long short-term memory neural networks (LSTM). The output ends of the three encoding units are all connected to the input ends of the addition unit, and the output ends of the addition unit are respectively connected to the input ends of the three long short-term memory neural networks.

[0046] The multi-scale fusion module includes a first encoding unit, a second encoding unit, and a third encoding unit. The first encoding unit includes a positional encoding unit (PE) and a multi-layer perceptron (MLP) connected in series. The second and third encoding units each include a convolutional filter (Conv1d) and a multi-layer perceptron connected in series. The convolutional filter is a 1-D convolutional filter.

[0047] The symmetric cross attention module includes three parallel multi-layer perceptrons, two completely symmetrical cross attention units, and a second concatenation unit (Concatenate); the cross attention unit includes a multiplication unit (MatMul), a scaling unit (Scale), a mask unit (Mask), an activation function (SoftMax), a multiplication unit (MatMul), a third concatenation unit (C), a gated linear unit (GLU), an addition unit, and a layer normalization (LN) connected in series.

[0048] The dynamic convolution layer consists of two branches. One branch includes the average pooling layer (AvgPool), multi-layer perceptron, activation function ReLU, multi-layer perceptron, and activation function softmax connected in series; the other branch includes a convolution unit (conv). The activation function softmax of the first branch is connected to the convolution unit of the second branch through the addition unit. The output of the convolution unit of the second branch is the output of the dynamic convolution layer.

[0049] The future trajectory decoder consists of an addition unit, a multi-layer perceptron, a long short-term memory neural network decoder layer and a multi-layer perceptron connected in series.

[0050] S102 , for any time scale, extracting embedding representation vectors of motion features and relative motion features, as well as traffic signal control information features of the remaining green light time corresponding to multiple observation moments.

[0051] In one embodiment, the three encoding units correspond to motion features, relative motion features of the target vehicle relative to surrounding vehicles, and remaining green light time corresponding to multiple observation moments, respectively.

[0052] Therefore, the embedding representation vector of the motion feature and the relative motion feature, and the traffic signal control information feature corresponding to the green light remaining time of the plurality of observation time points are extracted by: inputting the motion feature, the relative motion feature and the traffic signal control information feature into a multi-scale fusion module; in a first encoding unit, converting the green light remaining time corresponding to the plurality of observation time points into an embedded representation by a sine-cosine function of a position encoding unit, and converting the embedded representation into a traffic signal control information feature by a multi-layer perception; in a second encoding unit, extracting a target feature of the motion feature at each observation time point by a convolution filter, and converting the target feature into an embedding representation vector by a multi-layer perception; and in a third encoding unit, extracting a relative feature of the relative motion feature at each observation time point by a convolution filter, and converting the relative feature into an embedding representation vector by a multi-layer perception.

[0053] Specifically, the motion state (target feature) and the relative motion feature of each observation timestamp are extracted by a 1-D convolution filter (kernel width = 3), and are converted into an embedding representation vector by a multi-layer perception.

[0054] (9) wherein, represents a convolution operation, is a weight of a convolution kernel, is a mapping function of a multi-layer perception, is a weight of a multi-layer perception, is an embedding representation vector of the vehicle; it should be noted that the multi-layer perception can include a full connection.

[0055] Since there is periodicity in the time frame, the time is converted into an embedded representation by using a position embedding based on a sine-cosine function, and the traffic signal control information feature is fused by using a multi-layer perception.

[0056] (10) wherein, represents a position embedding function based on a sine-cosine function without a learnable parameter. is a weight of a multi-layer perception, is a traffic signal control information feature.

[0057] S103, the embedding representation vector and the hidden state feature of the traffic signal control information feature are extracted, and the motion feature, the relative motion feature and the green light remaining time are extracted by attention respectively, to obtain a first attention feature between the target vehicle and the surrounding vehicles, and a second attention feature between the target vehicle and the traffic signal.

[0058] Each encoder structure consists of a long short-term memory (LSTM) neural network and an attention mechanism, designed to effectively model the complex spatiotemporal interactions between vehicles, as well as the temporal interactions between vehicles and traffic signals. Specifically, the hidden state features of the embedded representation vectors and traffic signal control information features are extracted by summing the embedded representation vectors and traffic signal control information features at all scales through an addition unit, and then extracting the hidden state features of the embedded representation vectors and traffic signal control information features through a LSTM neural network at the corresponding scale.

[0059] That is, the present invention uses a long short-term memory neural network encoder to encode the features of vehicle and traffic signal control information to capture time correlation.

[0060] (11) (12) in, represents the encoder of the long short-term memory neural network, and is the set of weight matrices in the LSTM unit. Indicates the timestamp t(L) Time Vehicle i The embedding of represents the hidden state features of the vector. Represents the hidden state features of the corresponding traffic signal control information features. For the target vehicle and its traffic signal control information features, the present invention obtains the corresponding feature representation and ,in , is the size of the encoding LSTM hidden state.

[0061] Optionally, attention extraction is performed on the motion features, relative motion features and green light remaining time respectively to obtain a first attention feature between the target vehicle and the surrounding vehicles, and a second attention feature between the target vehicle and the traffic signal, including: inputting the motion features, relative motion features and green light remaining time into a symmetric cross attention module, and converting the motion features into query vectors, the relative motion features into key vectors and value vectors, and the green light remaining time into key vectors and value vectors through three parallel multi-layer perceptrons; splicing the query vector, and the key vector and value vector of the relative motion feature to obtain the first attention feature between the target vehicle and the surrounding vehicles; splicing the query vector, and the key vector and value vector of the green light remaining time to obtain the second attention feature between the target vehicle and the traffic signal.

[0062] In particular, the symmetric cross-attention module (SCAM) aims to jointly capture the interactions between vehicles (vehicle-vehicle) and between vehicles and traffic signals (vehicle-traffic signal) to build a global interaction model. The motion features of the target vehicle are first transformed into query vectors , the relative motion features of surrounding vehicles are transformed into key vectors and value vectors . Similarly, the traffic signal control information features are also transformed into key vectors and value vectors .

[0063] (13) (14) (15) wherein, is a single-layer MLP, , and are learnable weights.

[0064] The cross-attention between vehicle features (motion features and relative motion features) and traffic signal control information features can be calculated as follows: (16) (17) wherein, and represent the dimensions of and , respectively. represents the spatiotemporal dynamic interaction dependency relationship between surrounding vehicles and the target vehicle at timestamp t . represents the time correlation between traffic signal control information and the target vehicle at timestamp t .

[0065] These independent attention mechanisms will be concatenated and then multiplied by weight matrices and , so as to calculate the first attention feature between the output target vehicle and surrounding vehicles and the second attention feature between the target vehicle and the traffic signal based on the multi-head attention mechanism.

[0066] (18) (19) wherein, represents a concatenation operation.

[0067] S104, extract first interaction features between the first attention features and the hidden state features of the target vehicle, and second interaction features between the second attention features and the hidden state features of the target vehicle; and concatenate the hidden state features of the target vehicle, the first interaction features and the second interaction features to obtain trajectory features.

[0068] Optionally, the first interaction features between the first attention features and the hidden state features of the target vehicle are extracted, and the second interaction features between the second attention features and the hidden state features of the target vehicle are extracted, and the hidden state features of the target vehicle, the first interaction features and the second interaction features are concatenated to obtain trajectory features, including: in the symmetric cross-attention module, the first interaction features between the first attention features and the hidden state features of the target vehicle are extracted by two completely symmetric cross-attention units, and the second interaction features between the second attention features and the hidden state features of the target vehicle are extracted; the hidden state features of the target vehicle, the first interaction features and the second interaction features are concatenated by a second splicing unit to obtain trajectory features.

[0069] Specifically, in order to suppress redundant features and retain important features, the application introduces Gated Linear Units (GLUs) to effectively extract complex interaction relationships, including first interaction features between first attention features and hidden state features of a target vehicle , and second interaction features between second attention features and hidden state features of the target vehicle .

[0070] (20) (21) wherein, and are the weights of the GLU. is a gated linear operation. denotes layer normalization.

[0071] In order to generate coarse-grained trajectory features, the application concatenates the hidden state features of the target vehicle, the first interaction features and the second interaction features to obtain trajectory features .

[0072] (22) S105, concatenate trajectory features of different scales, generate fusion weights of each trajectory feature according to the concatenated trajectory features, and fuse trajectory features of different scales through the fusion weights of trajectory features of different scales to obtain global features of multi-scale trajectory information.

[0073] The structure diagram of the dynamic convolution layer is shown in Figure 6 , which aims to efficiently extract key information from multi-scale trajectory features.

[0074] First, the trajectory features of different scales are spliced, including: splicing the trajectory features of different scales through a first splicing unit to obtain spliced trajectory features , as follows: (23) Next, according to the spliced trajectory features, the fusion weight of each trajectory feature is generated, and the trajectory features of different scales are fused through the fusion weight of each scale of the trajectory feature to obtain the global feature of the multi-scale trajectory information, including: inputting the spliced trajectory features into the dynamic convolution layer, generating the fusion weight of each trajectory feature through the first branch of the dynamic convolution layer; the trajectory features are convolved through the convolution unit in the second branch to obtain the dynamic convolution result; the dynamic convolution results of the trajectory features under different scales are weighted and fused through the fusion weight to obtain the global feature of the multi-scale trajectory information.

[0075] Specifically, the first branch composed of an average pooling layer and a multi-layer perceptron is used to generate the fusion weight of each feature, as follows: (24) (25) wherein, is the normalized weight, denotes the average pooling operation, and is the weight of the multi-layer perceptron.

[0076] Finally, the dynamic convolution results of the trajectory features under different scales are weighted and merged to obtain the globally fused multi-scale trajectory information .

[0077] (26) wherein, denotes the trajectory features of three scales. denotes the dynamic convolution operation. is the dynamic convolution kernel parameter.

[0078] S106, according to the global feature, the future trajectory of the target vehicle is predicted to obtain the trajectory prediction result of the target vehicle at the future time.

[0079] In one embodiment, the future trajectory of the target vehicle is predicted according to the global feature, and a trajectory prediction result of the target vehicle at a future time is obtained, including: in the driving intention feature fusion module, the probabilities of the lateral maneuver class and the longitudinal maneuver class and different timestamp combination features are determined according to the global feature; in the future trajectory decoder, the probabilities of the lateral maneuver class and the longitudinal maneuver class and the different timestamp combination features are fused to obtain an encoded feature; and the encoded feature is decoded to obtain a position distribution of the target vehicle at the future time, that is, the trajectory prediction result.

[0080] The encoder provided by the application avoids the problem of insufficient utilization of secondary features in the model, enables extraction of key information from multiple representation subspaces, and effectively captures long-term dependencies. At the same time, it can accurately capture the dynamic correlation between multi-scale trajectory features, thereby realizing a flexible and efficient feature fusion strategy, further extracting key information from multi-scale features, and enabling the model to balance between global features and local features, improving its adaptability and prediction accuracy in complex scenarios.

[0081] In the driving intention feature fusion module, the probabilities of the lateral maneuver class and the longitudinal maneuver class and different timestamp combination features are determined according to the global feature, and the specific operations are as follows: Due to the complexity of intersection traffic scenes, vehicles are not only affected by surrounding vehicles, but also strongly affected by traffic signals. For intersection scenes, driving maneuvers can be divided into three lateral maneuvers: straight driving (GS), left lane changing (LLC), and right lane changing (RLC), and three longitudinal maneuvers: acceleration (ACC), deceleration (DEC), and constant speed (CON). Among them, under the influence of traffic signals, longitudinal control is more intuitive. For example, when the remaining time of the green light is short, the vehicle will choose to accelerate or decelerate and wait according to its distance from the intersection. When the green light is on for a long time, the vehicle will usually choose to pass through the intersection at a constant speed. Therefore, the predicted trajectory under different maneuvers should be different, which requires different features as input. However, it is not clear which features provided by the MSF-SCA perform more positively in predicting trajectories. To solve this problem, the application adopts an intention feature fusion module to explicitly consider the importance of different intention features to trajectory prediction. Specifically, the application first uses a multi-layer perceptron with a softmax activation function to calculate the probabilities of the lateral maneuver class and the longitudinal maneuver class and the longitudinal maneuver class , as follows:

[0082] (27) (28) (29) wherein, , Then, the lateral weight matrix is ​​learned from the trajectory data in an end-to-end manner and the vertical weight matrix , which are combined at different timestamps as follows: (30) in, Represents the combined features of different timestamps, and Represents the eigenvectors In timestamp The contribution of each feature to predicting the horizontal and vertical positions. More important features usually have larger weights, while less important features have smaller weights.

[0083] In the future trajectory decoder, the probabilities of lateral and longitudinal maneuvers, as well as the combined features of different timestamps, are fused to obtain coded features. The coded features are decoded to obtain the position distribution of the target vehicle at the future moment, i.e., the trajectory prediction result. This includes:

[0084] In the problem statement, the future trajectory is parameterized as a bivariate Gaussian distribution. Therefore, the goal of this invention is to obtain the probability that describes the multimodal trajectory of Parameters. First, Expand as follows:

[0085] (31) in, is the parameter of the binary Gaussian distribution of the target vehicle’s future trajectory. By vehicle x and y The mean, variance and correlation coefficient of the coordinates reflect the uncertainty of the trajectory.

[0086] In order to predict the probability distribution of future trajectories according to different driving intentions, we first combine the driving intentions with the features. Fusion is performed by splicing it with the predicted probability of driving intention, and then inputting the fused result into the weighted In the multi-layer perceptron, the encoding features are obtained The specific process is as follows:

[0087] (32) Finally, the encoded features Input LSTM decoder, and then obtain the prediction parameters of future vehicle position distribution through multi-layer perceptron a position distribution of the target vehicle at a future time.

[0088] (33) wherein, are the learned weights of the LSTM decoder. are the weights used to convert the hidden state output by the LSTM decoder into the parameters of a bi-variate Gaussian distribution. is the hidden state vector of the decoder at time .

[0089] In one embodiment, the present application provides a training method of a trajectory prediction model. Since the optimization process at the initial stage of model training is relatively simple, a complex loss function is not needed to avoid increasing the burden of the model and slowing down the training speed. Therefore, the loss function of the model of the present application is composed of the mean square error loss and the negative log likelihood loss . The model is trained by minimizing .

[0090] (34) wherein, denotes the number of training rounds of the model, , . Specifically, in the first ten periods, the mean square error (MSE) loss is minimized as follows:

[0091] (35) wherein, is the position predicted by the model. is the true position of the target vehicle at the future timestamp . Then, the present application uses the negative log likelihood (NLL) loss to further train the model.

[0092] (36) The beneficial effects of the present application are as follows: (1) In terms of the interaction between traffic signal control data and vehicle trajectory, existing studies often choose to input traffic signal control information and vehicle trajectory data into the model together, ignoring the multi-source heterogeneity of the input data. This results in insufficient data utilization, irrational mechanism analysis, and poor trajectory prediction. Based on the heterogeneity of dynamic environmental data such as traffic signal control and its interaction with target vehicles, the present invention designs a symmetrical cross-attention module to analyze the above-mentioned mechanism and integrates the heterogeneity of dynamic environmental data such as traffic signal control, so that the model can model the dynamic interaction between dynamic traffic environments such as traffic signal control and target vehicles in parallel. This method solves the above-mentioned problems existing in current research from a methodological perspective. This is the main innovation of the present invention.

[0093] (2) In terms of feature extraction from historical trajectory data, existing studies have not fully considered the problem that minor features in trajectory data are not involved in model expression. Ignoring these features, the model may produce excellent results in the short-term prediction domain, but poor results in the long-term prediction domain, which limits the model's predictive ability and applicability. Based on the heterogeneity of historical trajectory data features at different scales, this paper designs a multi-scale feature fusion encoder to effectively extract local features, mid-term features, and global features of historical trajectory data, so as to fully mine the rich information contained in the historical trajectory, solve the problem that minor features are not involved in model expression, improve the accuracy and applicability of the model, and promote the application of this method in the field of autonomous driving.

[0094] (3) In terms of feature fusion, existing studies have mostly used the most basic convolutional neural network (CNN) to fuse trajectory features of different scales or directly splice these features. This does not fully consider the dynamic correlation between trajectory features of different scales, which limits the fusion effect. Based on the dynamic correlation between trajectory features of different scales, this paper introduces a dynamic convolutional network to adaptively adjust the fusion weights of features of different scales, achieving a more flexible and accurate feature fusion strategy, thereby enhancing the model's ability to model vehicle behavior patterns in complex traffic scenarios and improving the accuracy of trajectory prediction.

[0095] When applying the multi-scale fusion intersection trajectory prediction method based on symmetric cross attention provided by the present invention, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.

[0096] The above is a multi-scale fusion intersection trajectory prediction method based on symmetric cross attention provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding multi-scale fusion intersection trajectory prediction device based on symmetric cross attention, which includes: An acquisition module is used to obtain the motion characteristics of a target vehicle at an intersection at multiple observation times, the relative motion characteristics of the target vehicle relative to surrounding vehicles, and the remaining time of the green light corresponding to the multiple observation times; A division module is used to divide the motion characteristics of the target vehicle at multiple observation moments, the relative motion characteristics of the target vehicle with respect to surrounding vehicles, and the remaining time of the green light corresponding to the multiple observation moments into multiple time scales; An extraction module is used to extract, for any time scale, embedded representation vectors of motion features and relative motion features, as well as traffic signal control information features of the remaining green light time corresponding to multiple observation moments; extract hidden state features of the embedded representation vectors and traffic signal control information features, and perform attention extraction on the motion features, relative motion features, and remaining green light time, respectively, to obtain a first attention feature between the target vehicle and surrounding vehicles, and a second attention feature between the target vehicle and the traffic signal; extract a first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and extract a second interaction feature between the second attention feature and the hidden state feature of the target vehicle; and concatenate the hidden state feature, the first interaction feature, and the second interaction feature of the target vehicle to obtain a trajectory feature; The fusion module is used to splice trajectory features of different scales and generate the fusion weight of each trajectory feature based on the spliced ​​trajectory features. The trajectory features of different scales are fused using the fusion weights of the trajectory features of each scale to obtain the global features of the multi-scale trajectory information. The prediction module is used to predict the future trajectory of the target vehicle based on the global features and obtain the trajectory prediction result of the target vehicle at the future moment.

[0097] Regarding the specific limitations of the multi-scale fusion intersection trajectory prediction device based on symmetric cross-attention, please refer to the limitations of the multi-scale fusion intersection trajectory prediction method based on symmetric cross-attention above, which will not be repeated here. The various modules in the above-mentioned multi-scale fusion intersection trajectory prediction device based on symmetric cross-attention can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0098] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A multi-scale fusion intersection trajectory prediction method based on symmetric cross attention is proposed.

[0099] The present invention also provides Figure 7The structural diagram of the computer device is shown as follows, Figure 7 As shown in the hardware layer, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to realize the above-mentioned Figure 1 The multi-scale fusion intersection trajectory prediction method based on symmetric cross attention is provided.

[0100] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in each embodiment of the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0101] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of each technical feature in the above-mentioned embodiments are not described, however, as long as the combination of these technical features does not exist contradictory, it should be considered as the scope of the present application.

Claims

1. A multi-scale fusion intersection trajectory prediction method based on symmetric cross attention, characterized by: include: Obtain the motion characteristics of the target vehicle at the intersection at multiple observation times, the relative motion characteristics of the target vehicle relative to the surrounding vehicles, and the remaining green light time corresponding to the multiple observation times; The motion characteristics of the target vehicle at multiple observation times, the relative motion characteristics of the target vehicle relative to surrounding vehicles, and the remaining time of the green light corresponding to the multiple observation times are divided into multiple time scales; For any time scale, the embedded representation vectors of motion features and relative motion features, as well as the traffic signal control information features of the remaining green light time corresponding to multiple observation moments, are extracted; Extract the hidden state features of the embedded representation vector and traffic signal control information features, and perform attention extraction on the motion features, relative motion features, and remaining green light time, respectively, to obtain the first attention feature between the target vehicle and surrounding vehicles, and the second attention feature between the target vehicle and the traffic signal; Extracting a first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and extracting a second interaction feature between the second attention feature and the hidden state feature of the target vehicle; and concatenating the hidden state feature of the target vehicle, the first interaction feature, and the second interaction feature to obtain a trajectory feature; The trajectory features of different scales are spliced ​​together, and the fusion weight of each trajectory feature is generated based on the spliced ​​trajectory features. The trajectory features of different scales are fused by the fusion weight of the trajectory features of each scale to obtain the global features of the multi-scale trajectory information; The future trajectory of the target vehicle is predicted based on the global features to obtain the trajectory prediction result of the target vehicle at the future moment.

2. The method according to claim 1, characterized in that Obtaining a trajectory prediction result through a trajectory prediction model; the trajectory prediction model includes an encoder, a driving intention feature fusion module, and a future trajectory decoder; The encoder includes a multi-scale fusion module, a symmetric cross-attention module, a first splicing unit, and a dynamic convolutional layer connected in series. The multi-scale fusion module includes three encoding units, an addition unit, and three long short-term memory neural networks connected in parallel. The outputs of the three encoding units are connected to the inputs of the addition unit, and the outputs of the addition unit are connected to the inputs of the three long short-term memory neural networks respectively. The symmetric cross attention module consists of three parallel multi-layer perceptrons, two completely symmetric cross attention units, and a second splicing unit. The cross attention unit consists of a multiplication unit, a scaling unit, a mask unit, an activation function, a multiplication unit, a third splicing unit, a gated linear unit, an addition unit, and layer normalization connected in series. The dynamic convolution layer consists of two branches. The first branch includes an average pooling layer, a multi-layer perceptron, an activation function ReLU, a multi-layer perceptron, and an activation function softmax connected in series. The second branch includes a convolution unit. The activation function softmax of the first branch is connected to the convolution unit of the second branch through an addition unit. The output of the convolution unit of the second branch is the output of the dynamic convolution layer. The future trajectory decoder consists of an addition unit, a multi-layer perceptron, a long short-term memory neural network decoder layer and a multi-layer perceptron connected in series.

3. The method according to claim 2, characterized in that The multi-scale fusion module includes a first encoding unit, a second encoding unit, and a third encoding unit. The first encoding unit includes a position encoding unit and a multi-layer perceptron connected in series. The second encoding unit and the third encoding unit each include a convolution filter and a multi-layer perceptron connected in series. The module extracts embedded representation vectors of motion features and relative motion features, as well as traffic signal control information features of the remaining green light time corresponding to multiple observation moments, including: The motion features, relative motion features, and traffic signal control information features are input into the multi-scale fusion module. In the first encoding unit, the remaining green light time corresponding to multiple observation moments is converted into an embedded representation using the sine-cosine function of the position encoding unit. The embedded representation is then converted into traffic signal control information features using a multi-layer perceptron. In the second coding unit, the target features of the motion features at each observation moment are extracted by a convolution filter, and the target features are converted into embedded representation vectors by a multi-layer perceptron; in the third coding unit, the relative features of the relative motion features at each observation moment are extracted by a convolution filter, and the relative features are converted into embedded representation vectors respectively by a multi-layer perceptron.

4. The method according to claim 2, characterized in that Extract hidden state features of the embedding representation vector and traffic signal control information features, including: The embedding representation vectors and traffic signal control information features of all scales are added together through the addition unit, and the hidden state features of the embedding representation vectors and traffic signal control information features are extracted through the long short-term memory neural network of the corresponding scale.

5. The method according to claim 2, characterized in that Attention is extracted from motion features, relative motion features, and remaining green light time, respectively, to obtain the first attention feature between the target vehicle and surrounding vehicles, and the second attention feature between the target vehicle and the traffic signal, including: The motion features, relative motion features, and remaining green time are input into the symmetric cross attention module. Three parallel multi-layer perceptrons are used to convert the motion features into query vectors, the relative motion features into key vectors and value vectors, and the remaining green time into key vectors and value vectors. The query vector, the key vector and the value vector of the relative motion feature are concatenated to obtain the first attention feature between the target vehicle and the surrounding vehicles; The query vector, the key vector, and the value vector of the remaining green light time are concatenated to obtain the second attention feature between the target vehicle and the traffic signal.

6. The method according to claim 5, characterized in that Extracting a first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and extracting a second interaction feature between the second attention feature and the hidden state feature of the target vehicle, and concatenating the hidden state feature of the target vehicle, the first interaction feature, and the second interaction feature to obtain a trajectory feature, including: In the symmetric cross attention module, two completely symmetric cross attention units are used to extract the first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and the second interaction feature between the second attention feature and the hidden state feature of the target vehicle; The hidden state feature, the first interaction feature and the second interaction feature of the target vehicle are spliced ​​together by the second splicing unit to obtain a trajectory feature.

7. The method according to claim 2, characterized in that The trajectory features of different scales are stitched together, including: The trajectory features of different scales are spliced ​​together through the first splicing unit.

8. The method according to claim 2, characterized in that Based on the spliced ​​trajectory features, the fusion weight of each trajectory feature is generated, and the trajectory features of different scales are fused through the fusion weight of the trajectory features of each scale to obtain the global features of the multi-scale trajectory information, including: The concatenated trajectory features are input into the dynamic convolution layer, and the fusion weight of each trajectory feature is generated through the first branch of the dynamic convolution layer; The trajectory features are convolved through the convolution unit in the second branch to obtain the dynamic convolution result; The dynamic convolution results of trajectory features at different scales are weightedly fused by fusion weights to obtain the global features of multi-scale trajectory information.

9. The method according to claim 2, characterized in that The future trajectory of the target vehicle is predicted based on the global features to obtain the trajectory prediction results of the target vehicle at the future moment, including: In the driving intention feature fusion module, the probabilities of lateral and longitudinal maneuvering classes, as well as the combined features of different timestamps, are determined based on the global features. In the future trajectory decoder, the probabilities of lateral and longitudinal maneuvering classes, as well as the combined features of different timestamps, are fused to obtain coded features. The coded features are decoded to obtain the position distribution of the target vehicle at the future moment, i.e., the trajectory prediction result.

10. A multi-scale fusion intersection trajectory prediction device based on symmetric cross attention, characterized in that: include: An acquisition module is used to obtain the motion characteristics of a target vehicle at an intersection at multiple observation times, the relative motion characteristics of the target vehicle relative to surrounding vehicles, and the remaining time of the green light corresponding to the multiple observation times; A division module is used to divide the motion characteristics of the target vehicle at multiple observation moments, the relative motion characteristics of the target vehicle with respect to surrounding vehicles, and the remaining time of the green light corresponding to the multiple observation moments into multiple time scales; An extraction module is used to extract the embedded representation vectors of motion features and relative motion features, as well as traffic signal control information features of the remaining green light time corresponding to multiple observation moments, for any time scale; Extract the hidden state features of the embedded representation vector and traffic signal control information features, and perform attention extraction on the motion features, relative motion features, and remaining green light time, respectively, to obtain the first attention feature between the target vehicle and surrounding vehicles, and the second attention feature between the target vehicle and the traffic signal; Extracting a first interaction feature between the first attention feature and the hidden state feature of the target vehicle, and extracting a second interaction feature between the second attention feature and the hidden state feature of the target vehicle; and concatenating the hidden state feature of the target vehicle, the first interaction feature, and the second interaction feature to obtain a trajectory feature; The fusion module is used to splice trajectory features of different scales and generate the fusion weight of each trajectory feature based on the spliced ​​trajectory features. The trajectory features of different scales are fused using the fusion weights of the trajectory features of each scale to obtain the global features of the multi-scale trajectory information. The prediction module is used to predict the future trajectory of the target vehicle based on the global features and obtain the trajectory prediction result of the target vehicle at the future moment.

Images