A path coding-based multi-modal trajectory prediction and path tracking control method

Abstract

The application provides a path coding-based multimodal trajectory prediction and path tracking control method, aiming to improve the trajectory prediction accuracy and path tracking performance in an intelligent driving system. The application relates to the field of intelligent driving. The application comprises a traffic scene information acquisition module, a reconstructed path coding module, a TraLSTM-GAN multimodal prediction module and a controller module. The traffic scene information acquisition module collects traffic scene information, and transmits the trajectories of traffic participants to the reconstructed path coding module, which is matched with the paths in the path coding library to generate reconstructed path coding. The TraLSTM-GAN multimodal prediction module predicts the future trajectories of surrounding vehicles according to real-time traffic environment information and reconstructed path coding, and outputs the future trajectories of surrounding vehicles to the controller module, which optimizes and solves the control instructions of the autonomous vehicle to realize path tracking of the autonomous vehicle.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving and intelligent transportation technology, specifically relating to a multimodal trajectory prediction and path tracking control method based on path coding. Background Technology

[0002] With the continuous development of intelligent vehicle technology and data processing capabilities, autonomous driving systems are placing higher demands on the accuracy and real-time performance of trajectory prediction. As a crucial supporting technology for achieving environmental understanding and decision-making control, trajectory prediction results directly impact the safety and robustness of the vehicle during operation.

[0003] Multimodal trajectory prediction methods play a crucial role in enhancing the safety decision-making capabilities of autonomous driving systems. Compared to traditional trajectory prediction methods that only model single interaction relationships, multimodal trajectory prediction methods can jointly consider the interaction characteristics of "vehicle-to-vehicle" and "vehicle-to-lane" interactions, thereby enhancing the ability to model behavioral uncertainties and diversity in dynamic traffic environments. They also have better applicability in complex traffic scenarios and high-speed dynamic environments.

[0004] Currently, various multimodal trajectory prediction methods have been proposed, each with its own advantages but also certain limitations. For example, patent CN202410983125 generates candidate trajectory endpoints by anchor points and combines them with global features for trajectory prediction, which has the advantage of low computational overhead. However, it still relies on path regression methods to generate complete trajectories and is highly dependent on navigation information, resulting in insufficient generalization ability in complex or unseen environments. Patent CN119329558 constructs an interactive graph to process graph-structured traffic scene data, thereby generating a final feature matrix and achieving multimodal prediction through trajectory decoding and score decoding modules. This method performs well in terms of spatial smoothness, but it is difficult to generate trajectories with large variations and consumes a lot of computational resources. Patent CN114626598 proposes a conditional variational generation model, which combines historical trajectories and semantic maps to generate future multimodal trajectories. It can better model trajectory uncertainty, but the computational complexity is high. In summary, these methods have problems such as relying on complete navigation information, limited generalization ability, difficulty in covering a wide range of trajectory changes, high inference overhead, or insufficient trajectory continuity. In addition, most existing methods are based on continuous spatial regression for trajectory generation, lacking effective use of trajectory structure information, and it is difficult to achieve structural constraints and expression unity in the trajectory generation process.

[0005] To overcome the shortcomings of existing technologies, and addressing the issues of lack of trajectory structure constraints, limited generalization ability, and insufficient coupling between prediction results and control execution, this invention proposes a multimodal trajectory prediction and path tracking control method based on path coding. This method processes the trajectories of traffic participants to generate reconstructed path codes, which are then input to the TraLSTM-GAN multimodal prediction module for multimodal trajectory generation, optimization, and filtering. The filtering results are then input to the controller module to output vehicle control commands, adapting to diverse traffic scenarios and dynamic environmental changes.

[0006] Compared with traditional continuous trajectory regression and control methods, this invention improves the structured expression capability of multimodal trajectory prediction by introducing reconstructed path encoding representation, and improves the connection between prediction results and control modules, thereby helping to improve the real-time response performance of the system and enhance the consistency and stability between prediction results and control execution. Summary of the Invention

[0007] To address the shortcomings of existing technologies in multimodal trajectory prediction and path tracking control, such as insufficient prediction accuracy, weak generalization ability, poor adaptability to complex scenarios, and high computational cost, this invention proposes a path coding-based multimodal trajectory prediction and path tracking control method. This method constructs a reconstructed path code to uniformly represent trajectories. By building a reconstructed path code representation of traffic participant trajectories and combining it with a multimodal generation model, this method generates and optimizes future trajectories, thereby achieving high-precision trajectory prediction and stable path tracking control.

[0008] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0009] This invention proposes a multimodal trajectory prediction and path tracking control method based on path coding, comprising a traffic scene information acquisition module, a path reconstruction coding module, a TraLSTM-GAN multimodal prediction module, and a controller module. The traffic scene information acquisition module acquires traffic scene information and extracts the trajectories of traffic participants and real-time traffic environment information. The path reconstruction coding module converts the trajectories of traffic participants into reconstructed path codes. The TraLSTM-GAN multimodal prediction module generates the future trajectory of the vehicle based on the reconstructed path codes and real-time traffic environment information. The controller module outputs vehicle control commands based on the future trajectory of the vehicle to achieve path tracking control. These modules are sequentially connected to form a closed-loop processing flow consisting of traffic scene information acquisition, path reconstruction coding, multimodal trajectory prediction and filtering, and control command output.

[0010] Step 1: Design of the traffic scene information collection module:

[0011] The traffic scene information acquisition module is used to acquire the trajectory information of traffic participants and real-time traffic environment information, and provides them as input to the path reconstruction coding module and the TraLSTM-GAN multimodal prediction module, respectively. The trajectory information of traffic participants includes information on surrounding traffic participants, of which vehicle information includes position, speed, acceleration, heading angle and corresponding timestamp. The traffic scene information acquisition module is also used to acquire vehicle status information, which includes vehicle position, speed, acceleration and heading angle. The real-time traffic environment information includes lane geometry information, traffic light status, road speed limit, traffic period and weather information.

[0012] The reconstructed path coding module is used to match and search the trajectories of traffic participants to generate reconstructed path codes, and uses the reconstructed path codes as the input representation of the TraLSTM-GAN multimodal prediction module.

[0013] The reconstructed path coding module includes a feature curve generation module, a path coding library, and a path matching module. Specifically, the feature curve generation module generates feature curves based on the trajectories of traffic participants transmitted by the traffic scene information collection module, using both a feature curve generation method based on traffic participant trajectories and a feature curve generation method based on road topology. The path coding library is used to summarize and store the generated feature curves, and establish corresponding coding indexes for each feature curve, forming a searchable set of feature curve codes. The path matching module is used to retrieve feature curve codes from the path coding library through search matching commands, and matches the traffic participant trajectories with the retrieved feature curve codes to reconstructed path codes.

[0014] Step 2.1, Design of the characteristic curve generation module:

[0015] The feature curve generation module includes a feature curve generation method based on traffic participant trajectories and a feature curve generation method based on road topology. The generated feature curves are used to characterize the typical movement patterns of traffic participants under road constraints and serve as the basic module for building a path coding library.

[0016] The method for generating feature curves based on the trajectories of traffic participants divides the trajectories of traffic participants output by the traffic scene information acquisition module into segments by determining segmentation points, and extracts the feature curves of traffic participants based on the segmentation results.

[0017] Define the trajectories of traffic participants in a traffic scenario as follows: And divide it into segments; among them Indicating the first position in the trajectory The two-dimensional position coordinates of each point; the selection of these points and the method of deriving the trajectory depend on the specific segmentation technology. The decomposition methods are as shown in equations (1) to (4), respectively using the changes in heading angle, curvature, speed and acceleration of traffic participants as the basis for generating characteristic curves.

[0018] The change in heading angle is defined as shown in equation (1) and is used to characterize the change in trajectory direction:

[0019] (1)

[0020] The curvature is defined as shown in equation (2) and is used to characterize the degree of curvature of the trajectory:

[0021] (2)

[0022] in, This represents the area of ​​the triangle formed by three adjacent trajectory points.

[0023] The velocity is defined as shown in equation (3) and is used to describe the velocity of the trajectory:

[0024] (3)

[0025] Acceleration is defined as shown in equation (4) and is used to describe the velocity change of a trajectory:

[0026] (4)

[0027] The trajectory is determined to be a segment point when any of the following conditions are met: , , , At that time, it was believed , , , The corresponding time point is the segmentation point; among which, Indicates the first Change in heading angle at each trajectory point Indicates the first Curvature at each trajectory point Indicates the first The velocity corresponding to each trajectory point Indicates the first The acceleration corresponding to each trajectory point; , , , These are the threshold values ​​for heading angle change, curvature, velocity, and acceleration, respectively. Represents Euclidean distance. This indicates the time interval between adjacent sampling.

[0028] The feature curve generation module uses a road topology-based feature curve generation method to extract trajectory segments corresponding to lane centerlines from a high-precision map to generate feature curves. The specific method is as follows:

[0029] The centerline of each lane is segmented, and local trajectory segments are obtained according to a set length to ensure that each trajectory segment has local motion semantic features and meets local geometric semantic constraints. Each trajectory segment is a feature curve.

[0030] Step 2.2, Path Encoding Library Construction:

[0031] The path coding library consists of feature curves generated by a feature curve generation method based on traffic participant trajectories and a feature curve generation method based on road topology. After normalization of these feature curves, a unified path coding library is formed. The normalization methods include: (1) translation normalization, (2) rotation normalization, and (3) scale normalization. All feature curves are converted to a standard form represented in a unified standard coordinate system. The path coding library can be represented as a set of normalized feature curve codes, serving as the search space for the path matching module. ,in The first in the path encoding library Encoding of each characteristic curve, , This represents the total number of feature curve codes stored in the path encoding library; the number of feature curve codes in the path encoding library The path encoding library is determined based on historical trajectory data and road topology data imported during the offline construction phase, and is formed after feature curve extraction, normalization processing, and duplicate sample removal. During the online inference phase, the path encoding library maintains a preset size and is available for use by the path matching module.

[0032] Step 2.3, Path Matching Module Design:

[0033] The path matching module retrieves feature curve codes from the path coding library through a search matching command, and represents the trajectory of traffic participants as a combination of multiple feature curve codes.

[0034] In the combination of feature curves, each feature curve segment is encoded as a triple: in, Indicates the first Segment feature curve encoding.

[0035] in, This represents the index of the feature curve in the path encoding library, and it is unique. The spatial offset vector code of this feature curve segment is used to characterize the spatial position of the feature curve. The geometric and dynamic attributes of the connection points are encoded to characterize the local geometric and motion features of the feature curves.

[0036] The trajectory of the traffic participants Classified as part, This represents the total number of trajectory sampling points. The path matching module searches for the optimal combination of feature curves in the path coding library to map the trajectories of traffic participants into reconstructed path coding sequences. As the output of the path matching module, Indicates the reconstructed path encoding. Indicates the number of segments in the trajectory.

[0037] The reconstructed path encoding The combination of feature curves with the highest matching degree with the trajectory of traffic participants, obtained by searching the path coding library through the path matching module, is used to perform a structured representation of the trajectory of traffic participants, as shown in Equation (5):

[0038] (5)

[0039] in This indicates that the feature curve encoding and splicing operation is performed sequentially.

[0040] Indicates the first The segment feature curve encoding involves searching for the feature curve combination that best matches the trajectory of traffic participants in the path encoding library. The A* search algorithm is used to optimize the search in the path space, which consists of all possible feature curve encoding combinations.

[0041] In the path space, each node corresponds to a candidate feature curve encoding segment. Cost function As shown in equation (6):

[0042] (6)

[0043] in This represents the actual cost from the starting node to the current node. Let be a heuristic function, representing the estimated matching cost from the current node to the endpoint of the target trajectory. This represents a node in the path space.

[0044] The cumulative matching error and heuristic functions The calculation formulas are shown in equations (7) and (8):

[0045] (7)

[0046] (8)

[0047] in, This represents the current search node in the path matching search process; and These represent the current matching feature curve and the target feature curve at the [number]th [time]. Encoding of geometric and dynamic properties of each point and These represent the spatial offset vectors of the corresponding points; , The total number of points on the characteristic curve. and These represent the current search node. Corresponding path endpoint coordinates; and These represent the coordinates of the endpoint of the target feature curve; This represents the Euclidean distance.

[0048] The estimated cost This is a heuristic estimation function used to estimate the matching error from the current node to the target trajectory. Starting from the current node, it expands all possible path coding segments. ,according to The value selects the next node until a path match is completed or the maximum search depth is reached.

[0049] Finally, the path matching module represents the trajectories of traffic participants as a combined sequence of feature curve codes. That is, reconstructing the path encoding and using the reconstructed path encoding as the input representation of the TraLSTM-GAN multimodal prediction module.

[0050] Step 3: Design of the TraLSTM-GAN multimodal prediction module:

[0051] The TraLSTM-GAN multimodal prediction module includes a TraLSTM feature extractor, a multimodal trajectory prediction module, a GAN trajectory optimization module, and a trajectory selection module. The TraLSTM feature extractor is used to perform temporal modeling of real-time traffic environment information, extract latent feature representations of the traffic environment, and integrate them with the reconstructed path encoding. These are used as conditional inputs to the multimodal trajectory prediction module; the multimodal trajectory prediction module generates multiple candidate future trajectories of the vehicle based on the generative model; the GAN trajectory optimization module optimizes the distribution of the generated trajectories based on the generative adversarial mechanism; and the trajectory selection module is used to filter target trajectories from the multimodal trajectory set.

[0052] The hidden state update of the TraLSTM feature extractor is shown in equation (9):

[0053] (9)

[0054] in Indicates the first Input of traffic environment characteristics at any given time. Indicates a hidden state. Indicates the module status. This represents a recursive update function;

[0055] The TraLSTM feature extractor introduces an attention-based multi-agent interaction modeling method to model the interaction relationships among multiple traffic participants.

[0056] Calculate during the attention-based feature interaction process The matrix process is as shown in equation (10):

[0057] (10)

[0058] in, Represents the input feature matrix. It is a query, key, and value matrix. It is a learnable transformation matrix.

[0059] The attention mechanism is calculated using the softmax function, as shown in equation (11):

[0060] (11)

[0061] in It is a dimension scaling factor, and softmax is used to normalize the output weights;

[0062] In the multimodal trajectory generation process, the inference and generation processes and the calculation formulas for the loss function are as shown in equations (12) and (13):

[0063] (12)

[0064] (13)

[0065] in, For the posterior distribution, For decoders, used to generate the first... Candidate future trajectories , Input as a condition; For the true future trajectory sequence; As latent variables, Let KL divergence be the KL divergence. For trajectory reconstruction terms, Indicates the encoding network parameters, Indicates the generation of network parameters. This represents the prior distribution of the latent variable.

[0066] The GAN trajectory optimization module optimizes the distribution of generated trajectories based on generative adversarial networks. It distinguishes between generated trajectories and real trajectories through a discriminator, thereby constraining the distribution of generated trajectories to approximate that of real trajectories. The adversarial loss function of GAN is shown in Equation (14):

[0067] (14)

[0068] in For generator, For discriminator, For conditional input, As latent variables, Sampling for real future trajectories, Sampling for latent variables, The output probability of the discriminator for the true trajectory. This represents the output probability of the discriminator for the generated trajectory.

[0069] The TraLSTM-GAN multimodal prediction module completes candidate future trajectory generation, probability evaluation, and trajectory selection within the module, and outputs the final future trajectory of the vehicle.

[0070] Step 3: Multimodal trajectory selection:

[0071] Based on the multimodal trajectory output by the TraLSTM-GAN multimodal prediction module, a trajectory selection method based on probability evaluation is adopted. The candidate trajectory is normalized by a probability normalization function, preferably a softmax function. The target trajectory is selected as the controller input based on the probability evaluation result to generate control commands.

[0072] Step 4, Controller Module:

[0073] The controller module is based on a policy network structure and is implemented using a deep reinforcement learning method in one embodiment, preferably a DQN structure. It is used to generate control commands based on the predicted future trajectory of the vehicle, the vehicle's state information, and traffic environment information to achieve path tracking control.

[0074] The inputs to the controller module include the future trajectory of the vehicle, the vehicle's state information, and the traffic environment information output by the TraLSTM-GAN multimodal prediction module. The future trajectory of the vehicle is represented relative to the vehicle's coordinate system and incorporates traffic scene constraint information. The vehicle is a controlled autonomous vehicle. The vehicle's coordinate system is a local coordinate system established with the vehicle's current position as the origin and the forward direction as the longitudinal axis.

[0075] The controller module selects control commands in a discrete action space based on the input future trajectory of the vehicle, the vehicle's status information, and traffic environment information. The control commands include accelerator pedal opening and steering wheel angle.

[0076] This invention acquires the trajectories of traffic participants and real-time traffic environment information through a traffic scene information acquisition module. The reconstructed path coding module converts the trajectories of traffic participants into reconstructed path coding representations, which are then input into the TraLSTM-GAN multimodal prediction module for the generation, optimization, and filtering of future vehicle trajectories. Finally, the controller module outputs control commands based on the predicted future vehicle trajectories, vehicle status information, and traffic environment information.

[0077] The beneficial effects of this invention are as follows: by introducing path coding representation, a structured expression of multimodal trajectories is realized, improving the accuracy and efficiency of trajectory prediction and enhancing the system's adaptability in complex traffic environments; at the same time, by combining path coding with trajectory prediction and control modules, the consistency between prediction results and control execution is improved, thereby enhancing the path tracking control performance of the autonomous driving system. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0079] Figure 1 This is a schematic diagram of a multimodal trajectory prediction and path tracking control based on path coding according to the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of this application, and not all embodiments. Improvements or modifications made by those skilled in the art without departing from the spirit of the technical solutions of this application are all equivalent implementations of the technical solutions of this application.

[0081] This invention proposes a multimodal trajectory prediction and path tracking control method based on path coding, comprising the following modules: a traffic scene information acquisition module, used to receive traffic scene information and output the trajectories of traffic participants and real-time traffic environment information; a path coding reconstruction module, used to convert the trajectories of traffic participants into reconstructed path coding representations; a TraLSTM-GAN multimodal prediction module, used to receive the reconstructed path coding and real-time traffic environment information as input, generate the future multimodal trajectory of the vehicle, and output the future trajectory of the vehicle within the module through trajectory optimization and trajectory selection processing; and a controller module, used to receive the future trajectory of the vehicle, output vehicle control commands, and realize path tracking control; (Refer to...) Figure 1 The illustration specifically includes the following steps:

[0082] Step 1, Traffic Scene Information Collection Module:

[0083] The traffic scene information acquisition module is responsible for acquiring traffic environment information, which, together with the historical trajectories of traffic participants, constitutes the input data for subsequent modules, including the past data of traffic participants within the traffic scene. The trajectory at each time step: ,in Indicates at time The trajectory points, and , respectively, represent the coordinates of the point in the plane coordinate system. and The coordinates of the direction; wherein, the historical trajectory of the traffic participant is transmitted to the path reconstruction coding module, and the real-time traffic environment information is uniformly represented as condition information. , which serves as the conditional input for the TraLSTM-GAN multimodal prediction module.

[0084] Step 2, Reconstruct the path encoding module design:

[0085] The reconstructed path encoding module is used to perform a structured representation of the trajectories of traffic participants, and receives the trajectories of traffic participants output by the traffic scene information acquisition module in step 1. As input, the output is the reconstructed path code. The path encoding module consists of three parts: (1) a feature curve generation module, (2) a path encoding library, and (3) a path matching module.

[0086] Step 2.1, Design of the characteristic curve generation module:

[0087] The feature curve generation module consists of two parts: (1) a feature curve generation method based on the trajectory of traffic participants; and (2) a feature curve generation method based on road topology. The feature curves generated by the two methods are used as the basis for building the path coding library and for subsequent path matching.

[0088] The method for generating feature curves based on the trajectories of traffic participants is as follows:

[0089] The trajectories of traffic participants in a traffic scenario consist of a continuously sampled sequence of two-dimensional points. in Indicates at time The spatial coordinates; the trajectory segmentation method includes the heading angle change method, the curvature change method, and the velocity and acceleration change method, which are used to divide the trajectory into several characteristic curve segments.

[0090] The heading angle variation method uses the turning angle formed by three consecutive trajectory points to characterize the local directional change characteristics of the trajectory and determine whether the trajectory needs to be segmented. For three consecutive points in a two-dimensional point sequence... The change in its heading angle is calculated as follows:

[0091]

[0092] in Representing Euclidean distance, when Exceeding the set threshold ,Right now ,think These are trajectory segmentation points, used to segment the trajectory. Perform structured segmentation to generate characteristic curves.

[0093] The curvature variation method uses curvature to reflect the degree of bending of the trajectory. At locations where curvature changes significantly, the trajectory is segmented to extract feature curves with geometric semantics. The curvature is calculated as follows:

[0094]

[0095] in, Represents Euclidean distance. The calculation calculates the area of ​​a local triangle formed by three adjacent trajectory points. Exceeding the set threshold At that time, that is ,think These are segmentation points used for dividing a two-dimensional point sequence. Perform structured decomposition to generate feature curves and participate in the construction of the path coding library.

[0096] The speed and acceleration change method analyzes sudden changes in the vehicle's motion state, such as sudden acceleration or deceleration; and calculates the speed. acceleration ,when or Overthreshold and At that time, it was believed These are segmentation points used for trajectory structured decomposition and to generate feature curves for participation in path coding library construction; among them, Indicates the speed threshold. The acceleration threshold is represented by the above segmentation method, which decomposes the original trajectory into several feature curve segments with local geometry and motion semantics, which are used to construct a path coding library.

[0097] The feature curve generation module uses a road topology-based feature curve generation method to efficiently extract representative trajectory segments from high-precision maps without performing complete path planning, in order to construct a standard feature curve set in the path coding library. The specific method is as follows:

[0098] Based on the road centerline information defined in the high-precision map, all lane segments are traversed to extract their centerline coordinate sequences.

[0099] Each centerline is segmented to obtain local trajectory segments of a set length, ensuring that each trajectory segment has local motion semantic features. Trajectory segments that satisfy local geometric semantic constraints are defined as feature curves.

[0100] Step 2.2, Path Encoding Library Construction:

[0101] The feature curve generation module generates feature curves using a feature curve generation method based on traffic participant trajectories and a feature curve generation method based on road topology. The feature curves are then normalized to construct a path coding library, which serves as the search space for the path matching module. The normalization process includes translating, rotating, and scaling each feature curve to represent them in a unified standard coordinate system.

[0102] The translation normalization moves the starting point of each curve to the origin of the coordinate system, so that the starting point positions of each curve are uniformly aligned, thus achieving a unified representation of the spatial reference.

[0103] The rotation normalization adjusts the curves to the same direction, uniformly rotating and aligning the starting direction of each feature curve to the positive direction of the horizontal axis of the coordinate system.

[0104] The scale normalization eliminates the influence of different scales between different curves, enabling the curves to be compared under a unified scale.

[0105] After the above normalization process, each feature curve is converted into a standard form with the origin as the starting point, the same direction, and the same scale. Curves from different sources and curves in different coordinate systems can be analyzed in a unified coordinate system. The set of feature curves constitutes a path coding library.

[0106] Step 2.3, Path Matching Module Design:

[0107] The path matching module performs a search and matching operation based on the path encoding library, and tracks the trajectories of traffic participants. , The mapping is a reconstructed path encoding representation formed by combining multiple feature curve codes. The path encoding library serves as the search space, and the path matching module selects the combination of feature curves with the highest matching degree from the encoding library through a search strategy to approximate the original trajectory.

[0108] Specifically, a characteristic curve segment in each combination Defined as a triple: ,in, This is the index of the feature curve in the encoding library; this index is unique. This segment is encoded using a spatial offset vector to quickly represent the spatial location features of the feature curve. Encode the geometric and dynamic properties of the connection points to characterize the geometric and motion properties of the characteristic curves.

[0109] Traffic participant trajectories Divided The segment, through a matching method, finds the path combination with the minimum matching cost in the encoding library. , making ,in This represents a matrix concatenation operation, where the reconstructed path encoding is used. As an input representation of the TraLSTM-GAN multimodal prediction module, it is used to constrain the multimodal trajectory generation process.

[0110] To find this optimal combination of trajectories in the path coding library, the A* search algorithm is used. This algorithm searches in the path space, which is defined as the set of all possible path coding segments.

[0111] In the path space, each node represents a path-coded segment. The starting point is the initial position of the trajectory. The endpoint is the end position of the trajectory. Cost function as follows:

[0112]

[0113] in This represents the actual cost from the starting point to the current node. This represents the estimated cost from the current node to the target node.

[0114] Cost function The actual cost in and The calculation formula is as follows:

[0115]

[0116] in, This represents the current search node in the path matching search process; and These represent the current matching feature curve and the target feature curve at the [number]th [time]. Encoding of geometric and dynamic properties of each point and These represent the spatial offset vectors of the corresponding points; , The total number of points on the characteristic curve. and These represent the current search node. Corresponding path endpoint coordinates; and These represent the coordinates of the endpoint of the target feature curve; This represents the Euclidean distance.

[0117] Estimated cost Let be a heuristic function representing the estimated matching cost from the current node to the target trajectory endpoint. Starting from the current node, expand all possible path encoding segments. ,according to The value selects the next node until the preset search termination condition is met;

[0118] Finally, the path matching module represents the trajectories of traffic participants as a combination of reconstructed path segments, thus effectively handling the optimal path splicing problem. This combined path is the reconstructed path code. , which serves as the input representation for the TraLSTM-GAN multimodal prediction module.

[0119] Step 3: Design of the TraLSTM-GAN multimodal prediction module:

[0120] The TraLSTM-GAN multimodal prediction module includes a TraLSTM feature extractor, a multimodal trajectory prediction module, a GAN trajectory optimization module, and a trajectory selection module. The TraLSTM feature extractor is used to extract temporal features from real-time traffic environment information to obtain latent feature representations of the traffic environment, and then integrates these representations with the reconstructed path encoding. These are used together as conditional inputs to the multimodal trajectory prediction module; the multimodal trajectory prediction module generates the future multimodal trajectory of the vehicle based on a conditional variational autoencoder; the generated multimodal trajectory is transmitted to the GAN trajectory optimization module for distribution optimization to improve the rationality and diversity of the trajectory; the optimized trajectory is further input to the trajectory selection module, and the final output future trajectory of the vehicle is obtained by filtering according to the set evaluation criteria.

[0121] This method is used to predict the future multimodal trajectories of vehicles based on real-time traffic environment information, and combines reconstructed path coding for constraint expression. The input data includes historical trajectory information of traffic participants, real-time traffic environment information, and reconstructed path coding. ;

[0122] The input information includes traffic participant trajectory information and its corresponding time series features, including: the traffic participants in the past The trajectory at each time step in and The point is located at and Coordinates of direction;

[0123] The formulas for vehicle speed and acceleration are as follows:

[0124]

[0125] in For speed, Represents Euclidean distance. It is a time interval. It is acceleration.

[0126] To constrain the future trajectory generation process, the model introduces conditional information, which includes real-time traffic environment information and reconstructed path coding. Used in the common constraint trajectory generation process, including:

[0127] Lane information, a geometric representation of the lane in which a traffic participant is located. ,in Lane width, These are the left and right boundaries of the lane, respectively.

[0128] Other in the environment The historical trajectory of a traffic participant is represented as follows: , in Indicates position coordinates, Represents the velocity component. Indicates acceleration. This indicates the change in heading angle.

[0129] Vehicles in a traffic light area need to consider the signal status: ,in Red light It represents a green light.

[0130] Considering road speed limits: , This indicates the maximum speed limit for that section of road.

[0131] Driving intention information indicates the vehicle's future actions: , These represent staying in the current lane, changing lanes to the left, and changing lanes to the right, respectively.

[0132] Time information: This includes the specific time, date, and weather.

[0133] Step 3.1, Temporal Feature Extraction Module in the TraLSTM Feature Extractor:

[0134] The TraLSTM feature extractor is used to perform temporal modeling of traffic environment information to extract latent feature representations of the traffic environment. Its hidden state update method is as follows:

[0135]

[0136] Among them, the input feature vector Indicates time step Input of traffic environment characteristics, It is in a hidden state. In module status, This is a recursive update function.

[0137] Step 3.2, Multi-agent interaction modeling module in the TraLSTM feature extractor:

[0138] The multi-agent interaction modeling module is based on the Transformer structure and is used to extract global feature representations of the trajectories of traffic participants;

[0139] For trajectory input Construct the trajectory feature matrix as shown in equation (15):

[0140] (15)

[0141] in It is trajectory point embedding. It is the position code of the trajectory point.

[0142] Calculation in Transformer , , The matrix process and attention mechanism are as follows:

[0143]

[0144] in It is a query, key, and value matrix; It is a learnable transformation matrix; This is a dimension scaling factor; softmax is used to normalize the output weights. The Transformer extracts global time series information through multi-layer self-attention calculation and combines it with the output features of the LSTM to form the TraLSTM feature extractor, transforming the input traffic participant trajectory information into a high-dimensional feature representation. ;

[0145] Step 3.3, Multimodal Trajectory Prediction Module:

[0146] The multimodal trajectory prediction module employs a generative model-based multimodal trajectory generation method, using a Conditional Variational Autoencoder (CVAE) for multimodal trajectory prediction; by introducing latent variables... Modeling the uncertainty of the trajectory, given input conditions. In this context, multiple candidate future trajectories for the vehicle are generated, forming a multimodal trajectory set; during the training phase, latent variables... Approximate posterior distribution by encoder Sampled; latent variables during the inference phase Based on prior distribution The sampled trajectory is obtained; the predicted trajectory is generated by the decoder, as shown in the following formula:

[0147]

[0148] To further understand the above generation process, the variables in the formula are explained below; among them, The input conditions represent historical trajectory, traffic environment information, and reconstructed path coding. ; For future real trajectory point sequence; The set of multimodal trajectories output by the model, where: For the first The future trajectory of the candidate weekly car; As latent variables, It is a posterior distribution; given the conditional input and the true trajectory, it infers possible latent variables. distributed, For decoders, used to generate the first... Candidate predicted trajectories The CVAE loss function is defined as follows, used to constrain the potential distribution and ensure trajectory reconstruction accuracy:

[0149]

[0150] in It is the KL divergence that makes Approximating the prior distribution, The posterior distribution approximated by the encoder network. It is the prior distribution of the latent variable; It is the logarithmic reconstruction probability, i.e., the reconstruction loss, which ensures the generation of the trajectory. Approaching the actual trajectory, It is the probability of the decoder reconstructing the trajectory. As the expected value, The KL divergence is used; CVAE enables the model to predict future trajectories in different modes, generating multimodal trajectories.

[0151] Step 3.4, GAN trajectory optimization module:

[0152] The GAN trajectory optimization module consists of a generator and a discriminator. The generator... Responsible for generating trajectories: Discriminator Assess the authenticity of the trajectory: .

[0153] The objective function of GAN is as follows:

[0154]

[0155] in For generator, For discriminator, The input conditions include historical trajectory information of traffic participants, traffic environment information, and reconstructed path codes. , As latent variables, Sampling for real trajectories, Sampling for latent variables, The output probability of the discriminator for the true trajectory. This represents the output probability of the discriminator for the generated trajectory.

[0156] Through adversarial training, the distribution of the generated optimized trajectory is made to approximate the real trajectory.

[0157] Step 3.5, Track Selection Module:

[0158] The trajectory selection module takes the set of optimized trajectories output by the GAN trajectory optimization module as input, evaluates and filters them using the trajectory probability distribution output by the generative model, and selects the optimal future trajectory from the optimized trajectories.

[0159] In the optimized trajectory output by the GAN trajectory optimization module, the first... The set of candidate future trajectories corresponding to each traffic participant is denoted as: The candidate trajectory probabilities output by the multimodal trajectory prediction module are obtained after normalization. The probability corresponding to each candidate trajectory is: The optimal future trajectory is selected through probability evaluation. The highest probability future trajectory is selected from the multimodal trajectories of each traffic participant, where... For potential representation, As a conditional input, the conditional input Consistent with the previous text.

[0160] Step 4, Controller Module:

[0161] The controller module is composed of a DQN structure based on deep reinforcement learning. The core objective of the controller is to output control commands, given the known vehicle state, future trajectories of surrounding vehicles, and traffic environment information, to achieve accurate tracking of the vehicle's target trajectory. The target trajectory is generated by the upper-level path planning module or determined based on the current lane centerline and navigation target, and is denoted as... .

[0162] The controller's inputs include: vehicle status information. And the optimal future trajectory output by the TraLSTM-GAN multimodal prediction module. Target driving trajectory of the vehicle Based on the characteristics of the scene and environment, the above information is uniformly encoded as part of the state input.

[0163] The controller is in state Input: Discrete action space Actions in Each of the actions This represents a specific throttle-steering combined control command, and the selection of the action depends on the state-action value function estimated by the Q-network. This function is approximated by a neural network.

[0164] The control strategy is as shown in equation (16):

[0165] (16)

[0166] in Indicates at time step The environmental conditions at that time, Indicates at time step At that time, the action selected by the agent, that action is from the action space The goal is to select from among these options to maximize long-term returns.

[0167] To train the Q-network, the cumulative reward is defined as shown in equation (17):

[0168] (17)

[0169] in This serves as a discount factor, controlling the weight of future rewards. For time step The immediate reward represents the feedback that the agent receives after performing an action.

[0170] The objective function is as shown in equation (18):

[0171] (18)

[0172] in, Indicates the state Next, take the optimal action. The maximum reward that can be obtained This indicates the target network parameters to be updated at a fixed period, preventing the target value from changing too quickly. For the next time step All possible actions at that time.

[0173] The Q-network is trained by minimizing the following mean squared error loss function. accomplish:

[0174] (19)

[0175] in Indicates the desired operation, from the experience replay pool. A batch of samples is randomly selected from the pool, and the average error is calculated. (Empirical replay pool) Used to store historical interaction samples to improve training stability For the present The Q-value predicted by the network. For the goal The Q-value predicted by the network.

[0176] reward function The design is intended to guide vehicles to predict trajectories. It enables high-precision, safe, and smooth tracking control. Its configuration can be flexibly adjusted according to specific application scenarios, typically including but not limited to: path deviation; heading angle error; control signal change rate; safe distance from surrounding obstacles or lane boundaries; and whether traffic rules or safety constraints are triggered.

[0177] The combined reward function used is shown in equation (20):

[0178] (20)

[0179] in This represents the lateral offset between the vehicle and the target trajectory. This represents the difference in heading between the vehicle and the target trajectory. Indicates the degree of drastic change in control commands. The minimum distance from surrounding obstacles or lane boundaries; As adjustable weight parameters, each term of the reward function is used to characterize trajectory tracking accuracy, heading consistency, control smoothness, and safety constraints.

[0180] After training is complete, the DQN controller operates using the target trajectory output by the trajectory selection module. With the goal of dynamically tracking and controlling the predicted trajectory, the system perceives the current state at every moment, selects the optimal action, and outputs it to the vehicle's actuators, ensuring trajectory tracking accuracy, safety, and control smoothness.

[0181] In summary, this invention proposes a multimodal trajectory prediction and path tracking control method based on path coding. It combines TraLSTM, CVAE, and GAN to generate multiple candidate future trajectories, and uses a DQN controller to make control decisions based on sensor data and the trajectory of the surrounding vehicle, ultimately achieving high-precision tracking control of the target trajectory. This method effectively improves the model's inference efficiency while ensuring safety and robustness, and can adapt to different environments and dynamic changes, showing broad application prospects, especially in trajectory prediction and path planning in the field of intelligent driving.

Claims

1. A multimodal trajectory prediction and path tracking control method based on path coding, characterized in that, include: The system comprises a traffic scene information acquisition module, a path reconstruction coding module, a TraLSTM-GAN multimodal prediction module, and a controller module. The "self vehicle" refers to a controlled autonomous vehicle, and the "surrounding vehicles" are vehicles located around the self vehicle and participating in traffic operations. The traffic scene information acquisition module receives traffic scene information from feedback, acquires trajectory information of traffic participants, real-time traffic environment information, and self vehicle status information, and transmits the trajectory information of traffic participants to the path reconstruction coding module and the real-time traffic environment information to the TraLSTM-GAN multimodal prediction module. The path reconstruction coding module performs path matching search on the trajectory information of traffic participants, generates a reconstructed path code, and uses this reconstructed path code as the input representation of the TraLSTM-GAN multimodal prediction module. The TraLSTM-GAN multimodal prediction module predicts the future trajectory of the surrounding vehicles based on the real-time traffic environment information and the reconstructed path code, generates the future multimodal trajectory of the surrounding vehicles, optimizes the future multimodal trajectory of the surrounding vehicles through a generative adversarial mechanism, and outputs the future trajectory of the surrounding vehicles to the controller module. The controller module outputs vehicle control commands based on the future trajectory of the vehicle and the vehicle's status information. These control commands act on the vehicle's actuators, enabling the vehicle to operate in the traffic scenario. The traffic scenario information acquisition module then re-acquires the updated traffic scenario information to achieve path tracking control of the vehicle.

2. The multimodal trajectory prediction and path tracking control method based on path coding according to claim 1, characterized in that, The reconstructed path coding module includes a feature curve generation module, a path coding library, and a path matching module. The feature curve generation module generates feature curves and constructs a path coding library based on the feature curves generated by the traffic participant trajectory transmitted by the traffic scene information acquisition module, using a feature curve generation method based on traffic participant trajectory and a feature curve generation method based on road topology. The path matching module retrieves feature curve codes from the path coding library using a path matching method based on a search strategy and matches the traffic participant trajectory into a reconstructed path code formed by combining multiple feature curve codes. The feature curve generation module uses a feature curve generation method based on traffic participant trajectories, which employs segmented points to represent traffic participant trajectories. Segmentation; among which Indicating the first position in the trajectory The two-dimensional position of each point Each point It can be represented as ,in and The point is located at and The coordinates of the direction are calculated as follows: When the change in heading angle is satisfied curvature , , Under any of the following conditions, it is considered that , , , The corresponding time points are the segmentation points, and the trajectory corresponding to the segmentation points is decomposed to generate the characteristic curve; among them, , , , These are the thresholds for heading angle change, curvature, velocity, and acceleration, respectively. Represents Euclidean distance. It is the time interval between adjacent trajectory points, used to represent the time span between two points. It calculates the area of ​​a local triangle formed by three adjacent trajectory points; The feature curve generation method based on road topology structure, based on the lane centerline information in high-precision map, traverses all lane segments, extracts the lane centerline coordinate sequence and performs segmentation processing to obtain feature curves with local motion semantic features. The feature curves generated by the traffic participant trajectory-based feature curve generation method and the road topology-based feature curve generation method together form a path coding library after preprocessing. The path matching module selects feature curve codes from the path encoding library through a search matching command. trajectories of traffic participants Matching for reconstructed path encoding ,in This indicates that the feature curve encoding and concatenation operation is performed sequentially. The search and matching instruction is optimized using the A* algorithm, and its cost function is: in: This represents the actual cost from the starting node to the current node. The heuristic estimation function representing the remaining path from the current node to the target feature curve is a heuristic function; the specific calculation is as follows: in, This represents the current search node in the path matching search process; and These represent the current matching feature curve and the target feature curve at the [number]th [time]. Encoding of geometric and dynamic properties of each point and These represent the spatial offset vectors of the corresponding points and the target feature curves at the th... Spatial offset vector at each corresponding sampling point , The total number of points on the characteristic curve. and These represent the current search node. Corresponding path endpoint coordinates; and These represent the coordinates of the endpoint of the target feature curve; Indicates Euclidean distance; The path matching module represents the trajectory of traffic participants as a combination of feature curve codes, i.e., reconstructed path codes, which serve as the output of the path matching module and as the input of the TraLSTM-GAN multimodal prediction module.

3. The multimodal trajectory prediction and path tracking control method based on path coding according to claim 1, characterized in that: The TraLSTM-GAN multimodal prediction module receives real-time traffic environment information and reconstructed path codes, and outputs the optimized and filtered future trajectories of vehicles. The TraLSTM-GAN multimodal prediction module includes: a TraLSTM feature extractor, a multimodal trajectory prediction module, a GAN trajectory optimization module, and a trajectory selection module. The TraLSTM feature extractor recursively encodes real-time traffic environment information to obtain latent feature representations of the traffic environment. Its hidden state update method is as follows: in It is a time step Traffic environment information; It is in a hidden state. It is the module status. This is a recursive computation function; The attention mechanism in the TraLSTM feature extractor performs multi-agent interaction modeling with the latent feature representation. The query matrix, key matrix, and value matrix are calculated as follows: in It is a query, key, and value matrix. It is a learnable transformation matrix. The input features are obtained by vectorizing the feature representation output by the TraLSTM feature extractor. The feature interaction calculation based on the attention mechanism is as follows: in It is the dimension scaling factor; The multimodal trajectory prediction module receives the traffic environment feature representation output by the TraLSTM feature extractor and the reconstructed path code output by the reconstructed path coding module, and generates a weekly vehicle future multimodal trajectory based on a conditional variational autoencoder, outputting a multimodal trajectory set consisting of multiple candidate weekly vehicle future trajectories; wherein, the conditional input The reconstruction is constructed based on the traffic environment feature representation and the reconstructed path encoding. Represents the future true trajectory sequence. This represents a latent variable used to characterize the uncertainty of future trajectory evolution, and its generation process is as follows: in, , This represents the set of multimodal trajectories consisting of multiple candidate future trajectories of the vehicle output by the multimodal trajectory prediction module, wherein... Indicates the first Candidate paths, Indicates the first The first trajectory corresponding to the The potential representation of subsamples; The generated multimodal trajectory set This includes multiple candidate future vehicle trajectories, with the corresponding loss function being: in, It is a posterior distribution. It is a decoder used to generate predicted trajectories. , Input as a condition; For future real trajectory point sequence; As latent variables, Let KL divergence be such that Approximating the prior distribution; It is the reconstruction loss. It is the probability of the decoder reconstructing the trajectory. This is the expected value; The GAN trajectory optimization module optimizes the distribution of the generated trajectories based on a generative adversarial mechanism, resulting in a multimodal trajectory set. As input to the GAN trajectory optimization module, its adversarial loss function is: in For generator, For discriminator, For conditional input, This represents the predicted trajectory output by the generator, including historical trajectories and traffic environment information. As latent variables, Sampling for real trajectories, Sampling for latent variables; The trajectory selection module uses a multimodal trajectory set composed of optimized trajectories to filter the future trajectory of the vehicle through probability evaluation; The controller module outputs control commands based on the future trajectory of the vehicle and applies them to the traffic scene, so that the traffic scene information changes due to the changes of the vehicle, thereby realizing path tracking control. The control commands include accelerator pedal opening and steering wheel angle.

Images