Multimodal dynamic trajectory prediction method, equipment, and medium based on spatiotemporal invariance.

Abstract

This invention relates to the field of target trajectory prediction, providing a method, device, and medium for multimodal dynamic trajectory prediction based on spatiotemporal invariance. The method includes: constructing a local spatiotemporal reference frame based on the target's historical trajectory, target state, and high-precision map features; generating a spatiotemporal invariant scene representation and caching it for reuse using a spatiotemporal convolutional network and a graph attention network on the local spatiotemporal reference frame; generating dynamic anchor points for the spatiotemporal invariant scene representation using an anchorless recurrent decoder and the local spatiotemporal reference frame; optimizing the dynamic anchor points by combining the spatiotemporal invariant scene representation and collision detection; and generating a multimodal trajectory from the optimized dynamic anchor points through a diffusion model and confidence inference. This invention can significantly reduce computational redundancy in dense scenes and achieve efficient modeling of spatiotemporal scene elements and accurate generation of multimodal trajectories through scene encoding and decoding mechanisms.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and autonomous driving, and more specifically, to a method, device, and medium for multimodal dynamic trajectory prediction based on spatiotemporal invariance. Background Technology

[0002] Target dynamic trajectory prediction, a core technology in intelligent transportation and autonomous driving, aims to accurately predict the future trajectory of a target by fusing historical motion data, scene environment information, and the interactive target state. With the development of deep learning technology, trajectory prediction methods based on spatiotemporal correlation modeling have made significant progress. These techniques, through spatiotemporal attention representation learning of scene elements, provide important insights into solving the problem of multi-target trajectory prediction in heterogeneous traffic scenarios.

[0003] However, existing technologies still face two major bottlenecks:

[0004] First, breakthroughs are urgently needed in real-time processing capabilities for high-density and complex scenarios. Current mainstream attention mechanism models require independent calculation of attention weights for each spatiotemporal scene element, resulting in an exponential increase in computational complexity in scenarios such as dense traffic flow and complex intersections.

[0005] Secondly, multimodal uncertainty modeling in long-term forecasting has inherent flaws. Existing methods generally lack deep integration of spatial prior knowledge such as road feasible area constraints and traffic rules, resulting in a rapid amplification of displacement errors in long-term forecasts over time. Summary of the Invention

[0006] The present invention aims to provide a multimodal dynamic trajectory prediction method, device and medium based on spatiotemporal invariance, in order to solve the problems of low real-time inference efficiency of existing trajectory prediction technology in high-density complex scenarios and insufficient accuracy of multimodal trajectory in long-term prediction.

[0007] In a first aspect, the present invention provides a multimodal dynamic trajectory prediction method based on spatiotemporal invariance, comprising:

[0008] Based on the target's historical trajectory, target status, and high-precision map features, a local spatiotemporal reference frame is constructed.

[0009] Spatiotemporal convolutional networks and graph attention networks are used for local spatiotemporal reference frames to generate spatiotemporally invariant scene representations and cache them for reuse.

[0010] For spatiotemporally invariant scene representation, dynamic anchor points are generated using an anchorless cyclic decoder and local spatiotemporal reference frames.

[0011] The dynamic anchor points are optimized by combining spatiotemporal invariant scene representation and collision detection;

[0012] For the optimized dynamic anchor points, multimodal trajectories are generated through diffusion models and confidence inference.

[0013] In a preferred embodiment, constructing a local spatiotemporal reference frame based on the target's historical trajectory, target state, and high-precision map features includes:

[0014] Using the target's current position as the origin, a local coordinate system is established based on the current heading angle. The target's historical trajectory in the global coordinate system is then transformed to the local coordinate system through translation and rotation.

[0015] The transformed target historical trajectory, target state, and high-precision map features are stitched together in a local coordinate system to obtain a local spatiotemporal reference frame.

[0016] In a preferred embodiment, the step of using a spatiotemporal convolutional network and a graph attention network on the local spatiotemporal reference frame to generate a spatiotemporally invariant scene representation and cache it for reuse includes:

[0017] Spatiotemporal convolutional networks and graph attention networks are used to extract multi-scale features from local spatiotemporal reference frames;

[0018] Multi-scale features extracted by spatiotemporal convolutional networks and graph attention networks are encoded as spatiotemporally invariant scene representations.

[0019] A sliding window is used to cache the spatiotemporally invariant scene representations of the most recent windows; the spatiotemporally invariant scene representations include dynamic target interaction features and static map constraint features.

[0020] In a preferred embodiment, incremental encoding is performed only on newly added dynamic targets within the window under a local spatiotemporal reference frame, reusing static map constraint features.

[0021] In a preferred embodiment, the characterization of the spatiotemporally invariant scene, utilizing an anchorless cyclic decoder and local spatiotemporal reference frames to generate dynamic anchors, includes:

[0022] Convert the coordinates of trajectory points under the local spatiotemporal reference frame to polar coordinates;

[0023] After concatenating the polar coordinates with the spatiotemporally invariant scene representation, the input is given to the anchorless loop decoder to generate polar coordinate path points for several future time steps segment by segment.

[0024] The polar coordinate path points are converted back to coordinates under the local spatiotemporal reference frame and used as dynamic anchor points.

[0025] In a preferred embodiment, the anchorless cyclic decoder consists of a two-layer long short-term memory network.

[0026] In a preferred embodiment, the optimization of dynamic anchor points by combining spatiotemporal invariant scene representation and collision detection includes:

[0027] Based on the static map constraint features in the spatiotemporal invariant scene representation, a binary mask matrix is ​​generated under the local spatiotemporal reference frame;

[0028] The safety of all dynamic anchors and targets is verified by a collision detection function, invalid dynamic anchors are eliminated and valid dynamic anchors are retained.

[0029] The retained dynamic anchor points are optimized based on the binary mask matrix. During optimization, the coordinate reference of the local spatiotemporal reference frame and the spatiotemporal invariant scene representation constraints are combined to minimize the geometric deviation, and the optimization process is constrained by cross-entropy loss and geometric loss.

[0030] In a preferred embodiment, generating multimodal trajectories from the optimized dynamic anchor points using a diffusion model and confidence inference includes:

[0031] The optimized dynamic anchor points are injected with random noise to generate diffusion starting points;

[0032] By combining the spatiotemporal invariant scene representation, the diffusion starting point is gradually denoised through the denoising network of the diffusion model to generate candidate trajectories;

[0033] Based on the spatiotemporal invariant scene representation, and taking the consistency of the optimized dynamic anchor points as the benchmark, the degree of matching between the candidate trajectory and the static map constraints is measured to obtain the confidence score of each candidate trajectory.

[0034] Based on the confidence score, output a set of multimodal trajectories containing several trajectories.

[0035] In a second aspect, the present invention provides an electronic device, comprising:

[0036] At least one processor; and a memory communicatively connected to said at least one processor;

[0037] The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the above-described method.

[0038] Thirdly, the present invention provides a computer-readable storage medium for storing instructions that, when executed, enable the above-described method to be implemented.

[0039] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0040] 1. This invention performs translation / rotation invariance modeling on scene elements using local spatiotemporal reference frames, and further combines sliding window caching and incremental encoding mechanisms to significantly reduce computational redundancy in dense scenes, thereby significantly shortening the single-frame inference time.

[0041] 2. The polar coordinate transformation of this invention endows the model with rotation invariance characteristics. Combined with the drivable area mask matrix and collision detection function, the dynamic anchor point trajectory is accurately optimized under scene constraints. Finally, through the multimodal trajectory generation mechanism, multiple potential modes of target motion are accurately captured, improving the robustness of the prediction results. Attached Figure Description

[0042] Figure 1 The flowchart shows a multimodal dynamic trajectory prediction method based on spatiotemporal invariance provided by the present invention.

[0043] Figure 2 This is a schematic diagram illustrating the implementation principle of a multimodal dynamic trajectory prediction method based on spatiotemporal invariance, provided in an embodiment of the present invention.

[0044] Figure 3 This is a schematic diagram of the structure of an electronic device provided by the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0046] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0047] To address the issues of low real-time inference efficiency in high-density and complex scenarios, and insufficient accuracy of multimodal trajectories in long-term prediction, existing trajectory prediction technologies provide a spatiotemporally invariant multimodal dynamic trajectory prediction method. This method utilizes scene encoding and decoding mechanisms to achieve efficient modeling of spatiotemporal scene elements and accurate generation of multimodal trajectories. Figure 1 As shown, the specific steps include the following:

[0048] S100. Construct a local spatiotemporal reference frame based on the target's historical trajectory, target status, and high-precision map features;

[0049] S200. Spatiotemporal convolutional networks and graph attention networks are used for local spatiotemporal reference frames to generate spatiotemporally invariant scene representations and cache them for reuse.

[0050] S300, for spatiotemporally invariant scene representation, uses an anchorless cyclic decoder and local spatiotemporal reference frames to generate dynamic anchors;

[0051] S400, combining spatiotemporal invariant scene representation and collision detection, optimizes dynamic anchor points;

[0052] S500 generates multimodal trajectories for the optimized dynamic anchor points through diffusion models and confidence inference.

[0053] like Figure 2 As shown, the present invention will be described in detail below through specific implementation processes.

[0054] In this embodiment, step S100 constructs a local spatiotemporal reference frame through coordinate transformation and feature fusion. Specifically, it includes:

[0055] S101. Using the target's current position as the origin, establish a local coordinate system based on the current heading angle. Transform the target's historical trajectory in the global coordinate system to the local coordinate system through translation and rotation to eliminate interference caused by translation and rotation in the global coordinate system. The calculation formula is expressed as follows:

[0056]

[0057] in, The current heading angle, The target's current position, The first in the target's historical trajectory in the global coordinate system A trajectory point, For the target's historical trajectory in the local coordinate system after transformation, the first... A trajectory point.

[0058] S102, Transform the target's historical trajectory and target state (velocity) acceleration The high-precision map features (lane lines, intersection polygons) are stitched together in the local coordinate system to obtain a local spatiotemporal reference frame. The calculation formula is expressed as follows:

[0059]

[0060] in, For local spatiotemporal reference frames, The number of trajectory points in the historical trajectory of the target table; For the first The speed of the target For the first The acceleration of each target For the target quantity; It represents high-precision map features in a local coordinate system.

[0061] In this embodiment, step S200 performs feature extraction and encoding under the local spatiotemporal reference frame constructed in step S100, specifically including:

[0062] S201, Local spatiotemporal reference frame We employ a spatio-temporal-convolutional network (ST-Conv) and a graph attention network (GAT) to extract multi-scale features.

[0063] (1) Using a spatiotemporal convolutional network to process local spatiotemporal reference frames The formula for extracting multi-scale features is expressed as follows:

[0064]

[0065] in, This indicates that the spatiotemporal convolutional network applies local spatiotemporal reference frames. Extracted multi-scale features t Used to identify different time steps of the trajectory in time. The convolutional kernel weights are used as parameters. The network enhances feature propagation capabilities through residual connections, ensuring effective capture of offsets at different time steps within the coordinate system of the local spatiotemporal reference frame. Spatial Scale The characteristic information.

[0066] (2) The graph attention network is based on the target positional relationships in the local spatiotemporal reference frame. It models the interaction process between targets through an 8-head attention mechanism. The calculation formula is expressed as follows:

[0067]

[0068]

[0069]

[0070] in, This indicates that the graph attention network is applied to local spatiotemporal reference frames. Extracted multi-scale features Indicate target For the goal The original attention score, target The goal The neighboring nodes, Indicate target For attention nodes k The original attention score, Indicate target With all attention nodes k The original set of attention scores. The target in the graph attention network. For the goal Attention weights Indicate target The set of neighboring nodes, Indicate target eigenvectors, Indicate target eigenvectors, The total number of attention nodes. For learnable weight matrix, This indicates that the data is concatenated according to the feature dimension.

[0071] S203. Multi-scale features extracted based on spatiotemporal convolutional networks and graph attention networks are encoded as spatiotemporally invariant scene representations, and the calculation formula is expressed as follows:

[0072]

[0073] in, Representation of spatiotemporal invariance scene characteristics This represents the join function.

[0074] S204. Utilize a sliding window to cache recent data. Spatiotemporal invariance scene representation of a window The spatiotemporal invariant scene representation This includes dynamic target interaction features and static map constraint features. When a new data frame arrives, only newly added dynamic targets within the window are incrementally encoded under a local spatiotemporal reference frame, reusing static map constraint features. Specifically, after encoding the new dynamic target as a spatiotemporally invariant scene representation, an optimal transmission algorithm is used to calculate its optimal match with historically cached spatiotemporally invariant scene representations. The optimal matching matrix is ​​then solved using the Hungarian algorithm, achieving alignment between dynamic target interaction features and static map constraint features, significantly reducing redundant computation.

[0075] In this embodiment, step S300 is characterized by the spatiotemporal invariant scene output in step S200. Using the local spatiotemporal reference frame constructed in step S100 as the core input, dynamic anchor point generation is carried out, specifically including:

[0076] S301. Convert the coordinates of the trajectory points under the local spatiotemporal reference frame to polar coordinates:

[0077]

[0078] in, The radial distance of the target relative to the origin. This is the azimuth angle relative to the horizontal axis of the reference frame.

[0079] S302, Representing the scene with polar coordinates and spatiotemporal invariance. After concatenation, the data is input into a non-anchored recurrent decoder consisting of two layers of Long Short-Term Memory (LSTM) networks to generate future segments sequentially. The polar coordinate path points at each time step are given by the formula:

[0080]

[0081] in, For the current time step (the 1st time step) The polar coordinate path points output by the time step. For the previous time step (the first time step) The output of the time step.

[0082] S303. Convert the polar coordinate path points back to coordinates under the local spatiotemporal reference frame, and use them as dynamic anchor points. The calculation formula is as follows:

[0083]

[0084] in, The coordinates of the dynamic anchor point in the local spatiotemporal reference frame are used to ensure consistency with the constraints of the static map.

[0085] In this embodiment, step S400 performs trajectory optimization based on the dynamic anchor points generated in step S300 and the scene representation cached in step S200, specifically including:

[0086] S401. Based on the static map constraint features in the spatiotemporal invariant scene representation, a binary mask matrix is ​​generated under the local spatiotemporal reference frame. The calculation formula is expressed as follows:

[0087]

[0088] in, For dynamic anchor points under local spatiotemporal reference frames, For this dynamic anchor point binary mask matrix, This indicates that the position of the dynamic anchor point conforms to the constraints of the static map.

[0089] S402. Verify the safety of all dynamic anchor points and the target using a collision detection function, discard invalid dynamic anchor points and retain valid ones, using the following formula:

[0090]

[0091] in, For dynamic anchor points under local spatiotemporal reference frames, This refers to the set of positions of dynamic targets in a space-time invariant scene representation. For the safety radius, To calculate the Euclidean distance; This represents the collision coefficient between the dynamic anchor point and the dynamic target. If the condition is met, the dynamic anchor point is marked as invalid and filtered; otherwise, the dynamic anchor point is marked as valid and retained.

[0092] S403. Optimize the retained dynamic anchor points based on the binary mask matrix; during optimization, minimize the geometric deviation by combining the coordinate reference of the local spatiotemporal reference frame with the spatiotemporal invariant scene representation constraints. The optimization formula is expressed as:

[0093]

[0094] in, It is an optimized dynamic anchor point. This is the set of feasible path points under a local spatiotemporal reference frame. It is a penalty coefficient that minimizes the distance between the original dynamic anchor point and the feasible region, as well as the penalty for prohibited regions, to ensure the binary mask matrix of the optimized trajectory. Satisfies the requirements and avoids collisions.

[0095] The optimization process employs cross-entropy loss and geometric loss constraints, and the loss function is expressed as follows:

[0096]

[0097] in, For the total loss, To calculate the cross-entropy loss between the predicted trajectory and the actual trajectory, This is the geometric loss based on Chamfer distance (a measure of deviation from the drivable area). This is the balance coefficient.

[0098] In this embodiment, step S500 uses the optimized dynamic anchor points from step S400 as initial input and combines them with the spatiotemporal invariant scene representation from S200 to generate a multimodal trajectory set, specifically including:

[0099] S501. Inject the optimized dynamic anchor points into random noise to generate diffusion start points. The noise injection formula is as follows:

[0100]

[0101] in, It is the first The diffusion start point of the diffusion step. For the first The noise figure of the diffusion step follows a cosine variance timescale. For standard normal noise, This is the optimized dynamic anchor point.

[0102] S502. Combining the spatiotemporal invariant scene representation, the diffusion initiation point is progressively denoised using a denoising network of the diffusion model to generate candidate trajectories. The denoising formula is expressed as:

[0103]

[0104] in, It is the predicted noise vector. UNet For noise reduction networks, Indicates the starting point of diffusion Representation of Space-Time Invariance Scenes splicing, Number of diffusion steps (total number of steps) ),through After denoising, the generated Candidate trajectories .

[0105] S503. Based on the spatiotemporal invariant scene representation, and using the consistency of the optimized dynamic anchor points as a benchmark, the degree of matching between the candidate trajectory and the static map constraints is measured to obtain the confidence score of each candidate trajectory, expressed by the formula:

[0106]

[0107] in, For the first Candidate trajectories The confidence score, For the first The candidate trajectory in the 1st The trajectory points of the time step, For the first The candidate trajectory in the 1st Trajectory points at each time step, To calculate the Euclidean distance, the consistency is quantified by calculating the average minimum distance between trajectories. The larger the value, the higher the matching degree of the trajectory with other modal trajectories, and the stronger the reliability of conforming to the static map constraints.

[0108] S504, Based on confidence scores, the output includes Multimodal trajectory set of trajectories This provides diverse and reliable predictive results for subsequent decision-making.

[0109] It should be noted that in the spatiotemporal invariance-based multimodal dynamic trajectory prediction method provided by this invention, step S100 constructs a local spatiotemporal reference frame with the target's current position as the origin, achieving decoupling between scene elements and global coordinates and providing a unified coordinate benchmark for subsequent extraction. Based on this, step S200 combines spatiotemporal convolution and graph attention mechanisms to extract multi-scale features, caches and reuses historical stable features (such as static map constraint features) through a sliding window, and incrementally encodes newly added dynamic targets, significantly reducing redundant computation in dense traffic scenarios and significantly improving online inference efficiency. Step S300 uses polar coordinate transformation to generate rotation-invariant radial distance and azimuth features, and combines this with an anchor-free cyclic decoder to generate dynamic anchor points segment by segment, enhancing the model's generalization ability for rotational movements such as target turning and U-turns. Step S400 optimizes the dynamic anchor point trajectory based on the binary mask matrix of the drivable area and a collision detection function, filtering invalid points through point-by-point geometric verification to ensure the trajectory compliance rate meets requirements (e.g., trajectory and pattern ≥ 95%), effectively solving the pattern collapse and compliance issues in long-term prediction. Finally, through step S500, the optimized dynamic anchor points are used to generate a multimodal trajectory set through diffusion model and confidence inference, thereby achieving accurate capture of multiple potential modes of target motion.

[0110] This method, through the collaborative innovation of scene encoding and decoding mechanisms, fully meets the actual needs of autonomous driving for low-latency and high-precision trajectory prediction: local spatiotemporal reference frames and cache reuse mechanisms ensure real-time processing capabilities in dense scenes, while rotation invariance features and scene constraint optimization improve the accuracy and reliability of long-term prediction. It achieves efficient and accurate prediction of multi-target trajectories in complex traffic environments, which is of great significance for improving the safety and decision reliability of intelligent transportation systems and has strong practicality.

[0111] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the multimodal dynamic trajectory prediction method based on spatiotemporal invariance provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic devices. Figure 3 As shown, the electronic device may include:

[0112] At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 3 The example used is the connection between the processor and memory via a bus. The bus... Figure 3 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 3The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0113] In this embodiment of the invention, the memory stores instructions that can be executed by at least one processor. By executing the instructions stored in the memory, at least one processor can execute the multimodal dynamic trajectory prediction method based on spatiotemporal invariance discussed above.

[0114] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0115] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0116] The processor can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the spatiotemporally invariant multimodal dynamic trajectory prediction method disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0117] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. In embodiments of the present invention, memory can also be a circuit or any other device capable of implementing storage functions, used to store program instructions and / or data.

[0118] By designing and programming the processor, the code corresponding to the multimodal dynamic trajectory prediction method based on spatiotemporal invariance described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the method described in the foregoing embodiments during runtime. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0119] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform a multimodal dynamic trajectory prediction method based on spatiotemporal invariance as described above.

[0120] In some alternative embodiments, the present invention also provides that various aspects of a spatiotemporally invariant multimodal dynamic trajectory prediction method can also be implemented as a program product comprising program code that, when the program product is run on a device, causes the control device to perform the steps in the spatiotemporally invariant multimodal dynamic trajectory prediction method according to various exemplary embodiments of the present invention described above.

[0121] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0122] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0125] In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal dynamic trajectory prediction method based on spatiotemporal invariance, characterized in that, include: Based on the target's historical trajectory, target status, and high-precision map features, a local spatiotemporal reference frame is constructed. Spatiotemporal convolutional networks and graph attention networks are used for local spatiotemporal reference frames to generate spatiotemporally invariant scene representations and cache them for reuse. For spatiotemporally invariant scene representation, dynamic anchor points are generated using an anchorless cyclic decoder and local spatiotemporal reference frames. The dynamic anchor points are optimized by combining spatiotemporal invariant scene representation and collision detection; For the optimized dynamic anchor points, multimodal trajectories are generated through diffusion models and confidence inference; The process of using a spatiotemporal convolutional network and a graph attention network on local spatiotemporal reference frames to generate spatiotemporally invariant scene representations and cache and reuse them includes: Spatiotemporal convolutional networks and graph attention networks are used to extract multi-scale features from local spatiotemporal reference frames; Multi-scale features extracted by spatiotemporal convolutional networks and graph attention networks are encoded as spatiotemporally invariant scene representations. A sliding window is used to cache the spatiotemporally invariant scene representations of the most recent windows; the spatiotemporally invariant scene representations include dynamic target interaction features and static map constraint features.

2. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1, characterized in that, The construction of a local spatiotemporal reference frame based on the target's historical trajectory, target state, and high-precision map features includes: Using the target's current position as the origin, a local coordinate system is established based on the current heading angle. The target's historical trajectory in the global coordinate system is then transformed to the local coordinate system through translation and rotation. The transformed target historical trajectory, target state, and high-precision map features are stitched together in a local coordinate system to obtain a local spatiotemporal reference frame.

3. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1, characterized in that, Only newly added dynamic targets within the window are incrementally encoded in the local spatiotemporal reference frame, reusing the constraint features of the static map.

4. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1, characterized in that, The spatiotemporally invariant scene representation utilizes an anchorless cyclic decoder and local spatiotemporal reference frames to generate dynamic anchor points, including: Convert the coordinates of trajectory points under the local spatiotemporal reference frame to polar coordinates; After concatenating the polar coordinates with the spatiotemporally invariant scene representation, the input is given to the anchorless loop decoder to generate polar coordinate path points for several future time steps segment by segment. The polar coordinate path points are converted back to coordinates under the local spatiotemporal reference frame and used as dynamic anchor points.

5. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 4, characterized in that, The anchorless cyclic decoder consists of two layers of long short-term memory network.

6. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1, characterized in that, The optimization of dynamic anchor points by combining spatiotemporal invariant scene representation and collision detection includes: Based on the static map constraint features in the spatiotemporal invariant scene representation, a binary mask matrix is ​​generated under the local spatiotemporal reference frame; The safety of all dynamic anchors and targets is verified by a collision detection function, invalid dynamic anchors are eliminated and valid dynamic anchors are retained. The retained dynamic anchor points are optimized based on the binary mask matrix. During optimization, the coordinate reference of the local spatiotemporal reference frame and the spatiotemporal invariant scene representation constraints are combined to minimize the geometric deviation, and the optimization process is constrained by cross-entropy loss and geometric loss.

7. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 6, characterized in that, The process of generating multimodal trajectories from optimized dynamic anchor points using diffusion models and confidence inference includes: The optimized dynamic anchor points are injected with random noise to generate diffusion starting points; By combining the spatiotemporal invariant scene representation, the diffusion starting point is gradually denoised through the denoising network of the diffusion model to generate candidate trajectories; Based on the spatiotemporal invariant scene representation, and taking the consistency of the optimized dynamic anchor points as the benchmark, the degree of matching between the candidate trajectory and the static map constraints is measured to obtain the confidence score of each candidate trajectory. Based on the confidence score, output a set of multimodal trajectories containing several trajectories.

8. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which executes the instructions stored in the memory to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, cause the method as described in any one of claims 1-7 to be implemented.

Images