Multi-modal dynamic trajectory prediction method and device based on space-time invariance and medium

Abstract

The invention relates to the field of target trajectory prediction, and provides a multi-modal dynamic trajectory prediction method and device based on space-time invariance and a medium, and the method comprises the steps: constructing a local space-time reference frame based on a target historical trajectory, a target state and a high-precision map feature; a space-time convolutional network and a graph attention network are adopted for the local space-time reference frame, and space-time invariance scene representation is generated and cached for reuse; representing a space-time invariance scene, and generating a dynamic anchor point by using an anchor-point-free cyclic decoder and a local space-time reference frame; optimizing the dynamic anchor point by combining space-time invariance scene representation and collision detection; and for the optimized dynamic anchor points, generating a multi-modal trajectory through a diffusion model and confidence reasoning. According to the method, the calculation redundancy in a dense scene can be greatly reduced, and efficient modeling of space-time scene elements and accurate generation of a multi-modal track are realized through a scene encoding and decoding mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent transportation and automatic driving, in particular to a multi-modal dynamic trajectory prediction method based on space-time invariance, equipment and medium. BACKGROUND

[0002] As a core technology in the field of intelligent transportation and automatic driving, the core task of target dynamic trajectory prediction is to accurately predict the future trajectory of the target by fusing the target historical motion data, scene environment information and interactive target state. With the development of deep learning technology, trajectory prediction methods based on space-time correlation modeling have made significant progress. Such technology learns the space-time attention representation of scene elements, providing an important idea for solving the problem of multi-target trajectory prediction in heterogeneous traffic scenes.

[0003] However, there are still two major bottlenecks in the existing technology: First, the real-time processing capability for high-density complex scenes needs to be broken through. The current mainstream attention mechanism model needs to calculate the attention weight for each space-time scene element independently, resulting in an exponential increase in computational complexity in dense traffic, complex intersections and other scenes; Second, there are inherent defects in the modeling of multi-modal uncertainty in long-term prediction. Existing methods generally lack deep fusion of spatial prior knowledge such as road feasible region constraints and traffic rules, resulting in rapid amplification of displacement error in long-term prediction. SUMMARY

[0004] The present application aims to provide a multi-modal dynamic trajectory prediction method based on space-time invariance, equipment and medium to solve the problems of low real-time inference efficiency of existing trajectory prediction technology in high-density complex scenes and insufficient accuracy of multi-modal trajectory in long-term prediction.

[0005] In a first aspect, the present application provides a multi-modal dynamic trajectory prediction method based on space-time invariance, comprising: Based on the target historical trajectory, target state and high-precision map features, a local space-time reference frame is constructed; A space-time convolution network and a graph attention network are used for the local space-time reference frame to generate space-time invariance scene representation and cache reuse; For the space-time invariance scene representation, a non-anchor point recurrent decoder and a local space-time reference frame are used to generate dynamic anchor points; The dynamic anchor points are optimized in combination with the space-time invariance scene representation and collision detection; The optimized dynamic anchor points are used to generate multi-modal trajectories through a diffusion model and confidence reasoning.

[0006] In a preferred embodiment, the constructing a local space-time reference frame based on the target historical trajectory, the target state and the high-precision map features comprises: Taking the target current position as the origin, a local coordinate system is established based on the current heading angle, and the target historical trajectory in the global coordinate system is transformed into the local coordinate system through translation and rotation; The transformed target historical trajectory, the target state and the high-precision map features are spliced in the local coordinate system to obtain the local space-time reference frame.

[0007] In a preferred embodiment, the local space-time reference frame is processed by a space-time convolution network and a graph attention network to generate a space-time invariance scene representation and cache reuse, comprising: The local space-time reference frame is processed by a space-time convolution network and a graph attention network to extract multi-scale features; Based on the multi-scale features extracted by the space-time convolution network and the graph attention network, a space-time invariance scene representation is encoded; The space-time invariance scene representation includes dynamic target interaction features and static map constraint features.

[0008] In a preferred embodiment, only the newly added dynamic target in the window is incrementally encoded in the local space-time reference frame, and the static map constraint features are reused.

[0009] In a preferred embodiment, the space-time invariance scene representation is processed by an anchor-free recurrent decoder and a local space-time reference frame to generate a dynamic anchor point, comprising: The coordinates of the trajectory points in the local space-time reference frame are converted into polar coordinates; After the polar coordinates are spliced with the space-time invariance scene representation, the polar coordinates are input into the anchor-free recurrent decoder to generate polar coordinate path points for a plurality of time steps; The polar coordinate path points are converted back to coordinates in the local space-time reference frame as dynamic anchor points.

[0010] In a preferred embodiment, the anchor-free recurrent decoder is composed of two layers of long short-term memory networks.

[0011] In a preferred embodiment, the dynamic anchor point is optimized in combination with the space-time invariance scene representation and collision detection, comprising: Based on the static map constraint features in the space-time invariance scene representation, a binary mask matrix is generated in the local space-time reference frame; Through a collision detection function, the safety of all dynamic anchor points and the target is checked, and invalid dynamic anchor points are removed and valid dynamic anchor points are retained; The reserved dynamic anchor points are optimized based on a binary mask matrix; during optimization, geometric deviation is minimized by combining the coordinate reference of the local space-time reference frame and the space-time invariance scene representation constraint, and a cross-entropy loss and a geometric loss are used to constrain the optimization process.

[0012] In a preferred embodiment, the optimized dynamic anchor points are used to generate multi-modal trajectories by a diffusion model and confidence reasoning, including: The optimized dynamic anchor points are injected into random noise to generate diffusion starting points; The diffusion starting points are gradually denoised by a denoising network of the diffusion model to generate candidate trajectories in combination with the space-time invariance scene representation; Based on the space-time invariance scene representation, the consistency of the optimized dynamic anchor points is used as a reference to measure the matching degree of the candidate trajectories and the static map constraint to obtain a confidence score of each candidate trajectory; Based on the confidence score, a multi-modal trajectory set containing several trajectories is output.

[0013] In a second aspect, the present application provides an electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; The memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory, so that the at least one processor executes the above-mentioned method.

[0014] In a third aspect, the present application provides a computer-readable storage medium for storing instructions, when the instructions are executed, the above-mentioned method is realized.

[0015] As described above, due to the adoption of the above technical solutions, the present application has the following advantages: 1. The present application models the translational / rotational invariance of scene elements by a local space-time reference frame, further combined with a sliding window buffer and an incremental encoding mechanism, which greatly reduces the computational redundancy in dense scenes, significantly shortening the single-frame inference time.

[0016] 2. The polar coordinate transformation of the present application gives the model rotational invariance, combined with the drivable area mask matrix and the collision detection function, so that the dynamic anchor point trajectory is accurately optimized under the scene constraint; finally, through the multi-modal trajectory generation mechanism, the multiple potential modes of target motion are accurately captured, and the robustness of the prediction result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of a multi-modal dynamic trajectory prediction method based on space-time invariance provided by the present application.

[0018] Figure 2 An implementation principle diagram of a multi-modal dynamic trajectory prediction method based on space-time invariance is provided for an embodiment of the present application.

[0019] Figure 3 A structural schematic diagram of an electronic device is provided for the present application. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0022] In view of the problems that the existing trajectory prediction technology has low real-time inference efficiency in high-density complex scenes, and the multi-modal trajectory precision is insufficient in long-time prediction, an embodiment of the present application provides a multi-modal dynamic trajectory prediction method based on space-time invariance. Through a scene encoding and decoding mechanism, efficient modeling of space-time scene elements and accurate generation of multi-modal trajectories are realized, as shown in Figure 1 The specific steps include the following steps: S100, constructing a local space-time reference frame based on a target historical trajectory, a target state and high-precision map features; S200, generating space-time invariance scene representation and caching reuse by using a space-time convolution network and a graph attention network on the local space-time reference frame; S300, generating dynamic anchor points by using an anchor-free recurrent decoder and the local space-time reference frame on the space-time invariance scene representation; S400, optimizing the dynamic anchor points in combination with the space-time invariance scene representation and collision detection; S500, generating multi-modal trajectories by a diffusion model and confidence reasoning on the optimized dynamic anchor points.

[0023] As shown in Figure 2 The present application will be described in detail below through a specific implementation process.

[0024] In the present embodiment, the step S100 constructs a local space-time reference frame by coordinate transformation and feature fusion. Specifically, it includes the following steps: S101, taking the target current position as an original point, establishing a local coordinate system based on a current heading angle, transforming the target historical trajectory under a global coordinate system to the local coordinate system through translation and rotation to eliminate interference caused by translation and rotation under the global coordinate system, and a calculation formula is represented as:

[0025] wherein, is the current heading angle, is the target current position, is the i-th trajectory point in the target historical trajectory under the global coordinate system, is the i-th trajectory point in the target historical trajectory under the local coordinate system after transformation.

[0026] S102, splicing the transformed target historical trajectory and target state (speed , acceleration ) and high-precision map features (lane line, intersection polygon) under the local coordinate system to obtain a local space-time reference frame, and a calculation formula is represented as:

[0027] wherein, is the local space-time reference frame, is the number of trajectory points in the target historical trajectory; is the speed of the i-th target, is the acceleration of the i-th target, is the number of targets; is the high-precision map feature under the local coordinate system. In the embodiment, step S200 carries out feature extraction and coding work under the local space-time reference frame constructed in step S100, specifically including: S201, for the local space-time reference frame

[0028] , a spatio-temporal convolution network (Spatio-Temporal-Conv, ST-Conv) and a graph attention network (Graph Attention Networks, GAT) are used to extract multi-scale features. (1) the calculation formula for extracting multi-scale features of the local space-time reference frame by the spatio-temporal convolution network is represented as:

[0029]

[0030] wherein, represents the multi-scale features of the local space-time reference frame​​​​ extracted multi-scale features, t for identifying different time steps of the trajectory in time, for the convolution kernel weight parameters, the network enhances the feature propagation ability through the residual connection, ensuring that in the coordinate system of the local spatio-temporal reference frame, different time step offsets are effectively captured and spatial scales feature information.

[0031] (2) The graph attention network models the interaction process between targets based on the target position relationship in the local spatio-temporal reference frame through 8 attention mechanisms, and the calculation formula is represented as:

[0032]

[0033]

[0034] wherein, denotes the multi-scale features extracted by the graph attention network in the local spatio-temporal reference frame extracted multi-scale features, denotes the target original attention score of the target , the target is the neighbor node of the target , and the target denotes the original attention score of the target for the attention node k , and the target denotes the original attention score set of the target and all attention nodes k , the target attention weight of the target for the target , the target denotes the neighbor node set of the target , the target denotes the feature vector of the target , the target denotes the feature vector of the target , the total number of attention nodes is , the learnable weight matrix is denotes the concatenation according to the feature dimension.

[0035] S203, based on the multi-scale features extracted by the spatio-temporal convolution network and the graph attention network, the multi-scale features are encoded into a spatio-temporally invariant scene representation, and the calculation formula is represented as:

[0036] wherein, representing the spatio-temporal invariance scene representation, representing the connection function.

[0037] S204, using a sliding window to cache the spatio-temporal invariance scene representation of the last windows. The spatio-temporal invariance scene representation includes dynamic target interaction features and static map constraint features. When a new data frame arrives, only the newly added dynamic target in the window is incrementally encoded in the local spatio-temporal reference frame, and the static map constraint features are reused. Specifically, after the newly added dynamic target is encoded into a spatio-temporal invariance scene representation, an optimal transport algorithm is used to calculate the optimal matching between the spatio-temporal invariance scene representation and the historical cached spatio-temporal invariance scene representation, and a Hungarian algorithm is used to solve the optimal matching matrix, so as to align the dynamic target interaction features and the static map constraint features, and significantly reduce the redundant calculation amount.

[0038] In this embodiment, step S300 takes the spatio-temporal invariance scene representation output by step S200 as the core input, and performs dynamic anchor point generation work in combination with the local spatio-temporal reference frame constructed in step S100. Specifically, it includes: S301, converting the coordinates of the trajectory points in the local spatio-temporal reference frame into polar coordinates:

[0039] wherein, is the radial distance of the target relative to the origin, is the azimuth angle relative to the horizontal axis of the reference frame.

[0040] S302, after splicing the polar coordinates and the spatio-temporal invariance scene representation , inputting into a anchor-free recurrent decoder composed of a two-layer long short-term memory (LSTM) network to generate polar coordinate path points for the future time steps in segments, and the formula is:

[0041] wherein, is the polar coordinate path point output at the current time step (the time step), is the output at the previous time step (the time step).

[0042] S303, converting the polar coordinate path point back to the coordinates in the local spatio-temporal reference frame to serve as a dynamic anchor point, and the calculation formula is:

[0043] wherein, is the coordinate of the dynamic anchor in the local spatio-temporal reference frame, ensuring consistency with the static map constraints.

[0044] In the present embodiment, step S400 performs trajectory optimization based on the dynamic anchor generated in step S300 and the scene representation cached in step S200, specifically including: S401, based on the static map constraint feature in the spatio-temporal invariance scene representation, a binary mask matrix is generated under the local spatio-temporal reference frame, and the calculation formula is represented as:

[0045] wherein, is the dynamic anchor under the local spatio-temporal reference frame, is the binary mask matrix of the dynamic anchor, indicates that the position of the dynamic anchor meets the static map constraint.

[0046] S402, the safety of all dynamic anchors and targets is checked by a collision detection function, and invalid dynamic anchors are removed and valid dynamic anchors are retained, and the formula is:

[0047] wherein, is the dynamic anchor under the local spatio-temporal reference frame, is the position set of the dynamic target in the spatio-temporal invariance scene representation, is the safety radius, is the calculation of the Euclidean distance; indicates the collision coefficient of the dynamic anchor and the dynamic target, if , the dynamic anchor is marked as invalid and filtered, otherwise the dynamic anchor is marked as valid and retained.

[0048] S403, the retained dynamic anchors are optimized based on the binary mask matrix; when optimizing, the coordinate reference of the local spatio-temporal reference frame and the spatio-temporal invariance scene representation constraint are combined to minimize the geometric deviation, and the optimization formula is represented as:

[0049] wherein, is the optimized dynamic anchor, is the set of feasible path points under the local spatio-temporal reference frame, is a penalty coefficient, by minimizing the distance between the original dynamic anchor and the feasible region and the penalty of the forbidden area, it is ensured that the binary mask matrix of the optimized trajectory satisfies and has no collision.

[0050] ​Wherein, the cross-entropy loss and the geometric loss constraint are adopted to optimize the process, and the loss function is represented as:

[0051] Wherein, is the total loss, is the cross-entropy loss of the predicted trajectory and the real trajectory, is the geometric loss based on Chamfer distance (measuring the deviation from the drivable area), is the balance coefficient.

[0052] In the embodiment, step S500 takes the dynamic anchor point optimized in step S400 as the initial input, and generates a multi-modal trajectory set in combination with the space-time invariance scene representation of S200, specifically including: S501, inject the optimized dynamic anchor point into random noise to generate a diffusion starting point, and the noise injection formula is represented as:

[0053] Wherein, is the diffusion starting point of the diffusion step, is the noise coefficient of the diffusion step, which follows the cosine variance schedule, is the standard normal noise, is the optimized dynamic anchor point.

[0054] S502, in combination with the space-time invariance scene representation, gradually denoising the diffusion starting point through the denoising network of the diffusion model to generate candidate trajectories. The denoising formula is represented as:

[0055] Wherein, is the predicted noise vector, UNet is the denoising network, denotes concatenating the diffusion starting point with the space-time invariance scene representation , is the number of diffusion steps (total steps ), after steps of denoising, a candidate trajectory is generated.

[0056] S503, based on the space-time invariance scene representation, taking the consistency of the optimized dynamic anchor point as the benchmark, measuring the matching degree of the candidate trajectory and the static map constraint, obtaining the confidence score of each candidate trajectory, and the formula is represented as:

[0057] Wherein, the first confidence score of the candidate trajectory, the trajectory point of the candidate trajectory at the time step, the trajectory point of the candidate trajectory at the time step, is calculated by calculating the average minimum distance between trajectories, the larger the value, the higher the matching degree of the trajectory with other modal trajectories, and the stronger the reliability of meeting the static map constraints.

[0058] S504, based on the confidence score, outputting a multi-modal trajectory set containing trajectories , providing diverse and reliable prediction results for subsequent decision-making.

[0059] It should be noted that in the multi-modal dynamic trajectory prediction method based on spatiotemporal invariance provided by the application, a local spatiotemporal reference frame with the target current position as the origin is constructed by step S100, which realizes the decoupling of scene elements and global coordinates and provides a unified coordinate reference for subsequent extraction; On this basis, step S200 extracts multi-scale features by combining spatiotemporal convolution and graph attention mechanism, reuses historical stable features (such as static map constraint features) through sliding window caching, and incrementally encodes new dynamic targets, which greatly reduces redundant calculations in dense traffic scenes and significantly improves online inference efficiency. By step S300, polar coordinate transformation is used to generate rotation-invariant radial distance and azimuth angle features, and dynamic anchor points are generated in segments with an anchor-free recurrent decoder, enhancing the model's generalization ability for target turning, U-turn and other rotational movements; Through step S400, the dynamic anchor point trajectory is optimized based on the drivable area binary mask matrix and the collision detection function, and invalid points are filtered through point-by-point geometric checking to ensure that the trajectory compliance rate meets the requirements (for example, trajectory and regularity ≥ 95%) and effectively solves the mode collapse and compliance problems in long-term prediction. Finally, through step S500, a multi-modal trajectory set is generated by diffusing the optimized dynamic anchor points through a diffusion model and confidence reasoning, realizing accurate capture of multiple potential modes of target motion.

[0060] This method fully meets the actual needs of automatic driving for low-latency and high-precision trajectory prediction through the synergistic innovation of scene encoding and decoding mechanism: the local spatiotemporal reference frame and the caching reuse mechanism guarantee the real-time processing capability in dense scenes, the rotation-invariant feature and the scene constraint optimization improve the accuracy and reliability of long-term prediction, and the efficient and accurate prediction of multi-target trajectory is realized in complex traffic environment, which has important significance for improving the safety and decision reliability of intelligent transportation system, and has strong practicality.

[0061] Based on the same technical concept, the embodiments of the present application also provide an electronic device which can implement the method and process of the multi-modal dynamic trajectory prediction based on the space-time invariance provided by the above-mentioned embodiments of the present application. In an embodiment, the electronic device can be a server, a terminal device or other electronic device. As shown in the figure, the electronic device can include: Figure 3 at least one processor, and a memory connected with the at least one processor, and the specific connection medium between the processor and the memory in the embodiments of the present application is not limited, Figure 3 for example, the connection between the processor and the memory through the bus is taken as an example. The bus is represented by a thick line in Figure 3 the figure, and the connection mode between other components is only schematically illustrated and is not limited. The bus can be divided into an address bus, a data bus, a control bus, etc., for the convenience of representation, Figure 3 only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. Alternatively, the processor can also be called a controller, and the name is not limited.

[0062] In the embodiments of the present application, the memory stores instructions executable by the at least one processor, and the at least one processor can execute the multi-modal dynamic trajectory prediction method based on the space-time invariance discussed above by executing the instructions stored in the memory.

[0063] Among them, the processor is the control center of the device, and can connect various parts of the entire control device through various interfaces and lines, and through the running or execution of the instructions stored in the memory and the calling of the data stored in the memory, the device Various functions and processing data, thereby overall monitoring the device.

[0064] In an alternative design, the processor can include one or more processing units, and the processor can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface and the application program, etc., and the modem processor mainly processes the wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor. In some embodiments, the processor and the memory can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips respectively.

[0065] ​The processor can be a general purpose processor, such as a CPU, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general purpose processor can be a microprocessor or any conventional processor, etc. The steps of a multi-modal dynamic trajectory prediction method based on space-time invariance disclosed in combination with the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0066] The memory is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read only memory (PROM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory in the embodiments of the present application can also be a circuit or any other device capable of realizing the storage function, used for storing program instructions and / or data.

[0067] By designing and programming the processor, the code corresponding to the multi-modal dynamic trajectory prediction method based on space-time invariance introduced in the foregoing embodiments can be fixed into the chip, so that the chip can execute the steps of the method of the foregoing embodiments when running. How to design and program the processor is a technology known to those skilled in the art, which will not be described here.

[0068] Based on the same inventive concept, the embodiments of the present application also provide a storage medium storing computer instructions, when the computer instructions run on a computer, the computer executes the multi-modal dynamic trajectory prediction method based on space-time invariance discussed above.

[0069] In some alternative embodiments, the various aspects of the spatio-temporal invariance based multi-modal dynamic trajectory prediction method can also be implemented as a program product in the form of a computer program or a processor program comprising program code to, when the program product is executed by an apparatus, cause the control device to perform the steps of a spatio-temporal invariance based multi-modal dynamic trajectory prediction method according to various example embodiments of the present application described above in the specification.

[0070] It should be noted that, although several units or sub-units of the apparatus are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. Indeed, features and functions of two or more units described above can be embodied in one unit, according to embodiments of the present application. Conversely, a unit described above can be further split into several sub-units, to perform the features and functions described for that unit. Moreover, although the operations of the method(s) of the present application are described in a particular, sequential order, this is not meant to be limiting, and one or more of these operations can be omitted, or performed in a different order, or performed in parallel, without departing from the scope of the present application.

[0071] Those skilled in the art will appreciate that embodiments of the present application can be supplied as a method, a system, or a computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0072] The present application is described in reference to the flowchart illustrations and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure One The flowchart illustrations and / or block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of the methods, apparatus (systems) and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart illustrations and / or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that, in some alternative implementations, the functions noted in the flowchart illustrations and / or block diagrams can occur out of the order noted in the illustrations. For example, two Figure One The apparatus for performing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0073] Program code to implement the application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments of the application covered herein are to be

[0074] In situations in which the remote computing device utilizes a network, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN). Such networks are well known to those having ordinary skill in the art and therefore will not be discussed herein in more detail.

[0075] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure One function specified in the flow or flows and / or blocks Figure One of the block or blocks.

[0076] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure One function specified in the flow or flows and / or blocks Figure One of the block or blocks.

[0077] The embodiments of the application described herein are to be used as illustration only, and are not intended to limit the scope of the application. Those skilled in the art will readily recognize variations and modifications of the embodiments discussed herein as falling within the scope of the present application. As such, the above description should not be taken as limiting the scope of the application, and the application that is defined by the claims and their equivalents.

Claims

1. A multimodal dynamic trajectory prediction method based on spatiotemporal invariance, characterized in that: include: Construct a local spatiotemporal reference frame based on the target's historical trajectory, target status, and high-precision map features; A spatiotemporal convolutional network and a graph attention network are used for local spatiotemporal reference frames to generate spatiotemporal invariant scene representations and cache them for reuse. For spatiotemporal invariant scene representation, dynamic anchors are generated using an anchor-free recurrent decoder and a local spatiotemporal reference frame. Combine spatiotemporal invariant scene representation and collision detection to optimize dynamic anchor points; For the optimized dynamic anchor points, multimodal trajectories are generated through diffusion model and confidence reasoning.

2. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1 is characterized in that: The construction of a local spatiotemporal reference frame based on the target's historical trajectory, target status, and high-precision map features includes: Taking the current position of the target as the origin, establishing a local coordinate system based on the current heading angle, and transforming the target's historical trajectory in the global coordinate system into the local coordinate system through translation and rotation; The transformed target historical trajectory, target state and high-precision map features are spliced ​​in the local coordinate system to obtain a local spatiotemporal reference frame.

3. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1 is characterized in that: The method uses a spatiotemporal convolutional network and a graph attention network to generate a spatiotemporal invariant scene representation and cache it for reuse, including: A spatiotemporal convolutional network and a graph attention network are used to extract multi-scale features from the local spatiotemporal reference frame. Multi-scale features extracted based on spatiotemporal convolutional networks and graph attention networks are encoded into spatiotemporal invariant scene representations; A sliding window is used to cache the spatiotemporal invariant scene representations of the most recent several windows; the spatiotemporal invariant scene representations include dynamic target interaction features and static map constraint features.

4. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 3 is characterized in that: Only the newly added dynamic targets in the window are incrementally encoded in the local spatiotemporal reference frame, and the static map constraint features are reused.

5. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1 is characterized in that: The method for representing a spatiotemporal invariant scene using an anchor-free cyclic decoder and a local spatiotemporal reference frame to generate dynamic anchors includes: Convert the coordinates of the trajectory points under the local space-time reference frame into polar coordinates; After concatenating the polar coordinates with the spatiotemporal invariant scene representation, the polar coordinates are input into the anchor-free recurrent decoder to generate polar coordinate path points for several future time steps step by step. Convert the polar coordinate path points back to the coordinates of the local spatiotemporal reference frame as dynamic anchor points.

6. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 5, characterized in that: The anchor-free recurrent decoder consists of a two-layer long short-term memory network.

7. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 1, characterized in that: The optimization of dynamic anchor points by combining spatiotemporal invariant scene representation and collision detection includes: Based on the static map constraint features in the spatiotemporal invariant scene representation, a binary mask matrix is ​​generated under the local spatiotemporal reference frame; Verify the safety of all dynamic anchors and targets through the collision detection function, remove invalid dynamic anchors and retain valid dynamic anchors; The retained dynamic anchor points are optimized based on a binary mask matrix. During the optimization, the coordinate reference of the local spatiotemporal reference frame and the spatiotemporal invariance scene representation constraints are combined to minimize the geometric deviation, and the cross entropy loss and geometric loss are used to constrain the optimization process.

8. The multimodal dynamic trajectory prediction method based on spatiotemporal invariance according to claim 7 is characterized in that: The optimized dynamic anchor points are used to generate multimodal trajectories through diffusion models and confidence reasoning, including: Inject the optimized dynamic anchor point into random noise to generate the diffusion starting point; Combining the spatiotemporal invariant scene representation, the diffusion starting point is gradually denoised through the denoising network of the diffusion model to generate candidate trajectories; Based on the spatiotemporal invariant scene representation, the consistency of the optimized dynamic anchor points is used as a benchmark to measure the degree of match between the candidate trajectory and the static map constraints, and the confidence score of each candidate trajectory is obtained; Based on the confidence scores, a multimodal trajectory set containing several trajectories is output.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the method according to any one of claims 1 to 8 by executing the instructions stored in the memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store instructions, and when the instructions are executed, the method according to any one of claims 1 to 8 is implemented.

Images