Method, device and medium for motion trajectory prediction based on dynamic local map

Abstract

The application discloses a kind of motion trajectory prediction methods, devices and media based on dynamic local map, equipment.The features include: obtaining the historical motion trajectory information and map information of target vehicle, by preset feature encoder respectively on the historical motion trajectory information and the map information data extraction, obtain the trajectory feature vector and map feature vector;By preset feature fusioner on the trajectory feature vector and the map feature vector feature fusion obtains map fusion vector, to the map fusion vector and the trajectory feature vector feature fusion obtains trajectory fusion vector;According to trajectory fusion vector, the motion trajectory of target vehicle is predicted, obtains the trajectory prediction result of target vehicle.Accurate prediction of the motion trajectory of target vehicle is realized, the fusion of multiple vectors is considered to improve the prediction accuracy, reduce the influence of other interference factors in road on motion prediction, improve the prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, and in particular to a method, apparatus, device, and medium for predicting motion trajectories based on dynamic local maps. Background Technology

[0002] Currently, autonomous driving is a key research focus in the industry. When vehicles are autonomously driving, motion prediction or trajectory prediction is crucial. Motion prediction and trajectory prediction can reflect the vehicle's movement on the road, and then make predictions based on the behavior of surrounding vehicles, pedestrians, and road equipment to help the vehicle drive autonomously on the road. Current technologies use machine learning algorithms to learn and predict the movement of traffic participants, make predictions based on the historical trajectory data of other traffic participants, or use neural network models to process historical trajectory data and perception data of vehicles on the road to predict road conditions. However, due to the complexity and unknown nature of road traffic, a large amount of training data and computing resources are required, making it impossible to predict unknown road traffic. Summary of the Invention

[0003] This invention provides a method, apparatus, device, and medium for predicting motion trajectories based on dynamic local maps, so as to achieve accurate prediction of the motion trajectories of vehicles on roads.

[0004] According to one aspect of the present invention, a method for predicting motion trajectories based on a dynamic local map is provided, comprising:

[0005] The historical trajectory information and map information of the target vehicle are obtained, and the historical trajectory information and map information are extracted by a preset feature encoder to obtain the trajectory feature vector and map feature vector respectively.

[0006] The trajectory feature vector and the map feature vector are fused using a preset feature fusion processor to obtain a map fusion vector, and the map fusion vector and the trajectory feature vector are fused to obtain a trajectory fusion vector.

[0007] The trajectory of the target vehicle is predicted based on the trajectory fusion vector, and the trajectory prediction result of the target vehicle is obtained.

[0008] According to another aspect of the present invention, a motion trajectory prediction device based on a dynamic local map is provided, comprising:

[0009] The feature encoding module is used to acquire the historical trajectory information and map information of the target vehicle, and to extract the historical trajectory information and map information by a preset feature encoder to obtain the trajectory feature vector and map feature vector.

[0010] The feature fusion module is used to perform feature fusion on the trajectory feature vector and the map feature vector through a preset feature fusion device to obtain a map fusion vector, and to perform feature fusion on the map fusion vector and the trajectory feature vector to obtain a trajectory fusion vector.

[0011] The feature decoding module is used to predict the trajectory of the target vehicle based on the trajectory fusion vector, and obtain the trajectory prediction result of the target vehicle.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the motion trajectory prediction method based on a dynamic local map according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the motion trajectory prediction method based on a dynamic local map as described in any embodiment of the present invention.

[0017] The technical solution of this invention acquires the historical trajectory information and map information of the target vehicle, extracts data from the historical trajectory information and the map information respectively through a preset feature encoder, and obtains the trajectory feature vector and map feature vector. Feature extraction considering the vehicle's historical trajectory information and map information allows for the incorporation of environmental information, improving prediction accuracy. A preset feature fusion device fuses the trajectory feature vector and the map feature vector to obtain a map fusion vector, and then fuses the map fusion vector and the trajectory feature vector to obtain a trajectory fusion vector. By performing phased trajectory prediction on the predicted vectors, the accuracy of the motion prediction results is further improved. The trajectory prediction result of the target vehicle is obtained by predicting the trajectory of the target vehicle based on the trajectory fusion vector. The precise prediction of the target vehicle's trajectory using the first trajectory fusion vector achieves phased prediction of the target vehicle's trajectory, solving the technical problem of accurately predicting vehicle trajectories in the prior art, reducing the impact of other interference factors on motion prediction, and improving prediction accuracy.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a motion trajectory prediction method based on a dynamic local map provided in Embodiment 1 of the present invention;

[0021] Figure 2 This is a flowchart of another motion trajectory prediction method based on a dynamic local map provided in Embodiment 2 of the present invention;

[0022] Figure 3 A flowchart of another motion trajectory prediction method based on a dynamic local map provided in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of a motion trajectory prediction device based on a dynamic local map provided in Embodiment 3 of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the motion trajectory prediction method based on dynamic local maps according to embodiments of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] Example 1

[0027] Figure 1This is a flowchart of a motion trajectory prediction method based on a dynamic local map, provided in Embodiment 1 of the present invention. This embodiment is applicable to predicting the motion trajectory of a target vehicle. The method can be executed by a motion trajectory prediction device based on a dynamic local map, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0028] S110. Obtain the historical trajectory information and map information of the target vehicle, and extract the data from the historical trajectory information and the map information through a preset feature encoder to obtain the trajectory feature vector and the map feature vector.

[0029] The historical trajectory information can be obtained by stitching together obstacle movement data perceived by the target vehicle within a historical time window and the target vehicle's trajectory within the historical time window. For example, the historical trajectory information can be at least one of the obstacle's coordinates, speed, and heading angle information.

[0030] The map information can be road information on which the target vehicle is traveling. For example, road information can include any one of the following: lane information on the map, coordinates of the sampling point of the vehicle's center line, traffic lights in the lane, and intersection information.

[0031] The feature encoder can be a pre-defined encoder used to encode data and extract feature representations. It should be noted that the feature encoder can employ deep learning, using deep learning to process, transform, and extract feature representations from the data input to the feature encoder.

[0032] Optionally, the feature encoder may include a trajectory feature encoder and a map feature encoder. The trajectory feature encoder can be used to encode historical motion trajectory information and extract feature representations, while the map feature encoder can be used to encode map information and extract feature representations. Specifically, the trajectory feature encoder can be a self-attention neural network, which captures important high-dimensional feature information of historical motion trajectory information; the map feature encoder can be a deep neural network based on a graph attention network structure, capable of encoding the surrounding information of each node in the map information.

[0033] Among them, the trajectory feature vector can be a feature vector used to characterize the historical motion trajectory information of the target vehicle;

[0034] Among them, the map feature vector can be a feature vector used to represent map information of the target vehicle.

[0035] Specifically, the obstacle movement data perceived by the target vehicle within the historical time window and the target vehicle's movement trajectory within the historical time window are stitched together to obtain historical movement trajectory information, and the map information of the target vehicle on the road is also obtained. The historical movement trajectory information and map information are respectively input into a preset feature encoder. The feature encoder extracts features from the historical movement trajectory information and map information to obtain trajectory feature vectors and map feature vectors.

[0036] Optionally, in another optional embodiment of the present invention, the feature encoder includes a trajectory feature encoder and a map feature encoder; the step of extracting data from the historical motion trajectory information and the map information using a preset feature encoder to obtain the trajectory feature vector and the map feature vector includes:

[0037] The trajectory feature vector is obtained by capturing the features of historical motion trajectory information through the third self-attention network of the trajectory feature encoder.

[0038] The map feature vector is obtained by acquiring the surrounding information of nodes in the map information through the graph attention network of the map feature encoder.

[0039] The third self-attention network can be a neural network structure that constitutes the trajectory feature encoder.

[0040] Among them, the graph attention network can be a neural network structure that constitutes the map feature encoder.

[0041] Specifically, historical motion trajectory information is input into the trajectory feature encoder, and the third self-attention network of the trajectory feature encoder captures the features of the historical motion trajectory information to obtain the trajectory feature vector; map information is input into the map feature encoder, and the graph attention network of the map feature encoder extracts features from the surrounding information of nodes in the map information to obtain the map feature vector.

[0042] S120. The trajectory feature vector and the map feature vector are fused using a preset feature fusion tool to obtain a map fusion vector, and the map fusion vector and the trajectory feature vector are fused to obtain a trajectory fusion vector.

[0043] The feature fusion unit can be a pre-defined fusion unit used to perform cross-feature fusion of trajectory feature vectors and map feature vectors.

[0044] Among them, the map fusion vector can be a feature vector obtained by fusing the trajectory feature vector and the map feature vector.

[0045] The trajectory fusion vector can be a feature vector obtained by fusing the map fusion vector and the trajectory feature vector.

[0046] Specifically, after obtaining the trajectory feature vector and the map feature vector, the trajectory feature vector and the map feature vector are input into the feature fusion unit. The feature fusion unit performs feature fusion on the trajectory feature vector and the map feature vector to obtain the map fusion vector. Then, the map fusion vector and the trajectory feature vector are performed to obtain the trajectory fusion vector.

[0047] Optionally, in another optional embodiment of the present invention, the step of fusing the trajectory feature vector and the map feature vector using a preset feature fusion processor to obtain a map fusion vector includes:

[0048] The first cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map feature vector to obtain the map loop feature vector;

[0049] The map fusion vector is obtained by performing self-attention calculation on the map cyclic feature vector through the first self-attention network of the feature fusion processor.

[0050] The first cross-attention network can be a neural network used by the feature fusion unit to perform feature fusion.

[0051] The first self-attention network can be a neural network used by the feature fusion unit to perform self-attention feature loops.

[0052] Specifically, the trajectory feature vector and the map feature vector are input into the feature fusion unit. The first cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map feature vector to obtain the map cyclic feature vector. Then, the first self-attention network of the feature fusion unit performs self-attention calculation on the map cyclic feature vector to obtain the map fusion vector.

[0053] Optionally, in another optional embodiment of the present invention, the step of performing feature fusion on the map fusion vector and the trajectory feature vector to obtain the trajectory fusion vector includes:

[0054] The second cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map fusion vector to obtain the trajectory loop vector;

[0055] The trajectory fusion vector is obtained by performing self-attention calculation on the trajectory cyclic vector through the second self-attention network of the feature fusion processor.

[0056] The second cross-attention network can be a neural network used for cross-attention fusion in the feature fusion unit;

[0057] The second word attention network can be a neural network used for self-attention fusion calculation in the feature fusion unit.

[0058] Specifically, the trajectory feature vector and the map fusion vector are cross-attention calculated through the second cross-attention network of the feature fusion unit to obtain the trajectory loop vector. Then, the trajectory loop vector is self-attention calculated through the second self-attention network of the feature fusion unit to obtain the trajectory fusion vector.

[0059] S130. Predict the trajectory of the target vehicle based on the trajectory fusion vector to obtain the trajectory prediction result of the target vehicle.

[0060] The trajectory prediction result can be the predicted trajectory of the target vehicle.

[0061] Optionally, a trajectory prediction decoder is pre-configured for motion trajectory prediction, and the trajectory prediction decoder is used to predict the motion trajectory of the target vehicle from the trajectory fusion vector.

[0062] Specifically, the trajectory of the target vehicle is predicted by using trajectory fusion vectors to obtain the trajectory prediction result of the target vehicle.

[0063] The technical solution of this invention acquires the historical trajectory information and map information of the target vehicle, extracts data from the historical trajectory information and the map information respectively through a preset feature encoder, and obtains the trajectory feature vector and map feature vector. Feature extraction considering the vehicle's historical trajectory information and map information allows for the incorporation of environmental information, improving prediction accuracy. A preset feature fusion device fuses the trajectory feature vector and the map feature vector to obtain a map fusion vector, and then fuses the map fusion vector and the trajectory feature vector to obtain a trajectory fusion vector. By performing phased trajectory prediction on the predicted vectors, the accuracy of the motion prediction results is further improved. The trajectory prediction result of the target vehicle is obtained by predicting the trajectory of the target vehicle based on the trajectory fusion vector. The precise prediction of the target vehicle's trajectory using the first trajectory fusion vector achieves phased prediction of the target vehicle's trajectory, solving the technical problem of accurately predicting vehicle trajectories in the prior art, reducing the impact of other interference factors on motion prediction, and improving prediction accuracy.

[0064] Example 2

[0065] Figure 2 This is a flowchart of another motion trajectory prediction method based on a dynamic local map provided in Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiments is that this is a specific method. Figure 2As shown, the motion trajectory prediction method based on dynamic local maps includes:

[0066] S210. Obtain the historical trajectory information and map information of the target vehicle, and extract the data from the historical trajectory information and the map information through a preset feature encoder to obtain the trajectory feature vector and the map feature vector.

[0067] S220. The trajectory feature vector and the map feature vector are fused using a preset feature fusion tool to obtain a map fusion vector, and the map fusion vector and the trajectory feature vector are fused to obtain a trajectory fusion vector.

[0068] S230. Predict the trajectory of the target vehicle based on the trajectory fusion vector to obtain the trajectory prediction result of the target vehicle.

[0069] S240. Using the trajectory prediction results, perform local map filtering on the map fusion vector to obtain local map feature vectors, and record the number of times the local map filtering is performed.

[0070] The local map feature vector can be a local feature vector of the map fusion vector. It should be noted that the local map feature vector is a subset of the map fusion vector.

[0071] The number of times the filtering is performed can be the number of times the local map filtering is performed on the map fusion vector based on the trajectory prediction results;

[0072] Specifically, after obtaining the trajectory prediction results, the map fusion vector is filtered locally based on the trajectory prediction results to obtain local map feature vectors. Each time a local map filter is performed, the number of filters is recorded. The initial number of filters can be 0, and after each local map filter, the number of filters is incremented by 1.

[0073] Optionally, in another optional embodiment of the present invention, the step of performing local map filtering on the map fusion vector using the trajectory prediction result to obtain a local map feature vector includes:

[0074] Based on the trajectory prediction results and the preset map filtering distance, the map fusion vector is filtered by distance to obtain the local map feature vector.

[0075] The preset map filtering distance can be a pre-set map distance used for filtering within the map. For example, the preset map filtering distance can be 10 meters on the map.

[0076] Specifically, a pre-set map filtering distance is obtained, and based on the trajectory prediction results, a subset of elements within the map filtering distance is obtained from the map fusion vector using the pre-set map filtering distance as local map feature vectors.

[0077] Optionally, in another optional embodiment of the present invention, the step of performing distance filtering on the map fusion vector based on the trajectory prediction result and a preset map filtering distance to obtain a local map feature vector includes: determining the map fusion vector within the map filtering distance centered on the trajectory prediction result as the local map feature vector.

[0078] Specifically, the trajectory prediction result is used as the center of the map fusion vector. The map fusion vector is then filtered based on the map filtering distance.

[0079] S250. If the number of filtering attempts is less than a preset number of filtering attempts threshold, update the map feature vector according to the local map feature vector, update the trajectory feature vector according to the trajectory fusion vector, and return to execute the operation of performing feature fusion on the trajectory feature vector and the map feature vector through a preset feature fusion machine to obtain the map fusion vector.

[0080] The preset filtering threshold can be a pre-set threshold used to determine whether the number of filtering iterations meets the feature iteration count. For example, the preset filtering threshold can be three distance-based filtering operations performed on the map fusion vector using the trajectory prediction result and a preset map filtering distance.

[0081] Specifically, in this embodiment of the invention, through a phased loop, when the number of filtering attempts does not meet the preset filtering attempt threshold, the local map feature vector is updated to the map feature vector, and the local map feature vector replaces the map feature vector. The trajectory fusion vector is updated to the trajectory feature vector, and the trajectory fusion vector replaces the trajectory feature vector. The updated trajectory feature vector and map feature vector are returned to the feature fusion unit, and the preset feature fusion unit is executed again to perform feature fusion on the trajectory feature vector and the map feature vector to obtain the map fusion vector.

[0082] S260. When the number of screenings reaches the preset screening threshold, the trajectory prediction result of the target vehicle is taken as the final prediction result.

[0083] When the number of screenings meets the preset screening threshold, it means that the phased loop has met the number of loops, and the final prediction result can be output. The trajectory prediction result of the target vehicle obtained in the last loop is used as the final prediction result.

[0084] The technical solution of this invention acquires the historical trajectory information and map information of a target vehicle. A preset feature encoder extracts data from the historical trajectory information and the map information to obtain trajectory feature vectors and map feature vectors, respectively. A preset feature fusion processor fuses the trajectory feature vectors and map feature vectors to obtain a map fusion vector. The map fusion vector and the trajectory feature vectors are then fused to obtain a trajectory fusion vector. The trajectory fusion vector is used to predict the target vehicle's trajectory to obtain a trajectory prediction result. The trajectory prediction result is used to perform local map filtering on the map fusion vector to obtain local map feature vectors, and this local map filtering is recorded. The number of filtering iterations, achieved through local map filtering, effectively reduces the computational load on the neural network, alleviates the computational burden on the vehicle, and improves prediction efficiency. When the number of filtering iterations is less than a preset threshold, the map feature vector is updated based on the local map feature vector, and the trajectory feature vector is updated based on the trajectory fusion vector. The process then returns to execute the operation of fusing the trajectory feature vector and the map feature vector using a preset feature fusion processor to obtain a map fusion vector. This phased, iterative fusion filtering further reduces computational load, and multiple iterative predictions improve prediction accuracy. When the number of filtering iterations reaches the preset threshold, the predicted trajectory of the target vehicle is used as the final prediction result. This method achieves phased prediction of the target vehicle's trajectory, solves the technical problem of accurately predicting vehicle trajectories in existing technologies, reduces the impact of other road interference factors on motion prediction, and reduces the computational load of prediction, thereby improving prediction efficiency and accuracy.

[0085] Optional, Figure 3 A flowchart illustrating another motion trajectory prediction method based on a dynamic local map provided in an embodiment of the present invention.

[0086] S1. Encoding Section. The encoding section is responsible for encoding and representing the input data. The input data includes the historical trajectory information of the target vehicle and map information. The encoding section processes and transforms the input historical trajectory information and map information through deep learning. It captures the features of the historical trajectory information through the third self-attention network of the trajectory feature encoder to obtain the trajectory feature vector. It obtains the surrounding information of the nodes in the map information through the graph attention network of the map feature encoder to obtain the map feature vector.

[0087] S2. Feature Fusion Section. The map feature vector and trajectory feature vector obtained from the encoding section are fused, so that the trajectory features perceived by the target vehicle can obtain environmental feature information from the map, which can enhance the understanding and prediction ability of obstacle movement behavior.

[0088] Specifically, the first cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map feature vector to obtain a map loop feature vector; the first self-attention network of the feature fusion unit performs self-attention calculation on the map loop feature vector to obtain a map fusion vector; the second cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map fusion vector to obtain a trajectory loop vector; and the second self-attention network of the feature fusion unit performs self-attention calculation on the trajectory loop vector to obtain a trajectory fusion vector.

[0089] S3. Decoding section. The trajectory fusion vector is input into the trajectory prediction decoder to obtain the trajectory prediction result of the target vehicle.

[0090] Multi-stage prediction section. If the number of filtering attempts is less than a preset filtering attempt threshold, the map feature vector is updated based on the local map feature vector, and the trajectory feature vector is updated based on the trajectory fusion vector. Then, the operation of fusing the trajectory feature vector and the map feature vector using a preset feature fusion processor to obtain the map fusion vector is performed. After repeating the filtering process three times, if the number of filtering attempts reaches the preset filtering attempt threshold, the obtained trajectory prediction result of the target vehicle is taken as the final prediction result.

[0091] The embodiments of the present invention realize the phased prediction of the target vehicle's trajectory, solve the technical problem of accurately predicting the vehicle's trajectory in the prior art, reduce the impact of other interference factors in the road on the motion prediction, reduce the amount of calculation required for prediction, and improve the efficiency and accuracy of prediction.

[0092] Example 3

[0093] Figure 4 This is a schematic diagram of a motion trajectory prediction device based on a dynamic local map provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a feature encoding module 410, a feature fusion module 420, and a feature decoding module 430, wherein,

[0094] The feature encoding module 410 is used to acquire the historical trajectory information and map information of the target vehicle, and to extract the historical trajectory information and map information by a preset feature encoder to obtain the trajectory feature vector and map feature vector.

[0095] The feature fusion module 420 is used to perform feature fusion on the trajectory feature vector and the map feature vector through a preset feature fusion device to obtain a map fusion vector, and to perform feature fusion on the map fusion vector and the trajectory feature vector to obtain a trajectory fusion vector.

[0096] The feature decoding module 430 is used to predict the motion trajectory of the target vehicle based on the trajectory fusion vector, and obtain the trajectory prediction result of the target vehicle.

[0097] The technical solution of this invention acquires the historical trajectory information and map information of the target vehicle, extracts data from the historical trajectory information and the map information respectively through a preset feature encoder, and obtains the trajectory feature vector and map feature vector. Feature extraction considering the vehicle's historical trajectory information and map information allows for the incorporation of environmental information, improving prediction accuracy. A preset feature fusion device fuses the trajectory feature vector and the map feature vector to obtain a map fusion vector, and then fuses the map fusion vector and the trajectory feature vector to obtain a trajectory fusion vector. By performing phased trajectory prediction on the predicted vectors, the accuracy of the motion prediction results is further improved. The trajectory prediction result of the target vehicle is obtained by predicting the trajectory of the target vehicle based on the trajectory fusion vector. The precise prediction of the target vehicle's trajectory using the first trajectory fusion vector achieves phased prediction of the target vehicle's trajectory, solving the technical problem of accurately predicting vehicle trajectories in the prior art, reducing the impact of other interference factors on motion prediction, and improving prediction accuracy.

[0098] Optionally, the device further includes a local map filtering module, a data update and iteration module, and a result output module; wherein:

[0099] The local map filtering module is used to filter the map fusion vector using the trajectory prediction result to obtain a local map feature vector and record the number of times the local map filtering is performed.

[0100] The data update iteration module is used to update the map feature vector according to the local map feature vector and update the trajectory feature vector according to the trajectory fusion vector when the number of filtering times is less than a preset number of filtering times threshold, and return to execute the operation of performing feature fusion on the trajectory feature vector and the map feature vector through a preset feature fusion machine to obtain a map fusion vector.

[0101] The result output module is used to take the obtained trajectory prediction result of the target vehicle as the final prediction result when the number of screenings reaches the preset screening threshold.

[0102] Optionally, the local map filtering module is specifically used for:

[0103] Based on the trajectory prediction results and the preset map filtering distance, the map fusion vector is filtered by distance to obtain the local map feature vector.

[0104] Optionally, the local map filtering module is further used for:

[0105] Centered on the trajectory prediction result, the map fusion vector within the map filtering distance is determined as the local map feature vector.

[0106] Optionally, the feature fusion module is specifically used for:

[0107] The first cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map feature vector to obtain the map loop feature vector;

[0108] The map fusion vector is obtained by performing self-attention calculation on the map cyclic feature vector through the first self-attention network of the feature fusion processor.

[0109] Optionally, the feature fusion module is further used for:

[0110] The second cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map fusion vector to obtain the trajectory loop vector;

[0111] The trajectory fusion vector is obtained by performing self-attention calculation on the trajectory cyclic vector through the second self-attention network of the feature fusion processor.

[0112] Optionally, the feature encoding module is specifically used for:

[0113] The trajectory feature vector is obtained by capturing the features of historical motion trajectory information through the third self-attention network of the trajectory feature encoder.

[0114] The map feature vector is obtained by acquiring the surrounding information of nodes in the map information through the graph attention network of the map feature encoder.

[0115] The motion trajectory prediction device based on dynamic local maps provided in the embodiments of the present invention can execute the motion trajectory prediction method based on dynamic local maps provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Example 4

[0117] Figure 5A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0118] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0119] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0120] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as motion trajectory prediction methods based on dynamic local maps.

[0121] In some embodiments, the motion trajectory prediction method based on a dynamic local map can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the motion trajectory prediction method based on a dynamic local map described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the motion trajectory prediction method based on a dynamic local map by any other suitable means (e.g., by means of firmware).

[0122] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0126] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0127] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0128] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0129] Example 5

[0130] This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the motion trajectory prediction method based on a dynamic local map as provided in any embodiment of the present invention. The method includes:

[0131] The historical trajectory information and map information of the target vehicle are obtained, and the historical trajectory information and map information are extracted by a preset feature encoder to obtain the trajectory feature vector and map feature vector respectively.

[0132] The trajectory feature vector and the map feature vector are fused using a preset feature fusion processor to obtain a map fusion vector, and the map fusion vector and the trajectory feature vector are fused to obtain a trajectory fusion vector.

[0133] The trajectory of the target vehicle is predicted based on the trajectory fusion vector, and the trajectory prediction result of the target vehicle is obtained.

[0134] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0135] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0136] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0137] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for predicting motion trajectories based on dynamic local maps, characterized in that, include: The historical trajectory information and map information of the target vehicle are obtained, and the historical trajectory information and map information are extracted by a preset feature encoder to obtain trajectory feature vector and map feature vector respectively. The trajectory feature vector and the map feature vector are fused using a preset feature fusion processor to obtain a map fusion vector, and the map fusion vector and the trajectory feature vector are fused to obtain a trajectory fusion vector. The trajectory of the target vehicle is predicted based on the trajectory fusion vector, and the trajectory prediction result of the target vehicle is obtained.

2. The method according to claim 1, characterized in that, After predicting the trajectory of the target vehicle based on the trajectory fusion vector to obtain the trajectory prediction result of the target vehicle, the method further includes: The local map feature vector is obtained by filtering the map fusion vector using the trajectory prediction results, and the number of times the local map filtering is performed is recorded. If the number of filtering attempts is less than a preset filtering attempt threshold, update the map feature vector according to the local map feature vector, update the trajectory feature vector according to the trajectory fusion vector, and return to execute the operation of performing feature fusion on the trajectory feature vector and the map feature vector through a preset feature fusion processor to obtain the map fusion vector; If the number of screenings reaches the preset screening threshold, the trajectory prediction result of the target vehicle will be used as the final prediction result.

3. The method according to claim 2, characterized in that, The step of filtering the map fusion vector using the trajectory prediction result to obtain a local map feature vector includes: Based on the trajectory prediction results and the preset map filtering distance, the map fusion vector is filtered by distance to obtain the local map feature vector.

4. The method according to claim 3, characterized in that, The step of performing distance filtering on the map fusion vector based on the trajectory prediction result and a preset map filtering distance to obtain a local map feature vector includes: Centered on the trajectory prediction result, the map fusion vector within the map filtering distance is determined as the local map feature vector.

5. The method according to claim 1, characterized in that, The step of fusing the trajectory feature vector and the map feature vector using a preset feature fusion processor to obtain the map fusion vector includes: The first cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map feature vector to obtain the map loop feature vector; The map fusion vector is obtained by performing self-attention calculation on the map cyclic feature vector through the first self-attention network of the feature fusion processor.

6. The method according to claim 1, characterized in that, The process of fusing the map fusion vector and the trajectory feature vector to obtain the trajectory fusion vector includes: The second cross-attention network of the feature fusion unit performs cross-attention calculation on the trajectory feature vector and the map fusion vector to obtain the trajectory loop vector; The trajectory fusion vector is obtained by performing self-attention calculation on the trajectory cyclic vector through the second self-attention network of the feature fusion processor.

7. The method according to claim 1, characterized in that, The feature encoder includes a trajectory feature encoder and a map feature encoder; The step of extracting data from the historical trajectory information and the map information using a preset feature encoder to obtain the trajectory feature vector and the map feature vector includes: The trajectory feature vector is obtained by capturing the features of historical motion trajectory information through the third self-attention network of the trajectory feature encoder. The map feature vector is obtained by acquiring the surrounding information of nodes in the map information through the graph attention network of the map feature encoder.

8. A motion trajectory prediction device based on a dynamic local map, characterized in that, include: The feature encoding module is used to acquire the historical trajectory information and map information of the target vehicle, and to extract the historical trajectory information and map information by a preset feature encoder to obtain trajectory feature vector and map feature vector respectively. The feature fusion module is used to perform feature fusion on the trajectory feature vector and the map feature vector through a preset feature fusion device to obtain a map fusion vector, and to perform feature fusion on the map fusion vector and the trajectory feature vector to obtain a trajectory fusion vector; The feature decoding module is used to predict the trajectory of the target vehicle based on the trajectory fusion vector, and obtain the trajectory prediction result of the target vehicle.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the motion trajectory prediction method based on a dynamic local map as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the motion trajectory prediction method based on any one of claims 1-7.

Images