Trajectory prediction method, trajectory prediction model training method, medium and device

Abstract

The invention provides a trajectory prediction method, a trajectory prediction model training method, a medium and equipment, and relates to the technical field of computers, in particular to the technical field of automatic driving and artificial intelligence. According to the implementation scheme, first information and a plurality of passing point sequences of a target vehicle are obtained, the first information comprises navigation information of the target vehicle and perception information of the surrounding environment of the target vehicle, and each passing point sequence is composed of a plurality of passing points in a future passing road section; for each approach point sequence in the plurality of approach point sequences, track prediction is carried out based on the first information under the guidance of the approach point sequence so as to acquire a prediction track corresponding to the approach point sequence and a score of the prediction track; and determining a target prediction trajectory in the plurality of prediction trajectories based on the score of each prediction trajectory in the plurality of prediction trajectories.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of autonomous driving and artificial intelligence technology, and specifically to a trajectory prediction method implemented by using a trajectory prediction model, a trajectory prediction model training method, device, electronic device, computer-readable storage medium, computer program product, and autonomous driving vehicle. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] Autonomous driving technology integrates many technologies such as recognition, decision-making, positioning, communication security, and human-computer interaction. Artificial intelligence learning can assist in generating autonomous driving strategies.

[0004] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0005] The present disclosure provides a trajectory prediction method implemented using a trajectory prediction model, a trajectory prediction model training method, an apparatus, an electronic device, a computer-readable storage medium, a computer program product, and an autonomous driving vehicle.

[0006] According to one aspect of the present disclosure, a trajectory prediction method is provided, comprising: obtaining first information of a target vehicle and multiple waypoint sequences, the first information comprising navigation information of the target vehicle and perception information of the target vehicle's surrounding environment, each waypoint sequence consisting of multiple waypoints within a future route segment; for each waypoint sequence in the multiple waypoint sequences, performing trajectory prediction based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; and determining a target predicted trajectory among the multiple predicted trajectories based on the scores of each predicted trajectory in the multiple predicted trajectories.

[0007] According to another aspect of the present disclosure, a method for training a trajectory prediction model is provided, the method comprising: obtaining sample data, the sample data comprising first information of a sample vehicle and a plurality of waypoint sequences, the first information comprising sample navigation information of the sample vehicle and sample perception information of the surrounding environment of the sample vehicle, each waypoint sequence consisting of a plurality of waypoints within a future route segment and each waypoint sequence being annotated with a corresponding score label, the plurality of waypoint sequences comprising a positive sample sequence; for each waypoint sequence of the sample data, inputting the waypoint sequence and the first information into a trajectory prediction model, so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence, and outputs a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; and adjusting parameters of the trajectory prediction model based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence.

[0008] According to another aspect of the present disclosure, a trajectory prediction device is provided, including: a first acquisition unit, configured to acquire first information of a target vehicle and multiple waypoint sequences, the first information including navigation information of the target vehicle and perception information of the target vehicle's surrounding environment, each waypoint sequence consisting of multiple waypoints within a future route segment; a second acquisition unit, configured to perform trajectory prediction based on the first information for each waypoint sequence in the multiple waypoint sequences under the guidance of the waypoint sequence, so as to acquire a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; and a first determination unit, configured to determine a target predicted trajectory among multiple predicted trajectories based on the scores of each predicted trajectory in the multiple predicted trajectories.

[0009] According to another aspect of the present disclosure, a training device for a trajectory prediction model is provided, the device comprising: a first acquisition unit, configured to acquire sample data, the sample data comprising first information of a sample vehicle and a plurality of waypoint sequences, the first information comprising sample navigation information of the sample vehicle and sample perception information of the surrounding environment of the sample vehicle, each waypoint sequence consisting of a plurality of waypoints within a future route segment and each waypoint sequence being annotated with a corresponding score label, the plurality of waypoint sequences comprising a positive sample sequence; a first input unit, configured to, for each waypoint sequence of the sample data, input the waypoint sequence and the first information into a trajectory prediction model, so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence, and outputs a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; and an adjustment unit, configured to adjust parameters of the trajectory prediction model based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence.

[0010] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned trajectory prediction method implemented by using the trajectory prediction model or the above-mentioned trajectory prediction model training method.

[0011] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned trajectory prediction method implemented by using the trajectory prediction model or the above-mentioned trajectory prediction model training method.

[0012] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-mentioned trajectory prediction method implemented by using the trajectory prediction model or the above-mentioned trajectory prediction model training method when executed by a processor.

[0013] According to another aspect of the present disclosure, an autonomous driving vehicle is provided, comprising the above-mentioned electronic device.

[0014] According to one or more embodiments of the present disclosure, the rationality of trajectory prediction of an autonomous driving vehicle can be improved.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0017] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0018] Figure 2 A flow chart of a trajectory prediction method implemented by using a trajectory prediction model according to an embodiment of the present disclosure is shown;

[0019] Figure 3 A flowchart of acquiring a plurality of waypoint sequences according to an embodiment of the present disclosure is shown;

[0020] Figure 4A schematic diagram showing a sequence of waypoints according to an exemplary embodiment of the present disclosure is shown;

[0021] Figure 5 A schematic diagram of the structure of a trajectory prediction model according to an embodiment of the present disclosure is shown;

[0022] Figure 6 A schematic diagram of the structure of a first prediction network according to an exemplary embodiment of the present disclosure is shown;

[0023] Figure 7 A flowchart of a method for training a trajectory prediction model according to an embodiment of the present disclosure is shown;

[0024] Figure 8 A schematic diagram showing a training method of a first prediction network according to an embodiment of the present disclosure is shown;

[0025] Fig. 9 A structural block diagram of a trajectory prediction device implemented using a trajectory prediction model according to an embodiment of the present disclosure is shown;

[0026] Fig.10 A structural block diagram of a trajectory prediction model training device according to an embodiment of the present disclosure is shown;

[0027] Fig.11 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0029] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0030] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0031] In the related technology, there are two main end-to-end trajectory prediction solutions:

[0032] First, the vehicle's driving trajectory for the entire section of the road in the future is directly predicted based on the perception information and navigation information on the vehicle side. Since the potential feasible vehicle driving trajectory is multi-modal, the label trajectory is only one of the solutions. This solution will result in insufficient lateral avoidance, deceleration, etc. of the entire driving trajectory obtained. For example, in a lane change and overtaking scenario, if the above solution is used for trajectory prediction, the obtained trajectory may not change lanes to the left or right, but output a trajectory similar to straight driving, resulting in a lane change failure.

[0033] Second, the future road is divided into sections, and the trajectory is predicted for each sub-section based on the vehicle-side perception information and navigation information. This method can enable the vehicle to choose the appropriate lane in each sub-section, but due to the discontinuity of the prediction, it is easy to cause the output trajectory to jump between multiple lanes, causing the vehicle to change lanes repeatedly and unable to drive normally.

[0034] The embodiment of the present disclosure provides a trajectory prediction method, which obtains multiple possible waypoint sequences on the road section that the target vehicle will pass in the future, and under the guidance of each waypoint sequence, predicts multiple predicted trajectories corresponding to the multiple waypoint sequences and the score of each predicted trajectory, and determines the target predicted trajectory based on the score of each predicted trajectory. This can avoid the trajectory jump problem caused by segmented trajectory prediction, and avoid the problem of insufficient lateral avoidance, deceleration, etc. in the entire predicted trajectory, thereby improving the rationality of trajectory prediction.

[0035] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0036] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes a motor vehicle 110 , a server 120 , and one or more communication networks 130 coupling the motor vehicle 110 to the server 120 .

[0037] In an embodiment of the present disclosure, the motor vehicle 110 may include a computing device according to an embodiment of the present disclosure and / or be configured to perform a method according to an embodiment of the present disclosure.

[0038] The server 120 may run one or more services or software applications that enable the trajectory prediction method. In some embodiments, the server 120 may also provide other services or software applications, which may include non-virtual environments and virtual environments. Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. A user of the motor vehicle 110 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0039] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0040] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0041] In some embodiments, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from motor vehicle 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of motor vehicle 110.

[0042] The network 130 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including, but not limited to, TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a satellite communications network, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (including, for example, Bluetooth, WiFi), and / or any combination of these and other networks.

[0043] The system 100 may also include one or more databases 150. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 150 may be used to store information such as audio files and video files. The data repository 150 may reside in various locations. For example, the data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data repository 150 may be of different types. In some embodiments, the data repository used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to a command.

[0044] In some embodiments, one or more of the databases 150 may also be used by an application to store application data. The databases used by the application may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0045] The motor vehicle 110 may include a sensor 111 for sensing the surrounding environment. The sensor 111 may include one or more of the following sensors: a visual camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, and a laser radar (LiDAR). Different sensors may provide different detection accuracy and range. The camera may be installed at the front, rear, or other location of the vehicle. The visual camera may capture the situation inside and outside the vehicle in real time and present it to the driver and / or passengers. In addition, by analyzing the images captured by the visual camera, information such as traffic light indications, intersection conditions, and the operating status of other vehicles may be obtained. The infrared camera may capture objects under night vision conditions. Ultrasonic sensors may be installed around the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of strong ultrasonic directionality. Millimeter wave radars may be installed at the front, rear, or other location of the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of electromagnetic waves. Laser radars may be installed at the front, rear, or other location of the vehicle to detect object edges and shape information, thereby identifying and tracking objects. Due to the Doppler effect, the radar device may also measure the speed change of the vehicle and the moving object.

[0046] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module capable of receiving satellite positioning signals (e.g., Beidou, GPS, GLONASS, and GALILEO) from satellites 141 and generating coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142, and the mobile communication network may implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, and other current or evolving wireless communication technologies (e.g., 5G technology). The communication device 112 may also have a vehicle-to-vehicle or vehicle-to-everything (V2X) module, which is configured to implement, for example, vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144. In addition, the communication device 112 may also have a module configured to communicate with a user terminal 145 (including but not limited to a smartphone, a tablet computer, or a wearable device such as a watch) by, for example, using a wireless LAN or Bluetooth of the IEEE802.11 standard. Using the communication device 112, the motor vehicle 110 may also access the server 120 via the network 130.

[0047] The motor vehicle 110 may also include a control device 113. The control device 113 may include a processor that communicates with various types of computer-readable storage devices or media, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors. The control device 113 may include an automatic driving system for automatically controlling various actuators in the vehicle. The automatic driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to inputs from multiple sensors 111 or other input devices to control acceleration, steering, and braking, respectively, without human intervention or limited human intervention. Some processing functions of the control device 113 can be implemented through cloud computing. For example, some processing can be performed using an on-board processor, while other processing can be performed using computing resources in the cloud. The control device 113 can be configured to perform a method according to the present disclosure. In addition, the control device 113 can be implemented as an example of a computing device on the motor vehicle side (client) according to the present disclosure.

[0048] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0049] According to the embodiments of the present disclosure, Figure 2 As shown, a trajectory prediction method is provided, the method comprising: step S201, obtaining first information of a target vehicle and multiple waypoint sequences, the first information comprising navigation information of the target vehicle and perception information of the target vehicle's surrounding environment, each waypoint sequence consisting of multiple waypoints in a future route segment; step S202, for each waypoint sequence in the multiple waypoint sequences, performing trajectory prediction based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; and step S203, determining a target predicted trajectory from the multiple predicted trajectories based on the scores of each predicted trajectory in the multiple predicted trajectories.

[0050] Therefore, by obtaining multiple possible waypoint sequences of the target vehicle on the road section that it will pass in the future, under the guidance of each waypoint sequence, multiple predicted trajectories corresponding to the multiple waypoint sequences and the score of each predicted trajectory are predicted, and the target predicted trajectory is determined based on the score of each predicted trajectory. This can avoid the trajectory jump problem caused by segmented trajectory prediction, and avoid the problem of insufficient lateral avoidance, deceleration, etc. in the entire predicted trajectory, thereby improving the rationality of trajectory prediction.

[0051] In some embodiments, first information of the current target vehicle may be acquired first, and the first information may include navigation information of the target vehicle and perception information of the surrounding environment of the target vehicle.

[0052] In some embodiments, the navigation information of the target vehicle may include vectorized navigation information and vectorized map information, and the vectorized navigation information and vectorized map information may be obtained by performing vectorization operations on one or more of lane-level or road-level navigation information and coarse positioning information.

[0053] In some embodiments, the perception information of the target vehicle's surroundings may include perception information from one or more cameras, perception information from one or more laser radars, and perception information from one or more millimeter-wave radars. It is understandable that the perception information of the target vehicle's surroundings is not limited to one of the above forms. For example, it may only include perception information from multiple cameras, but not perception information from one or more laser radars and perception information from one or more millimeter-wave radars. The perception information obtained by the camera may be perception information in the form of a picture or video, and the perception information obtained by the laser radar may be perception information in the form of a radar point cloud (e.g., a three-dimensional point cloud).

[0054] In some embodiments, the perception information may further include current perception information of the surrounding environment of the target vehicle during the driving process of the vehicle and historical perception information corresponding to multiple historical moments.

[0055] In some embodiments, the perception information may also be information on the types and current status of multiple road elements and obstacles in the target vehicle's surrounding environment detected by the perception detection module based on the original sensor data. For example, it may include the size and position of the obstacle, as well as the vehicle type, the current state of the vehicle (such as whether the turn signal, high beam, and other long-tail information are turned on), the position and length of the lane line, etc. It will be understood that the classification for the obstacle may be one or more of the predefined multiple categories.

[0056] In some embodiments, the first information may also include at least one of map information and current motion state information of the target vehicle. Thus, by inputting more abundant information for prediction, the effect of trajectory prediction can be further improved.

[0057] In some embodiments, the navigation map may include a road-level map (SD Map), a lane-level map (LD Map), and a high-precision map (HD Map). The road-level map is mainly composed of granular road topology information, and the navigation positioning accuracy is low (for example, the accuracy is about 15 meters). It is mainly used to help drivers navigate and cannot meet the needs of autonomous driving. Lane-level maps and high-precision maps can be used for autonomous driving. Lane-level maps add lane-level topology information, and have higher accuracy than road-level maps. The accuracy is generally at the sub-meter level, and can include road information (such as lane lines) and lane-related ancillary facilities information (such as traffic lights, road signs, parking spaces, etc.), which can be used to assist autonomous driving. Compared with lane-level maps, high-precision maps have higher map data accuracy (accuracy reaches the centimeter level), richer map data types, and higher map update frequency, and can be used for autonomous driving.

[0058] In some embodiments, the current motion state information of the target vehicle may include information such as the current position, speed, acceleration, and direction of travel of the target vehicle.

[0059] It is understandable that the above-mentioned first information can be determined based on actual needs and is not limited here.

[0060] In some embodiments, a plurality of waypoint sequences may be composed of a plurality of possible waypoints of the target vehicle on the future driving section. In some embodiments, the waypoints on the future driving section may be obtained by random sampling or preset interval sampling on the future driving section.

[0061] In some embodiments, for each waypoint sequence in a plurality of waypoint sequences, trajectory prediction is performed based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory. The waypoint sequence and the first information can be input into a trajectory prediction model together. The trajectory prediction model, under the guidance of the waypoint sequence, performs trajectory prediction and trajectory score prediction based on the first information to output a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory.

[0062] In some embodiments, the above trajectory prediction model can be constructed using any Transformer network structure, for example. It is understandable that relevant technicians can also determine the network structure of the trajectory prediction model according to actual needs, which is not limited here.

[0063] Subsequently, a target predicted trajectory can be determined from multiple predicted trajectories based on the scores of each predicted trajectory. In some embodiments, the predicted trajectory with the highest score can be determined as the target predicted trajectory. In some embodiments, a preset number of predicted trajectories with the highest scores can also be obtained, and the optimal predicted trajectory can be determined through further screening strategies, wherein the above screening strategies can, for example, be screening predicted trajectories whose number of lane changes is not higher than a preset number or selecting the most appropriate predicted trajectory according to the road conditions ahead, etc., which are not limited here.

[0064] Therefore, by obtaining multiple possible waypoint sequences of the target vehicle on the road section that it will pass in the future, under the guidance of each waypoint sequence, multiple predicted trajectories corresponding to the multiple waypoint sequences and the score of each predicted trajectory are predicted, and the target predicted trajectory is determined based on the score of each predicted trajectory. This can avoid the trajectory jump problem caused by segmented trajectory prediction, and avoid the problem of insufficient lateral avoidance, deceleration, etc. in the entire predicted trajectory, thereby improving the rationality of trajectory prediction.

[0065] In some embodiments, Figure 3 As shown, the acquisition of multiple waypoint sequences can also be obtained based on the following operations: step S301, based on the current motion state of the target vehicle, predicting the future route section that the target vehicle may pass through within a preset time period in the future; step S302, sampling the future route section at a preset distance interval to obtain multiple sampling positions on the future route section; step S303, acquiring at least one waypoint on the road section at each sampling position in the multiple sampling positions; and step S304, based on each waypoint at each sampling position in the multiple sampling positions, traversing the sequence of all possible waypoints that the target vehicle may pass through to obtain multiple waypoint sequences.

[0066] In some embodiments, the future path area that the target vehicle may pass through in the future preset period can be determined on the current navigation route of the target vehicle based on the current motion state of the target vehicle. In some embodiments, the future path area can be calculated based on information such as the current speed and acceleration of the target vehicle.

[0067] In some embodiments, the future preset time period may be, for example, several seconds in the future. It is understandable that relevant technicians may set it based on actual needs, and no limitation is made here.

[0068] In some embodiments, after determining the future route area, sampling can be performed at preset intervals for the future route area to obtain multiple sampling positions. For example, the future route area can be a road section within a subsequent 100-meter driving distance on the current navigation route, and this road section can be sampled every 30 meters, so that 3 sampling positions can be obtained.

[0069] In some embodiments, the current navigation route may also be sampled in advance at preset distance intervals (e.g., 30 meters) to obtain multiple sampling locations on the navigation section; after determining the future path area based on the current vehicle motion state, multiple sampling locations contained in the area may be determined based on the future path area.

[0070] On the basis of the above, at least one waypoint on the road section at each sampling position can be obtained by sampling. The above sampling can be random sampling or sampling at a preset interval. For example, for a road section at a sampling position, sampling can be performed on the sampling surface at a sampling interval of 4 meters to obtain multiple sampling points; then, a sampling point can be selected at each sampling position as a waypoint to form a waypoint sequence, so as to obtain multiple waypoint sequences in the future route area based on the above sampling points.

[0071] In this way, the efficiency of acquiring the waypoint sequence can be improved, and at the same time, the rationality of the waypoint sequence can be improved, further improving the effect of trajectory prediction.

[0072] In some embodiments, at least one waypoint includes an intersection of a road section at a corresponding sampling position and at least one lane centerline.

[0073] Therefore, by traversing the waypoint sequence based on the lane centerline, the waypoint sequence used to guide trajectory prediction can be made more consistent with the actual traffic scenario, thereby further improving the effect of trajectory prediction.

[0074] Figure 4 A schematic diagram of a waypoint sequence according to an exemplary embodiment of the present disclosure is shown.

[0075] In some exemplary embodiments, Figure 4 As shown, in the future path area 400, sampling surfaces 411, 412, and 413 are obtained by sampling at preset distance intervals, and the sampling surfaces 411, 412, and 413 have 1, 2, and 3 intersections with the lane centerline 420, respectively. Based on each waypoint at each sampling surface in the multiple sampling surfaces, the sequence of all possible waypoints that the target vehicle may pass through is traversed, and 1×2×3=6 waypoint sequences can be obtained.

[0076] In some embodiments, the above-mentioned multiple waypoint sequences may include multiple waypoint sequences for each future preset period in multiple future preset periods. Exemplarily, multiple future preset periods may be set, and the possible future driving section of the target vehicle may be estimated for each future preset period, and based on the above-mentioned method, multiple possible waypoint sequences in each future driving section may be obtained. Thus, the waypoint sequences can further comprehensively cover the various states of the target vehicle in the future, thereby making the trajectory prediction more accurate.

[0077] In some embodiments, the above-mentioned trajectory prediction method can be implemented based on a trajectory prediction model, which may include a trajectory prediction network and a scoring network. For each waypoint sequence in a plurality of waypoint sequences, trajectory prediction is performed based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a score corresponding to the predicted trajectory. The method may include: for each waypoint sequence in a plurality of waypoint sequences, using a trajectory prediction network, trajectory prediction is performed based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a predicted trajectory feature corresponding to the predicted trajectory; and multiple predicted trajectory features corresponding to the multiple waypoint sequences are respectively input into the scoring network to obtain the score of each predicted trajectory in the multiple predicted trajectories corresponding to the multiple waypoint sequences output by the scoring network.

[0078] Therefore, predicting the corresponding predicted trajectory score based on the predicted trajectory features can improve the efficiency and accuracy of score prediction.

[0079] Figure 5 A schematic diagram of the structure of a trajectory prediction model according to an embodiment of the present disclosure is shown.

[0080] In some embodiments, Figure 5 As shown, the trajectory prediction network 510 in the above trajectory prediction model can adopt a Transformer network structure with an encoder (Encoder) and a decoder (Decoder). It can be understood that the trajectory prediction network 510 can also be other neural network models based on the Transformer network structure, which is not limited here. The Transformer architecture can calculate the implicit representation of the model input and output through the self-attention mechanism. In other words, the Transformer architecture can be an Encoder-Decoder model built based on this self-attention mechanism.

[0081] In some embodiments, for each waypoint sequence, the first information and the waypoint sequence can be directly input into the trajectory prediction network 510, and the trajectory prediction network 510 performs feature extraction and feature interaction on the first information and the implicit feature information in the waypoint sequence, so that the trajectory prediction network 510, under the guidance of the waypoint sequence, predicts the predicted trajectory features and the corresponding predicted trajectory corresponding to the waypoint sequence.

[0082] In some embodiments, the predicted trajectory feature may be obtained based on the output of an encoder in a Transformer network, and the predicted trajectory may be obtained by inputting the predicted trajectory feature into a decoder in the Transformer network and decoding the decoder.

[0083] In some embodiments, the scoring network 520 in the trajectory prediction model can be any feedforward neural network. For example, a multilayer perceptron (MLP) network can be used to construct the scoring network. The scoring network 520 is configured to analyze each predicted trajectory feature output by the trajectory prediction network to output a score corresponding to the predicted trajectory.

[0084] In some embodiments, the trajectory prediction network may include a first encoding network and a first prediction network. For each waypoint sequence in a plurality of waypoint sequences, using the trajectory prediction network, trajectory prediction is performed based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a predicted trajectory feature corresponding to the predicted trajectory. The method may include: inputting the first information into the first encoding network so that the first encoding network performs feature extraction on the first information and outputs the first feature; and for each waypoint sequence in a plurality of waypoint sequences, inputting the waypoint sequence and the first feature into the first prediction network so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory.

[0085] Therefore, by extracting the intrinsic features of the first information in advance, the first features that can better reflect the intrinsic laws and intrinsic features of the first information are obtained, and when performing trajectory prediction for each waypoint sequence, the first features are directly input, so that more critical feature information can be directly given to the first prediction network, thereby improving the trajectory prediction effect.

[0086] In some embodiments, the first encoding network can utilize the encoder structure in the Transformer network to perform encoding calculations on the first information, extract intrinsic features of the first information, and output the first features.

[0087] In some embodiments, the first encoding network can be configured to first map the multimodal first information (for example, including images, videos, point clouds, etc.) to a preset space (for example, BEV space) to obtain an intermediate feature representation, and use a spatial self-attention mechanism to extract the required spatial features from the image features, thereby aggregating spatial information; in addition, a temporal self-attention mechanism can be used to fuse historical information, thereby aggregating temporal information.

[0088] In some embodiments, the first prediction network may, for example, adopt a Transformer network structure having an encoder and a decoder. The first prediction network may be configured to extract features from the input waypoint sequence, further interact and fuse the extracted features with the first features, and then output the predicted trajectory features corresponding to the waypoint sequence; subsequently, the predicted trajectory features may be decoded to obtain the corresponding predicted trajectory.

[0089] Thus, the first information is extracted to obtain rich intrinsic feature information (such as time series and spatial information) through the above method, thereby further improving the accuracy of subsequent trajectory prediction. At the same time, since the first information is multi-modal in many cases and the data volume is large, by first extracting its intrinsic features and converting it into a first feature of a fixed dimension, the first feature can be directly input in each trajectory prediction, thereby reducing the overall calculation amount of the model, saving computing resources, and improving computing efficiency.

[0090] In some embodiments, the first prediction network may include a multi-layer prediction sub-network. For each waypoint sequence in a plurality of waypoint sequences, the waypoint sequence and the first feature are input into the first prediction network, so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory. The following operations may be performed: for each waypoint sequence in a plurality of waypoint sequences, based on each layer of the multi-layer prediction sub-network, the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the layer of prediction sub-network are input into the layer of prediction sub-network to obtain the second trajectory output by the layer of prediction sub-network. Features and corresponding second predicted trajectories, wherein the first trajectory features corresponding to the first layer of prediction subnetworks in the multi-layer prediction subnetworks are preset initial trajectory features, the first trajectory point sequence corresponding to the first layer of prediction subnetworks is a waypoint sequence, the first trajectory features of each layer of prediction subnetworks other than the first layer of prediction subnetworks in the multi-layer prediction subnetworks are the second trajectory features output by the previous layer of prediction subnetworks, and the first trajectory point sequences of other layers of prediction subnetworks are obtained by sampling trajectory points of the second predicted trajectory output by the previous layer of prediction subnetworks; and the second trajectory features and the second predicted trajectory output by the last layer of prediction subnetworks in the multi-layer prediction subnetworks are respectively determined as the predicted trajectory features and the predicted trajectory corresponding to the waypoint sequence.

[0091] Therefore, based on the first feature and each waypoint sequence, a multi-layer prediction sub-network is applied to continuously iteratively optimize the trajectory features and the predicted trajectory, so as to obtain a more accurate predicted trajectory.

[0092] In some embodiments, the first prediction network may include multiple layers of prediction subnetworks, and each layer of prediction subnetworks may be a network constructed based on a Transformer network structure. It is understandable that the Transformer network structure may be a Transformer network structure based on a self-attention mechanism, a Transformer network structure based on a multi-head attention mechanism, etc., which is not limited here.

[0093] In some embodiments, the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of this layer can be input into the corresponding prediction subnetwork together, so that the prediction subnetwork, under the guidance of the first trajectory point sequence, extracts and interacts the input features and information with intrinsic features, outputs the second trajectory feature and the corresponding second predicted trajectory, and updates the first trajectory feature and the first trajectory point sequence based on the output second trajectory feature and the trajectory point sequence obtained by sampling the second predicted trajectory, and inputs them into the prediction subnetwork of the next layer.

[0094] In some embodiments, each prediction subnetwork in a multi-layer prediction subnetwork may include a feature interaction subnetwork and a decoding subnetwork. Based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, inputting the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of the layer into the prediction subnetwork of the layer to obtain the second trajectory feature and the corresponding second predicted trajectory output by the prediction subnetwork of the layer may include: based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, inputting the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of the layer into the feature interaction subnetwork of the layer prediction subnetwork, so that the feature interaction subnetwork of the layer prediction subnetwork performs interactive analysis on the first feature and the first trajectory feature under the guidance of the first trajectory point sequence and outputs the second trajectory feature; and inputting the second trajectory feature output by the feature interaction subnetwork of the layer prediction subnetwork into the decoding subnetwork of the layer prediction subnetwork to obtain the second predicted trajectory output by the decoding subnetwork of the layer prediction subnetwork.

[0095] Therefore, firstly, through the feature interaction sub-network of each layer, under the guidance of the current first trajectory point sequence, the first trajectory features and the first features are interacted to obtain the optimized trajectory features (i.e., the second trajectory features), and the second predicted trajectory is obtained through the decoding sub-network, which can further improve the effect of trajectory prediction.

[0096] In some embodiments, each layer of prediction subnetwork may include a feature interaction subnetwork and a decoding subnetwork, wherein the feature interaction subnetwork may be a network constructed based on a Transformer network structure, and the decoding subnetwork may apply a decoder structure in a Transformer network. It is understandable that the Transformer network structure may be a Transformer network structure based on a self-attention mechanism, a Transformer network structure based on a multi-head attention mechanism, etc., which is not limited here.

[0097] In some embodiments, the preset initial trajectory features may be features obtained by random initialization.

[0098] In some embodiments, the preset initial trajectory feature may also be a feature obtained through continuous training and updating during the model training process.

[0099] Figure 6 A schematic structural diagram of a first prediction network according to an exemplary embodiment of the present disclosure is shown.

[0100] In some embodiments, Figure 6As shown, the first prediction network may include n layers of prediction subnetworks. The initial trajectory feature, the first feature, and the initial waypoint sequence may be input into the feature interaction subnetwork 611 in the first layer prediction subnetwork, so that the feature interaction subnetwork 611 extracts the intrinsic features of each input information, and performs feature interaction and fusion on the intrinsic features of each input information under the guidance of the waypoint sequence, and outputs a second trajectory feature that fuses the waypoint sequence feature and the first information feature, and then the second trajectory feature is input into the decoding subnetwork 612 in the first layer prediction subnetwork to decode and obtain the corresponding second predicted trajectory.

[0101] On this basis, the second predicted trajectory output by the first layer prediction sub-network can be sampled by trajectory points, so as to obtain a trajectory point sequence and use it as the first trajectory point sequence for inputting into the second layer prediction sub-network, and at the same time, the second trajectory feature output by the first layer prediction sub-network is used as the first trajectory feature for inputting into the second layer prediction sub-network. The first trajectory point sequence, the first trajectory feature and the first feature are input into the feature interaction sub-network 621 in the second layer prediction sub-network, so that the feature interaction sub-network 621 extracts the intrinsic features of each input information, and performs feature interaction and fusion on the intrinsic features of each input information under the guidance of the first trajectory point sequence, and outputs a second trajectory feature that is further optimized on the basis of the first trajectory feature, and then the second trajectory feature is input into the decoding sub-network 622 in the second layer prediction sub-network to decode and obtain the iteratively optimized second predicted trajectory.

[0102] Subsequently, the second trajectory feature and the second predicted trajectory are iteratively optimized through each layer of prediction sub-network by a method similar to the above, until the second trajectory feature output by the feature interaction sub-network 631 in the n-th layer prediction sub-network and the second predicted trajectory output by the decoding sub-network 632 are obtained, and the second trajectory feature and the second predicted trajectory are respectively determined as the predicted trajectory feature and the predicted trajectory corresponding to the waypoint sequence.

[0103] It is understandable that the number of layers of the above prediction sub-network can be set based on actual needs and is not limited here.

[0104] In some embodiments, Figure 7As shown, a method for training a trajectory prediction model is provided, the method comprising: step S701, obtaining sample data, the sample data comprising first information of a sample vehicle and multiple waypoint sequences, the first information comprising sample navigation information of the sample vehicle and sample perception information of the surrounding environment of the sample vehicle, each waypoint sequence consisting of multiple waypoints in a future route section and each waypoint sequence is annotated with a corresponding score label, and the multiple waypoint sequences include a positive sample sequence; step S702, for each waypoint sequence of the sample data, inputting the waypoint sequence and the first information into a trajectory prediction model, so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence, and outputs a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; and step S703, adjusting parameters of the trajectory prediction model based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence.

[0105] Through the above method, a trajectory prediction model can be trained. The model can obtain multiple possible waypoint sequences of the target vehicle on the road section in the future, and under the guidance of each waypoint sequence, predict multiple predicted trajectories corresponding to the multiple waypoint sequences and the score of each predicted trajectory, and determine the target predicted trajectory based on the score of each predicted trajectory. In this way, the trajectory jump problem caused by segmented trajectory prediction can be avoided, and the problem of insufficient lateral avoidance, deceleration, etc. in the entire predicted trajectory can be avoided, thereby improving the rationality of trajectory prediction.

[0106] Among them, the trajectory prediction model can be constructed using the network structure described above. The sample navigation information and sample perception information of the surrounding environment of the sample vehicle included in the first information of the sample vehicle are similar to the navigation information and perception information described above, and the method for obtaining multiple waypoint sequences of the sample vehicle is similar to the operation method described above, which will not be repeated here.

[0107] In some embodiments, the first information may also include at least one of map information and sample motion state information of a sample vehicle.

[0108] In some embodiments, adjusting the parameters of the trajectory prediction model based on the positive sample sequence and the corresponding predicted trajectory and the scores and score labels corresponding to each waypoint sequence may include: calculating a first loss based on the positive sample sequence and the corresponding predicted trajectory; calculating a second loss based on the scores and score labels corresponding to each waypoint sequence of the sample data; and adjusting the parameters of the trajectory prediction model based on the first loss and the second loss.

[0109] In some embodiments, based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence, adjusting the parameters of the trajectory prediction model can be based on the difference between each positive sample sequence and the corresponding predicted trajectory, calculating a part of the loss based on the difference between the score and the score label corresponding to each waypoint sequence, and adjusting the parameters of the trajectory prediction model based on the two parts of the loss.

[0110] In some embodiments, multiple waypoint sequences can also be obtained based on the following operations: based on the sample motion state of the sample vehicle, determining the future route segment that the sample vehicle may pass through within a preset time period in the future; sampling the future route segment at a preset distance interval to obtain multiple sampling positions on the future route segment; acquiring at least one waypoint on the road section at each sampling position in the multiple sampling positions; and based on each waypoint at each sampling position in the multiple sampling positions, traversing the sequence of all possible waypoints that the sample vehicle may pass through to obtain multiple waypoint sequences.

[0111] In some embodiments, at least one waypoint includes an intersection of a road section at a corresponding sampling position and at least one lane centerline.

[0112] In some embodiments, a positive sample sequence is marked among the multiple waypoint sequences to represent the optimal trajectory of the sample vehicle under the conditions corresponding to the sample data; accordingly, other waypoint sequences other than the positive sample sequence are marked as negative sample sequences. Each waypoint sequence is marked with a score label, for example, the score of the positive sample sequence can be marked as 1, and the score of the negative sample sequence can be marked as 0.

[0113] In some embodiments, the trajectory prediction model includes a trajectory prediction network and a scoring network. For each waypoint sequence of the sample data, the waypoint sequence and the first information are input into the trajectory prediction model so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence. Outputting the predicted trajectory corresponding to the waypoint sequence and the score of the predicted trajectory may include: for each waypoint sequence of the sample data, using the trajectory prediction network, performing trajectory prediction based on the first information under the guidance of the waypoint sequence to obtain the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory features corresponding to the predicted trajectory; and inputting the predicted trajectory features corresponding to each waypoint sequence of the sample data into the scoring network respectively to obtain the score of the predicted trajectory corresponding to the waypoint sequence output by the scoring network.

[0114] Among them, the network structure used by the trajectory prediction network and the scoring network and the operations performed are similar to the trajectory prediction network and the scoring network mentioned above, and will not be repeated here.

[0115] In some embodiments, the trajectory prediction network may include a first encoding network and a first prediction network. For each waypoint sequence corresponding to the sample data, the trajectory prediction network is used to perform trajectory prediction based on the first information of the sample data under the guidance of the waypoint sequence to obtain the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory. The method may include: inputting the first information of the sample data into the first encoding network so that the first encoding network extracts intrinsic features of the first information of the sample data and outputs the first feature; and for each waypoint sequence corresponding to the sample data, inputting the waypoint sequence and the first feature into the first prediction network so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory.

[0116] Among them, the network structure applied by the first encoding network and the first prediction network in the trajectory prediction network and the operations performed are similar to the first encoding network and the first prediction network described above, and are not described in detail here.

[0117] In some embodiments, the first prediction network may include a multi-layer prediction subnetwork. For each waypoint sequence corresponding to the sample data, the waypoint sequence and the first feature are input into the first prediction network so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory. The method may include: for each waypoint sequence corresponding to the sample data, performing the following operations: based on each layer of prediction subnetworks in the multi-layer prediction subnetwork, the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the layer of prediction subnetwork are input into the layer of prediction subnetwork to obtain the second trajectory feature and the corresponding second predicted trajectory output by the layer of prediction subnetwork, wherein the first trajectory feature corresponding to the first layer of prediction subnetwork in the multi-layer prediction subnetwork is the preset initial trajectory feature, and the first trajectory feature of each layer of prediction subnetwork other than the first layer of prediction subnetwork in the multi-layer prediction subnetwork is the second trajectory feature output by the previous layer of prediction subnetwork; and the second trajectory feature and the second predicted trajectory output by the last layer of prediction subnetwork in the multi-layer prediction subnetwork are determined as the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory.

[0118] Among them, the network structure applied by the multi-layer prediction sub-network in the first prediction network and the operations performed are similar to the multi-layer prediction sub-network in the first prediction network described above, and will not be described in detail here.

[0119] In some embodiments, similar to the multi-layer prediction sub-network described above, the first trajectory point sequence of the first layer prediction sub-network can be a waypoint sequence, and the first trajectory point sequence of each subsequent layer prediction sub-network can be obtained by sampling the trajectory points of the predicted trajectory output by the previous layer prediction sub-network.

[0120] In some embodiments, when the waypoint sequence is a positive sample sequence, the first trajectory point sequence of each layer of prediction subnetwork in the multi-layer prediction subnetwork is the waypoint sequence after adding noise, and as the number of layers of the prediction subnetwork increases, the added noise gradually decreases. When the waypoint sequence is a negative sample sequence, the first trajectory point sequence corresponding to the first layer of prediction subnetwork is the waypoint sequence, and the first trajectory point sequence of other layers of prediction subnetworks is obtained by trajectory point sampling of the second prediction trajectory output by the previous layer of prediction subnetwork.

[0121] Therefore, by adding noise that decreases layer by layer to the positive sample sequence, the process of iterative optimization of the prediction trajectory by the multi-layer prediction subnetwork is simulated, so that the trained multi-layer prediction subnetwork can continuously iteratively optimize the prediction trajectory to output a more accurate prediction trajectory.

[0122] In some embodiments, each prediction subnetwork in a multi-layer prediction subnetwork may include a feature interaction subnetwork and a decoding subnetwork. Based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, inputting the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of the layer into the prediction subnetwork of the layer to obtain the second trajectory feature and the corresponding second predicted trajectory output by the prediction subnetwork of the layer may include: based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, inputting the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of the layer into the feature interaction subnetwork of the layer prediction subnetwork, so that the feature interaction subnetwork of the layer prediction subnetwork performs interactive analysis on the first feature and the first trajectory feature under the guidance of the first trajectory point sequence and outputs the second trajectory feature; and inputting the second trajectory feature output by the feature interaction subnetwork of the layer prediction subnetwork into the decoding subnetwork of the layer prediction subnetwork to obtain the second predicted trajectory output by the decoding subnetwork of the layer prediction subnetwork.

[0123] Among them, the network structure used by the feature interaction subnetwork and the decoding subnetwork and the operations performed are similar to the feature interaction subnetwork and the decoding subnetwork described above, and will not be repeated here.

[0124] In some embodiments, during the training phase, for the positive sample sequence corresponding to each sample data, the positive sample sequence to which gradually decreasing noise is added is used as the first trajectory point sequence of each layer of the prediction sub-network, so that the process of iterative optimization of the predicted trajectory can be simulated during the training phase, thereby enabling the model to learn to gradually optimize the predicted trajectory and obtain the ability to accurately predict the trajectory.

[0125] In some embodiments, the added noise may be random noise or Gaussian noise, which is not limited herein.

[0126] In some embodiments, during the training process of the model, the intensity of the noise can be adjusted as the model training progresses to control the difficulty of the trajectory prediction task, so that the model can generate a predicted trajectory that is as close to the positive sample sequence as possible at different difficulties.

[0127] Figure 8 A schematic diagram of a training method for a first prediction network according to an embodiment of the present disclosure is shown.

[0128] In some embodiments, Figure 8 As shown, the first prediction network may include n layers of prediction subnetworks. Before the positive sample sequence is input into the first layer prediction subnetwork, the positive sample sequence may be first input into the noise adding module of the first layer prediction subnetwork to obtain the positive sample sequence after noise addition; then, the initial trajectory feature, the first feature and the positive sample sequence after noise addition are input into the feature interaction subnetwork 811 in the first layer prediction subnetwork, so that the feature interaction subnetwork 811 extracts the intrinsic features of each input information, and performs feature interaction and fusion on the intrinsic features of each input information under the guidance of the waypoint sequence, outputs the second trajectory feature that fuses the waypoint sequence feature and the first information feature, and then, inputs the second trajectory feature into the decoding subnetwork 812 in the first layer prediction subnetwork to decode and obtain the corresponding second prediction trajectory.

[0129] On this basis, the second trajectory feature output by the first layer prediction subnetwork is used as the first trajectory feature for input into the second layer prediction subnetwork; the positive sample sequence is input into the noise adding module of the second layer prediction subnetwork to obtain the positive sample sequence after noise addition; then, similar to the above, the first trajectory feature, the first feature and the positive sample sequence after noise addition are input into the feature interaction subnetwork 821 in the first layer prediction subnetwork to obtain the second trajectory feature output by the feature interaction subnetwork 821, and the second trajectory feature is input into the decoding subnetwork 822 in the second layer prediction subnetwork to decode and obtain the second prediction trajectory corresponding to the second layer prediction subnetwork.

[0130] Subsequently, the second trajectory features and the second predicted trajectory output by each layer of the prediction sub-network are obtained by a method similar to the above, until the second trajectory features output by the feature interaction sub-network 831 in the n-th layer of the prediction sub-network and the second predicted trajectory output by the decoding sub-network 832 are obtained, and the second trajectory features and the second predicted trajectory are respectively determined as the predicted trajectory features and the predicted trajectory corresponding to the waypoint sequence.

[0131] In some embodiments, calculating the first loss based on the positive sample sequence and the corresponding prediction trajectory includes: calculating the first loss based on the positive sample sequence of the sample data and the second prediction trajectory output by each layer of the prediction subnetwork in the multi-layer prediction subnetwork corresponding to the positive sample sequence.

[0132] In some embodiments, Figure 8 As shown, the first loss can be calculated based on the difference between the positive sample sequence and the second prediction trajectory output by each layer of the prediction subnetwork. In some embodiments, the losses of each layer can be added to obtain the first loss, and the overall first prediction network can be trained, thereby improving the training efficiency and training effect of the model.

[0133] In some embodiments, after obtaining the loss corresponding to each layer of the prediction sub-network, the prediction sub-network of this layer and its previous layers can be trained based on the loss, thereby further making the prediction trajectory output by each layer closer to the true value trajectory, thereby further improving the overall prediction accuracy of the model.

[0134] In some embodiments, the preset initial trajectory feature may be a randomly generated feature.

[0135] In some embodiments, the training method of the trajectory prediction model may further include: training initial trajectory features based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence.

[0136] Therefore, by continuously iterating and updating the initial trajectory features while training the model, in the model application stage, by applying the trained initial trajectory features, the model can perform trajectory prediction based on a better performing initial trajectory feature, thereby further improving the prediction effect of the trajectory prediction model.

[0137] In some embodiments, Fig. 9As shown, a trajectory prediction device 900 is provided, and the device 900 includes: a first acquisition unit 910, configured to obtain first information of a target vehicle and multiple waypoint sequences, the first information includes navigation information of the target vehicle and perception information of the surrounding environment of the target vehicle, and each waypoint sequence is composed of multiple waypoints in a future route segment; a second acquisition unit 920, configured to perform trajectory prediction based on the first information for each waypoint sequence in the multiple waypoint sequences under the guidance of the waypoint sequence, so as to obtain a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; a first determination unit 930, configured to determine a target predicted trajectory from multiple predicted trajectories based on the scores of each predicted trajectory in the multiple predicted trajectories.

[0138] In some embodiments, Fig.10 As shown, a training device 1000 for a trajectory prediction model is provided, and the device 1000 includes: a first acquisition unit 1010, configured to acquire sample data, the sample data includes first information of a sample vehicle and multiple waypoint sequences, the first information includes sample navigation information of the sample vehicle and sample perception information of the surrounding environment of the sample vehicle, each waypoint sequence is composed of multiple waypoints in a future route section and each waypoint sequence is annotated with a corresponding score label, and the multiple waypoint sequences include a positive sample sequence; a first input unit 1020, configured to input the waypoint sequence and the first information into the trajectory prediction model for each waypoint sequence of the sample data, so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence, and outputs a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; and an adjustment unit 1030, configured to adjust the parameters of the trajectory prediction model based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence.

[0139] It should be understood that Fig. 9 The various units of the device 900 shown in FIG. Figure 2 Thus, the operations, features and advantages described above for the trajectory prediction method implemented using the trajectory prediction model are also applicable to the device 900 and the various units included therein; and Fig.10 The various units of the device 1000 shown in FIG. Figure 7 The steps in the training method of the trajectory prediction model described above correspond to each other. Therefore, the operations, features and advantages described above for the training method of the trajectory prediction model are also applicable to the device 1000 and the various units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0140] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0141] refer to Fig.11 , a block diagram of an electronic device 1100 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0142] like Fig.11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the electronic device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0143] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including: an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. The input unit 1106 can be any type of device that can input information to the electronic device 1100. The input unit 1106 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 1107 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1108 can include but is not limited to a disk and an optical disk. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device and / or the like.

[0144] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the trajectory prediction method implemented using the trajectory prediction model or the training method of the trajectory prediction model. For example, in some embodiments, the trajectory prediction method implemented using the trajectory prediction model or the training method of the trajectory prediction model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the trajectory prediction method implemented by the trajectory prediction model or the training method of the trajectory prediction model can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to execute the trajectory prediction method implemented by the trajectory prediction model or the training method of the trajectory prediction model by any other appropriate means (for example, by means of firmware).

[0145] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0147] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0149] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0150] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0151] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0152] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A trajectory prediction method, comprising: Acquire first information of a target vehicle and a plurality of waypoint sequences, wherein the first information includes navigation information of the target vehicle and perception information of the surrounding environment of the target vehicle, and each waypoint sequence is composed of a plurality of waypoints in a future route segment; For each waypoint sequence in the plurality of waypoint sequences, performing trajectory prediction based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; as well as Based on the scores of the respective predicted trajectories in the plurality of predicted trajectories, a target predicted trajectory is determined among the plurality of predicted trajectories.

2. The method according to claim 1, wherein: The method is implemented based on a trajectory prediction model, the trajectory prediction model includes a trajectory prediction network and a scoring network, and for each waypoint sequence in the plurality of waypoint sequences, trajectory prediction is performed based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a score corresponding to the predicted trajectory, including: For each waypoint sequence in the plurality of waypoint sequences, using the trajectory prediction network, under the guidance of the waypoint sequence, perform trajectory prediction based on the first information to obtain a predicted trajectory corresponding to the waypoint sequence and a predicted trajectory feature corresponding to the predicted trajectory; and The multiple predicted trajectory features corresponding to the multiple waypoint sequences are respectively input into the scoring network to obtain the score of each predicted trajectory in the multiple predicted trajectories corresponding to the multiple waypoint sequences output by the scoring network.

3. The method according to claim 2, wherein: The trajectory prediction network includes a first encoding network and a first prediction network. For each waypoint sequence in the plurality of waypoint sequences, the trajectory prediction network is used to perform trajectory prediction based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a predicted trajectory feature corresponding to the predicted trajectory, including: Inputting the first information into the first encoding network so that the first encoding network extracts intrinsic features from the first information and outputs first features; and For each waypoint sequence in the multiple waypoint sequences, the waypoint sequence and the first feature are input into the first prediction network, so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory.

4. The method according to claim 3, wherein: The first prediction network includes a multi-layer prediction sub-network, and for each waypoint sequence in the plurality of waypoint sequences, the waypoint sequence and the first feature are input into the first prediction network, so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory, including: For each waypoint sequence in the plurality of waypoint sequences, perform the following operations: Based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the layer of prediction subnetwork are input into the layer of prediction subnetwork to obtain the second trajectory feature and the corresponding second predicted trajectory output by the layer of prediction subnetwork, wherein the first trajectory feature corresponding to the first layer of prediction subnetwork in the multi-layer prediction subnetwork is a preset initial trajectory feature, the first trajectory point sequence corresponding to the first layer of prediction subnetwork is the waypoint sequence, the first trajectory feature of each layer of prediction subnetwork other than the first layer of prediction subnetwork in the multi-layer prediction subnetwork is the second trajectory feature output by the previous layer of prediction subnetwork, and the first trajectory point sequence of the other layers of prediction subnetwork is obtained by sampling trajectory points of the second predicted trajectory output by the previous layer of prediction subnetwork; and The second trajectory features and the second predicted trajectory output by the last layer of prediction subnetwork in the multi-layer prediction subnetwork are determined as the predicted trajectory features and the predicted trajectory corresponding to the waypoint sequence.

5. The method according to claim 4, wherein: Each prediction subnetwork in the multi-layer prediction subnetwork includes a feature interaction subnetwork and a decoding subnetwork, and the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of each layer in the multi-layer prediction subnetwork are input into the prediction subnetwork of the layer to obtain the second trajectory feature and the corresponding second predicted trajectory output by the prediction subnetwork of the layer, including: Based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of the layer are input into the feature interaction subnetwork of the prediction subnetwork of the layer, so that the feature interaction subnetwork of the prediction subnetwork of the layer performs interactive analysis on the first feature and the first trajectory feature under the guidance of the first trajectory point sequence, and outputs a second trajectory feature; and The second trajectory feature output by the feature interaction subnetwork of the prediction subnetwork of this layer is input into the decoding subnetwork of the prediction subnetwork of this layer to obtain the second prediction trajectory output by the decoding subnetwork of the prediction subnetwork of this layer.

6. The method according to any one of claims 1 to 5, wherein: The plurality of waypoint sequences are obtained based on the following operations: Based on the current motion state of the target vehicle, predicting the future route section that the target vehicle may pass through within a future preset time period; Sampling the future route section at preset distance intervals to obtain a plurality of sampling positions on the future route section; Acquire at least one waypoint on a road section at each sampling position among the plurality of sampling positions; as well as Based on each waypoint at each sampling position in the multiple sampling positions, a sequence of all possible waypoints that the target vehicle may pass through is traversed to obtain the multiple waypoint sequences.

7. The method according to claim 6, wherein: The at least one waypoint includes an intersection of a road section at a corresponding sampling position and at least one lane centerline.

8. The method according to any one of claims 1 to 7, wherein: The first information also includes at least one of map information and current motion state information of the target vehicle.

9. A method for training a trajectory prediction model, the method comprising: Acquire sample data, the sample data including first information of a sample vehicle and a plurality of waypoint sequences, the first information including sample navigation information of the sample vehicle and sample perception information of the surrounding environment of the sample vehicle, each waypoint sequence consisting of a plurality of waypoints in a future route segment and each waypoint sequence being annotated with a corresponding score label, and the plurality of waypoint sequences including a positive sample sequence; For each waypoint sequence of the sample data, inputting the waypoint sequence and the first information into the trajectory prediction model, so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence, and outputs a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; as well as Based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence, the parameters of the trajectory prediction model are adjusted.

10. The method according to claim 9, wherein: The trajectory prediction model includes a trajectory prediction network and a scoring network. For each waypoint sequence of the sample data, the waypoint sequence and the first information are input into the trajectory prediction model, so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence, and outputs a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory, including: For each waypoint sequence of the sample data, using the trajectory prediction network, under the guidance of the waypoint sequence, perform trajectory prediction based on the first information to obtain a predicted trajectory corresponding to the waypoint sequence and a predicted trajectory feature corresponding to the predicted trajectory; and The predicted trajectory features corresponding to each waypoint sequence of the sample data are respectively input into the scoring network to obtain the score of the predicted trajectory corresponding to the waypoint sequence output by the scoring network.

11. The method according to claim 10, wherein: The trajectory prediction network includes a first encoding network and a first prediction network. For each waypoint sequence of the sample data, the trajectory prediction network is used to perform trajectory prediction based on the first information under the guidance of the waypoint sequence to obtain a predicted trajectory corresponding to the waypoint sequence and a predicted trajectory feature corresponding to the predicted trajectory, including: Inputting the first information into the first encoding network so that the first encoding network extracts intrinsic features from the first information and outputs first features; and For each waypoint sequence corresponding to the sample data, the waypoint sequence and the first feature are input into the first prediction network, so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory.

12. The method according to claim 11, wherein: The first prediction network includes a multi-layer prediction subnetwork, and for each waypoint sequence corresponding to the sample data, the waypoint sequence and the first feature are input into the first prediction network, so that the first prediction network performs trajectory prediction based on the first feature under the guidance of the waypoint sequence and outputs a predicted trajectory corresponding to the waypoint sequence and a predicted trajectory feature corresponding to the predicted trajectory, including: For each waypoint sequence corresponding to the sample data, perform the following operations: Based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the layer of prediction subnetwork are input into the layer of prediction subnetwork to obtain the second trajectory feature output by the layer of prediction subnetwork and the corresponding second predicted trajectory, wherein the first trajectory feature corresponding to the first layer of prediction subnetwork in the multi-layer prediction subnetwork is a preset initial trajectory feature, and the first trajectory feature of each layer of prediction subnetwork other than the first layer of prediction subnetwork in the multi-layer prediction subnetwork is the second trajectory feature output by the previous layer of prediction subnetwork; and The second trajectory feature and the second predicted trajectory output by the last layer of prediction subnetwork in the multi-layer prediction subnetwork are determined as the predicted trajectory corresponding to the waypoint sequence and the predicted trajectory feature corresponding to the predicted trajectory.

13. The method according to claim 12, wherein: When the waypoint sequence is a positive sample sequence, the first trajectory point sequence of each layer of prediction subnetwork in the multi-layer prediction subnetwork is the waypoint sequence after adding noise, and as the number of layers of the prediction subnetwork increases, the added noise gradually decreases. When the waypoint sequence is a negative sample sequence, the first trajectory point sequence corresponding to the first layer of prediction subnetwork is the waypoint sequence, and the first trajectory point sequences of the prediction subnetworks in other layers are obtained by performing trajectory point sampling on the second prediction trajectory output by the prediction subnetwork in the previous layer.

14. The method according to claim 12 or 13, wherein: Each prediction subnetwork in the multi-layer prediction subnetwork includes a feature interaction subnetwork and a decoding subnetwork, and the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of each layer in the multi-layer prediction subnetwork are input into the prediction subnetwork of the layer to obtain the second trajectory feature and the corresponding second predicted trajectory output by the prediction subnetwork of the layer, including: Based on each layer of prediction subnetwork in the multi-layer prediction subnetwork, the first feature and the first trajectory feature and the first trajectory point sequence corresponding to the prediction subnetwork of the layer are input into the feature interaction subnetwork of the prediction subnetwork of the layer, so that the feature interaction subnetwork of the prediction subnetwork of the layer performs interactive analysis on the first feature and the first trajectory feature under the guidance of the first trajectory point sequence, and outputs a second trajectory feature; and The second trajectory feature output by the feature interaction subnetwork of the prediction subnetwork of this layer is input into the decoding subnetwork of the prediction subnetwork of this layer to obtain the second prediction trajectory output by the decoding subnetwork of the prediction subnetwork of this layer.

15. The method according to any one of claims 12 to 14, wherein: The adjusting the parameters of the trajectory prediction model based on the positive sample sequence and the corresponding predicted trajectory and the scores and score labels corresponding to each waypoint sequence includes: Calculating a first loss based on the positive sample sequence and the corresponding predicted trajectory; Calculating a second loss based on the scores and score labels corresponding to each waypoint sequence of the sample data; and Based on the first loss and the second loss, parameters of the trajectory prediction model are adjusted.

16. The method according to claim 15, wherein: The calculating the first loss based on the positive sample sequence and the corresponding predicted trajectory includes: The first loss is calculated based on the positive sample sequence of the sample data and the second prediction trajectory output by each layer of the prediction subnetwork in the multi-layer prediction subnetwork corresponding to the positive sample sequence.

17. The method according to any one of claims 12 to 16, further comprising: The initial trajectory feature is trained based on the positive sample sequence and the corresponding predicted trajectory as well as the score and score label corresponding to each waypoint sequence.

18. The method according to any one of claims 9 to 17, wherein: The plurality of waypoint sequences are obtained based on the following operations: Based on the sample motion state of the sample vehicle, determining the future route section that the sample vehicle may pass through within the future preset time period; Sampling the future route section at preset distance intervals to obtain a plurality of sampling positions on the future route section; Acquire at least one waypoint on a road section at each sampling position among the plurality of sampling positions; as well as Based on each waypoint at each sampling position in the multiple sampling positions, a sequence of all possible waypoints passed by the sample vehicle is traversed to obtain the multiple waypoint sequences.

19. The method according to claim 18, wherein: The at least one waypoint includes an intersection of a road section at a corresponding sampling position and at least one lane centerline.

20. The method according to any one of claims 9 to 19, wherein: The first information also includes at least one of map information and sample motion state information of the sample vehicle.

21. A trajectory prediction device, comprising: A first acquisition unit is configured to acquire first information of a target vehicle and a plurality of waypoint sequences, wherein the first information includes navigation information of the target vehicle and perception information of a surrounding environment of the target vehicle, and each waypoint sequence is composed of a plurality of waypoints in a future route segment; a second acquisition unit configured to perform trajectory prediction based on the first information for each waypoint sequence in the plurality of waypoint sequences under the guidance of the waypoint sequence, so as to acquire a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; as well as The first determining unit is configured to determine a target predicted trajectory among the multiple predicted trajectories based on a score of each predicted trajectory among the multiple predicted trajectories.

22. A training device for a trajectory prediction model, the device comprising: a first acquisition unit, configured to acquire sample data, the sample data including first information of a sample vehicle and a plurality of waypoint sequences, the first information including sample navigation information of the sample vehicle and sample perception information of the surrounding environment of the sample vehicle, each waypoint sequence consisting of a plurality of waypoints in a future route section and each waypoint sequence being annotated with a corresponding score label, and the plurality of waypoint sequences including a positive sample sequence; a first input unit configured to input, for each waypoint sequence of the sample data, the waypoint sequence and the first information into the trajectory prediction model, so that the trajectory prediction model performs trajectory prediction based on the first information under the guidance of the waypoint sequence, and outputs a predicted trajectory corresponding to the waypoint sequence and a score of the predicted trajectory; as well as The adjustment unit is configured to adjust the parameters of the trajectory prediction model based on the positive sample sequence and the corresponding predicted trajectory and the score and score label corresponding to each waypoint sequence.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 20.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-20.

25. A computer program product comprising a computer program, wherein: The computer program implements the method according to any one of claims 1 to 20 when executed by a processor.

26. An autonomous driving vehicle comprising: The electronic device as claimed in claim 23.

Images