Target object trajectory determination method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a target object trajectory determination method and device, electronic equipment and a storage medium, wherein the method comprises: generating a target scene image corresponding to a current time frame based on perception results corresponding to each time frame in at least one time frame and a first area map corresponding to the current time frame; determining first features corresponding to each target object in the current time frame based on the target scene image; determining target features corresponding to each target object in the current time frame based on the perception results corresponding to each time frame, the target features comprising time sequence features corresponding to the target object and / or interaction features corresponding to the target object; and determining driving trajectories of each target object in a subsequent time frame based on the first features and the target features corresponding to each target object. The embodiments of the present disclosure can effectively improve the prediction accuracy, so that higher prediction accuracy can be obtained in a dense obstacle scene.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for determining the trajectory of a target object. Background Technology

[0002] In applications such as autonomous driving and assisted driving, obstacle trajectory prediction is a key task. Obstacle trajectory prediction can predict the trajectory of an obstacle over a future period based on historical information obtained from perception over a certain period, providing obstacle reference information for vehicle planning and control. Related technologies typically employ convolutional neural networks (CNNs) for obstacle trajectory prediction from a bird's-eye view (BEV) perspective, based on historical obstacle information and map element information. However, CNN-based trajectory prediction methods have lower prediction accuracy in obstacle-dense scenarios. Summary of the Invention

[0003] To address the aforementioned technical problems, such as low prediction accuracy in obstacle-dense scenarios, embodiments of this disclosure provide a method, apparatus, electronic device, and storage medium for determining the trajectory of a target object, thereby effectively improving prediction accuracy.

[0004] The first aspect of this disclosure provides a method for determining the trajectory of a target object, comprising: acquiring perception results corresponding to each of at least one time frame and a first region map corresponding to the current time frame, wherein the at least one time frame includes the current time frame, and the perception results include state information of at least one target object in a first coordinate system; generating a target scene image corresponding to the current time frame based on the perception results corresponding to each of the time frames and the first region map; determining a first feature corresponding to each of the target objects corresponding to the current time frame based on the target scene image; determining a target feature corresponding to each of the target objects corresponding to the current time frame based on the perception results corresponding to each of the time frames, wherein the target feature includes a temporal feature corresponding to the target object and / or an interaction feature corresponding to the target object, the interaction feature representing the correlation between the target object and at least one other target object in its vicinity; and determining the driving trajectory of each target object in a later time frame based on the first feature and the target feature corresponding to each of the target objects.

[0005] A second aspect of this disclosure provides a trajectory determination device for a target object, comprising: a first acquisition module, configured to acquire perception results corresponding to each of at least one time frame and a first region map corresponding to the current time frame, wherein the at least one time frame includes the current time frame, and the perception results include state information of at least one target object in a first coordinate system; a first processing module, configured to generate a target scene image corresponding to the current time frame based on the perception results corresponding to each of the time frames and the first region map; a second processing module, configured to determine a first feature corresponding to each of the target objects corresponding to the current time frame based on the target scene image; a third processing module, configured to determine a target feature corresponding to each of the target objects corresponding to the current time frame based on the perception results corresponding to each of the time frames, wherein the target feature includes a temporal feature corresponding to the target object and / or an interaction feature corresponding to the target object, and the interaction feature characterizes the correlation between the target object and at least one other target object in its vicinity; and a fourth processing module, configured to determine the driving trajectory of each target object in a later time frame based on the first feature and the target feature corresponding to each of the target objects.

[0006] A third aspect of this disclosure provides a computer-readable storage medium storing a computer program for executing the trajectory determination method for a target object as described in any of the above embodiments of this disclosure.

[0007] A fourth aspect of this disclosure provides an electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the trajectory determination method for a target object as described in any of the above embodiments of this disclosure.

[0008] A fifth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs the trajectory determination method for a target object as described in any of the above embodiments of this disclosure.

[0009] Based on the target object trajectory determination method, apparatus, electronic device and storage medium provided in the above embodiments of this disclosure, when predicting the future trajectory of the target object, the historical information, current information of the target object, as well as the temporal characteristics of the target object and / or the interaction characteristics with other objects can be combined to predict the future driving trajectory of the target object. Since the temporal characteristics help to capture the historical motion patterns of the target object, and the interaction characteristics can characterize the relationship between the target object and other surrounding objects, the prediction accuracy can be effectively improved, and higher prediction accuracy can be obtained even in obstacle-dense scenes.

[0010] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0011] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is an exemplary application scenario of the trajectory determination method for the target object provided in this disclosure;

[0013] Figure 2 This is a flowchart illustrating a method for determining the trajectory of a target object provided in an exemplary embodiment of this disclosure;

[0014] Figure 3 This is a flowchart illustrating a method for determining the trajectory of a target object provided in another exemplary embodiment of this disclosure;

[0015] Figure 4 This is a schematic diagram of a raster image provided in an exemplary embodiment of the present disclosure;

[0016] Figure 5 This is a schematic diagram of the current coordinate system of the target object provided in an exemplary embodiment of this disclosure;

[0017] Figure 6 This is a schematic diagram illustrating the cropping principle of local image features provided in an exemplary embodiment of this disclosure;

[0018] Figure 7 This is a flowchart illustrating a method for determining the trajectory of a target object provided in yet another exemplary embodiment of this disclosure;

[0019] Figure 8 This is a schematic diagram of an anchoring trajectory provided as an exemplary example of this disclosure;

[0020] Figure 9 This is a schematic diagram of the network structure of a trajectory prediction model provided in an exemplary embodiment of this disclosure;

[0021] Figure 10 This is a schematic diagram of the structure of an attention interaction network provided in an exemplary embodiment of this disclosure;

[0022] Figure 11 This is a schematic diagram of the structure of a trajectory determination device for a target object provided in an exemplary embodiment of the present disclosure;

[0023] Figure 12 This is a schematic diagram of the structure of a target object trajectory determination device provided in another exemplary embodiment of this disclosure;

[0024] Figure 13 This is a structural diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0025] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0026] It should be noted that, unless otherwise specifically stated, the components and steps described in these embodiments are similar.

[0027] The scope of this disclosure is not limited by the arrangement, numerical expressions, and numerical values.

[0028] This disclosure outlines

[0029] In developing this disclosure, the inventors discovered that obstacle trajectory prediction is a key task in applications such as autonomous driving and assisted driving. Obstacle trajectory prediction can predict the trajectory of an obstacle over a future period based on historical information obtained from perception over a certain time, providing obstacle reference information for vehicle planning and control. In related technologies, obstacle trajectory prediction is typically performed using convolutional neural networks based on historical obstacle information and map element information from a bird's-eye view (BEV) perspective. However, trajectory prediction methods based on convolutional neural networks have low prediction accuracy in obstacle-dense scenarios.

[0030] Exemplary Overview

[0031] Figure 1 This is an exemplary application scenario of the trajectory determination method for the target object provided in this disclosure.

[0032] In scenarios such as autonomous driving and assisted driving, the surrounding environment can be perceived based on sensors on the vehicle during driving, and perception results can be obtained for subsequent prediction of the future trajectory of obstacles (target objects), thereby providing obstacle reference information for vehicle planning and control. Using the target object trajectory determination method disclosed herein, during vehicle driving, the perception results corresponding to each time frame in at least one time frame (the perception results of multiple time frames are referred to as perception result data) and a first area map corresponding to the current time frame can be obtained in the current time frame. The first area map can be a map of a preset range covering each object around the vehicle in the current time frame that contains the perception results, such as a map within a rectangular area of ​​50 meters in front and behind the vehicle, and 30 meters to the left and right. The specific range of the first area map is not limited. Based on the perception results and the first region map corresponding to each time frame, a target scene image corresponding to the current time frame is generated. The target scene image can be an image that fuses object information from the perception results of each time frame and the first region map. Based on the target scene image, the first feature corresponding to each target object in the current time frame is determined. Based on the perception results corresponding to each time frame, the target feature corresponding to each target object in the current time frame is determined. The target feature includes the temporal feature corresponding to the target object and / or the interaction feature corresponding to the target object. The interaction feature characterizes the correlation between the target object and at least one other target object in its vicinity. Based on the first feature and target feature corresponding to each target object, the driving trajectory of each target object in the subsequent time frame is determined. Since the temporal feature helps to capture the historical motion pattern of the target object, and the interaction feature can characterize the relationship between the target object and other objects in its vicinity, the prediction accuracy can be effectively improved, enabling higher prediction accuracy even in obstacle-dense scenes.

[0033] Exemplary methods

[0034] Figure 2 This is a flowchart illustrating a method for determining the trajectory of a target object according to an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically, for example, in-vehicle computing platforms. Figure 2 As shown, it includes the following steps:

[0035] Step 201: Obtain the perception results corresponding to each time frame in at least one time frame and the first region map corresponding to the current time frame.

[0036] The system includes at least one time frame, including the current time frame, and the perception result includes the state information of at least one target object in a first coordinate system. The state information may include at least one of the following: the target object's position, velocity, acceleration, orientation (angle), size, and type. The type refers to the target object's classification type, such as vehicle, pedestrian, or other possible types. The first coordinate system can be a world coordinate system, a map coordinate system, or the vehicle coordinate system of the corresponding time frame, which can be set according to actual needs. The coordinate systems can be converted to each other. For the first area map corresponding to the current time frame, a map of a preset range around the vehicle's current position can be obtained as the first area map corresponding to the current time frame, based on the vehicle's current position on the map. The specific preset range can be set according to actual needs and is not limited in this disclosure.

[0037] In one alternative embodiment, the first area map may include map element information such as lane lines, curbs, zebra crossings, and road signs.

[0038] In an optional embodiment, at least one time frame may also include at least one historical time frame. The specific number of time frames can be set according to actual needs. For example, at least one time frame may include 4 frames, 5 frames, 6 frames, etc.

[0039] In an optional embodiment, the time interval between adjacent time frames can be referred to as the first time interval. The first time interval can be set according to actual needs, such as 0.5 seconds. The first time interval can be greater than the time interval of the perception result data obtained by the vehicle perception module in the perception processing. When it is necessary to determine the trajectory of the target object, sampling can be performed on the perception result data obtained by the perception module based on the first time interval to obtain the perception result corresponding to each time frame of at least one time frame.

[0040] Step 202: Based on the perception results and the first region map corresponding to each time frame, generate the target scene image corresponding to the current time frame.

[0041] The target scene image can be an image that fuses object information from perception results across different time frames with a first-region map. The specific fusion method can be set according to actual needs. For example, based on the state information of each object in the perception results of each time frame, each object can be distributed onto the grid coordinate system corresponding to the first-region map in a certain representation. For example, objects can be distributed onto the grid coordinate system in a preset shape (such as rectangle, circle, ellipse, etc.) according to the ratio between the object size and the grid coordinate system, generating a grid image. The grid image is then fused with the first-region map to obtain the target scene image.

[0042] Step 203: Based on the target scene image, determine the first feature corresponding to each target object in the current time frame.

[0043] The first feature can be obtained by any feasible feature extraction method, such as extracting features from the target scene image based on a pre-trained feature extraction network. The feature extraction network can adopt any feasible network structure, such as any feature extraction network based on a convolutional neural network. This disclosure does not limit the method.

[0044] Step 204: Based on the perception results corresponding to each time frame, determine the target features corresponding to each target object in the current time frame. The target features include the temporal features and / or the interaction features corresponding to the target object.

[0045] Interaction features characterize the correlation between a target object and at least one other target object in its vicinity. Temporal features characterize the historical movement patterns of an object.

[0046] In one alternative embodiment, the temporal features can be obtained based on any feasible temporal feature extraction method. For example, the temporal features of the target object can be extracted based on a pre-trained temporal feature extraction network.

[0047] In an alternative embodiment, the interaction features corresponding to the target object can be extracted based on an attention interaction network.

[0048] Step 205: Based on the first feature and target feature corresponding to each target object, determine the driving trajectory of each target object in the later time frame.

[0049] The subsequent time frame can include at least one future time frame, such as multiple time frames within the next 6 seconds. Taking a first time interval of 0.5 seconds as an example, the subsequent time frame can include 12 future time frames, that is, each driving trajectory includes 12 trajectory points of the target object within the next 6 seconds.

[0050] In an alternative embodiment, the driving trajectory of each target object in a later time frame can be obtained based on a pre-trained decoding network and a prediction head network.

[0051] The trajectory determination method for the target object provided in this embodiment can predict the future trajectory of the target object by combining the target object's historical information, current information, temporal features and / or interaction features with other objects. Since temporal features help to capture the historical movement patterns of the target object, and interaction features can characterize the relationship between the target object and other surrounding objects, the prediction accuracy can be effectively improved, enabling higher prediction accuracy even in obstacle-dense scenarios.

[0052] Figure 3 This is a flowchart illustrating a method for determining the trajectory of a target object provided in another exemplary embodiment of this disclosure.

[0053] In an optional example, step 202, based on the perception results and the first region map corresponding to each time frame, generates the target scene image corresponding to the current time frame, including:

[0054] Step 2021: For any time frame, generate a raster image corresponding to that time frame based on the perception result corresponding to that time frame.

[0055] The raster image can be an image in the raster coordinate system corresponding to the map of the first region. The raster coordinate system can be the map coordinate system or a coordinate system with a different origin than the map coordinate system; the specific system can be set according to actual needs. The raster image can include the distribution of each target object in the perception result of that time frame in the raster coordinate system. In the raster image, each target object can be represented by a preset shape, such as a rectangle, a circle, etc.

[0056] Step 2022: Fuse the raster images corresponding to each time frame with the first region map to obtain the target scene image.

[0057] The size of the raster image can be consistent with the size of the first area map or at least consistent in one dimension. The fusion method can be set according to actual needs, such as stitching in a certain dimension.

[0058] This embodiment generates raster images corresponding to each time frame, which facilitates fusion with the first area map. The fused target scene image includes historical state information, current state information, and map element information around each target object, providing effective data for subsequent trajectory determination.

[0059] In an optional embodiment, step 2021, generating a raster image corresponding to the time frame based on the perception result corresponding to the time frame, includes:

[0060] Based on the state information of each target object in the perception result corresponding to the time frame, the position and orientation of each target object in the raster coordinate system corresponding to the raster image are determined; based on the position and orientation of each target object in the raster coordinate system, each target object is rendered in the raster coordinate system according to the preset rendering method to obtain the raster image.

[0061] The target object's state information includes its position, orientation, velocity, acceleration, size, and type in the first coordinate system. Based on this state information, the target object's position and orientation in the first coordinate system can be determined. Furthermore, based on the transformation relationship between the first and raster coordinate systems, the target object's position and orientation in the first coordinate system can be transformed to the raster coordinate system, thus obtaining the target object's position and orientation in the raster coordinate system. Preset rendering methods can be set according to actual needs. These preset rendering methods can include the target object's corresponding rendering shape, shape size, rendering color, etc., as well as the raster image background color. For example, the raster image background can be rendered as black, and the target object as a white rectangular area. The target object's shape and size can be determined based on the ratio between the target object's dimensions and the raster coordinate system. The shape and size of the target object can be determined before rendering based on the ratio between the target object's dimensions and the raster coordinate system, or it can be determined in real-time during rendering based on the ratio between the target object's dimensions and the raster coordinate system; the specific determination is not limited. Different types of target objects can use the same or different shapes, which can be set according to actual needs.

[0062] For example, Figure 4 This is a schematic diagram of a raster image provided in an exemplary embodiment of this disclosure. In this example, the background of the raster image is black, and the target object is rendered in white. White rectangles or white dots in the image represent the target object.

[0063] In an optional embodiment, the first region map is represented as an H*W*C image, and the raster image corresponding to each time frame can be represented as an H*W*1 image, where H represents the image height, W represents the image width, and C represents the number of image channels. If the first region map is an RGB image, the number of channels in the first region map is usually 3, which may include three color channels: R (RED), G (GREEN), and B (BLUE).

[0064] In an optional embodiment, step 2022, fusing the raster images corresponding to each time frame with the first region map to obtain the target scene image, includes:

[0065] The raster images corresponding to each time frame are stitched together with the first region map by channel to obtain the target scene image.

[0066] Among them, stitching by channel means stitching the raster image and the first region map in the channel dimension, so that the size of the target scene image relative to the first region map remains unchanged, but the number of channels increases.

[0067] For example, the target scene image obtained by stitching together the first region map of H*W*3 and the raster images of H*W*1 corresponding to the four time frames by channel is an image of H*W*7.

[0068] In this embodiment, the raster images of each time frame are stitched together with the first region map by channel to form the target scene image, which is used for subsequent trajectory prediction. Since the target scene image includes the historical state, current state and map elements of each target object, it helps to extract global features of the scene and provide effective features for target object trajectory prediction.

[0069] In an optional embodiment, step 203, determining the first feature corresponding to each target object in the current time frame based on the target scene image, includes:

[0070] Step 2031: Use the first feature extraction network in the pre-trained trajectory prediction model to process the target scene image and obtain the first image features.

[0071] The trajectory prediction model is a pre-trained neural network model for predicting object trajectories. The first feature extraction network can be any feasible feature extraction network, such as a feature extraction network based on a convolutional neural network, and can be specifically configured according to actual needs.

[0072] Step 2032: Based on the first image features, determine the local image features corresponding to each target object.

[0073] For any target object, the local image features corresponding to the target object are the features within a certain range around the target object.

[0074] In an optional embodiment, local image features corresponding to the target object can be cropped from the first image features based on the position of the target object in the target scene image and the positional correspondence between the target scene image position and the first image features.

[0075] Step 2033: Use the second feature extraction network in the trajectory prediction model to extract features from the local image features corresponding to each target object to obtain the first feature corresponding to each target object.

[0076] The second feature extraction network can be any feasible feature extraction network, such as a feature extraction network based on a convolutional neural network, and can be set according to actual needs.

[0077] This embodiment further extracts features from the local image features corresponding to each target object to obtain the first feature corresponding to each target object. Thus, the finer-grained local features centered on the target object are used for trajectory prediction of the target object, which can further improve the accuracy of the prediction results.

[0078] In an optional embodiment, step 2032, which determines the local image features corresponding to each target object based on the first image features, includes:

[0079] For any target object, based on the current state information of the target object in the current time frame, the transformation relationship between the current coordinate system of the target object and the coordinate system corresponding to the first image feature is determined. The current coordinate system takes the current position of the target object as the origin and the current orientation of the target object as the first coordinate axis. Based on the transformation relationship, the first image feature is transformed into the current coordinate system to obtain the target image feature corresponding to the first image feature. Based on the preset cropping rules, the sub-image feature corresponding to the target object is determined from the target image feature. The sub-image feature is used as the local image feature corresponding to the target object.

[0080] Here, the current position is the location of the target object in the current time frame, which can be obtained from the current state information. The current coordinate system uses the target object's current position as the origin, the target object's current orientation as the first coordinate axis, and the direction perpendicular to the current orientation as the second coordinate axis. Preset cropping rules can be set according to actual needs; for example, it could crop an M*N pixel region centered on the target object from the target image features as the corresponding sub-image features.

[0081] For example, if the target object is a vehicle, then the current coordinate system is the vehicle coordinate system of the vehicle in the current time frame, that is, with the center of the rear axle of the vehicle as the origin, the length direction of the vehicle as the first coordinate axis, and the width direction of the vehicle as the second coordinate axis.

[0082] For example, Figure 5 This is a schematic diagram of the current coordinate system of a target object provided in an exemplary embodiment of this disclosure. Here, x0y represents the current coordinate system of the target object. Only one target object's current coordinate system is shown in the figure.

[0083] In an optional embodiment, the transformation relationship between the current coordinate system and the coordinate system corresponding to the first image feature can be determined based on the transformation relationship between the current coordinate system and the first coordinate system, the transformation relationship between the first coordinate system and the map coordinate system of the first region map, the mapping relationship between the first region map and the target scene image, and the pixel correspondence between the target scene image and the first image feature.

[0084] In an optional embodiment, the four corner points of the area to be cropped can be determined based on the current coordinate system of the target object. Based on the above transformation relationship, the four corner points are projected onto the first image feature to obtain four projected corner points. The sub-image features corresponding to the target object are cropped based on the four projected corner points, and then the sub-image features are transformed to the current coordinate system to obtain the local image features of the target object.

[0085] For example, Figure 6 This is a schematic diagram illustrating the cropping principle of local image features provided in an exemplary embodiment of this disclosure. The area within the dashed box represents the cropped sub-image features of the target object, serving as local image features of that target object. As can be seen from the target scene image, the height and width directions of the cropped local image features are consistent with the coordinate axis directions of the target object's current coordinate system.

[0086] This embodiment determines local image features in the current coordinate system centered on the target object for trajectory prediction, which can better capture the relevant features of the target object and further improve the accuracy of the predicted trajectory.

[0087] In an optional embodiment, at least one time frame further includes at least one historical time frame; step 204, based on the perception results corresponding to each time frame, determines the target features corresponding to each target object in the current time frame, including:

[0088] Step 2041a: For any target object, determine the position sequence corresponding to the target object based on the state information of the target object in each time frame.

[0089] The position sequence can include the position of the target object in the current coordinate system, which is the state information of each time frame.

[0090] For example, at least one time frame includes four time frames. The current coordinate system is established based on the state of the target object in the current time frame. The position of the target object in each historical time frame is transformed to the current coordinate system to obtain three positions in the current coordinate system. These positions are then combined with the origin of the current coordinate system to form a position sequence including four positions.

[0091] Step 2042a: Using the temporal feature extraction network in the pre-trained trajectory prediction model, the position sequences corresponding to each target object are processed to obtain the temporal features corresponding to each target object.

[0092] The temporal feature extraction network can be any feature extraction network capable of extracting temporal features, such as a temporal convolutional network (TCN). A temporal convolutional network performs one-dimensional convolution on the temporal dimension of the target object's position sequence, encoding the target object's trajectory information to obtain its temporal representation, which serves as the target object's temporal feature.

[0093] This embodiment processes the position sequence of each target object through a temporal feature extraction network to obtain the temporal features of each target object, which is used to assist in the trajectory prediction of the target object and can further improve the accuracy of the prediction results.

[0094] In an optional embodiment, when determining the temporal features of the target object, a second time interval smaller than the first time interval of the aforementioned time frame can be used to sample the perception results, thereby obtaining more granular temporal features to assist in trajectory prediction of the target object and further improve the accuracy of the prediction results. Specifically, this can be set during model training, and the time interval is consistent with the model training process during application.

[0095] For example, the first time interval is 0.5 seconds, the second time interval can be 0.25 seconds, at least one time frame for generating the first feature can include 4 time frames (2 seconds), and the perception result for generating the temporal feature can include 8 frames (2 seconds). The specific time interval can be set according to actual needs.

[0096] In an optional embodiment, step 204, which determines the target features corresponding to each target object in the current time frame based on the perception results corresponding to each time frame, includes:

[0097] Step 2041b: For any time frame, based on the state information of each target object in that time frame, determine the relative relationship information of each target object in that time frame. The relative relationship information of any target object includes the relative relationship between the target object and other target objects in that time frame.

[0098] The relative relationship between the target object and other target objects can refer to at least one of the following relationships: relative position, relative orientation, relative velocity, relative acceleration, etc.

[0099] In an optional embodiment, for any target object, after converting the state information of the target object in each historical time frame to the current coordinate system of the target object, for any time frame, the relative relationship information of the target object in that time frame can be represented as at least one of the information of other target objects around the target object in the current coordinate system of the target object, such as position, orientation, speed, size, type, etc.

[0100] Step 2042b: For any target object, determine the relative relationship sequence corresponding to the target object based on the relative relationship information corresponding to the target object in each time frame.

[0101] The relative relationship series includes the relative relationship information of the target object in each time frame.

[0102] For example, the target scene image includes 12 target objects and 4 time frames. For each target object, the relative relationship of the target object includes the relative relationship of the target object with itself and the relative relationship of the target object with 11 other target objects. The relative relationship with each other target object includes the features of the other target object in the target object's current coordinate system in five dimensions: horizontal coordinate, vertical coordinate, velocity, acceleration, and orientation. The relative relationship sequence of the target object can be determined based on a 1*12*5*4 feature map with a time dimension of 4. The feature map is represented as a sequence format as the relative relationship sequence of the target object.

[0103] Step 2043b: Using the attention interaction network in the pre-trained trajectory prediction model, process the relative relationship sequences corresponding to each target object to obtain the interaction features corresponding to each target object.

[0104] The attention interaction network can be implemented using a neural network based on an attention mechanism. By utilizing information such as the position, velocity, and orientation of each target object, the dynamic correlation between any target object and other target objects is explicitly encoded, generating the interaction features between each target object and other surrounding target objects.

[0105] This embodiment obtains the interaction features corresponding to each target object through an attention interaction network to assist in the trajectory prediction of each target object, which can effectively improve the trajectory prediction accuracy of obstacles in complex road scenes and scenes with a high degree of obstacle interaction.

[0106] In an optional embodiment, when determining the interaction features corresponding to the target object, a third time interval smaller than the first time interval of the aforementioned time frame can be used to sample the perception results. The third time interval can be the same as or different from the second time interval of the temporal features, thereby further improving the accuracy of obstacle trajectory prediction in complex road scenes and scenes with a high degree of obstacle interaction. Specifically, it can be set during model training, and the time interval is consistent with the model training process during application.

[0107] Figure 7 This is a flowchart illustrating a method for determining the trajectory of a target object provided in another exemplary embodiment of this disclosure.

[0108] In an optional embodiment, step 204, which determines the target features corresponding to each target object in the current time frame based on the perception results corresponding to each time frame, includes:

[0109] Step 2041c: Based on the perception results corresponding to each time frame and the perception results corresponding to at least one intermediate frame between time frames, determine the target features corresponding to each target object.

[0110] The time interval between intermediate frames and adjacent time frames can be set according to actual needs. For example, the first time interval of time frames can be 0.5 seconds. At least one intermediate frame's perception result is then interpolated between adjacent time frames, making the second time interval for determining target features 0.25 seconds or less. Multiple intermediate frames can be interpolated between adjacent time frames to obtain perception results corresponding to multiple new time frames with smaller time intervals, used to determine the temporal and / or interaction features of the target object. The principle for determining temporal and interaction features in the new time frames is similar to that of the time frames mentioned above and will not be elaborated further here.

[0111] This embodiment uses higher frame rate sampling of perception results to determine the temporal features and / or interaction features of the target object, and then uses the temporal features and / or interaction features to assist in the prediction of the future trajectory of the target object, thereby further improving the accuracy of trajectory prediction.

[0112] In an optional embodiment, step 205, which determines the driving trajectory of each target object in a later time frame based on the first feature and target feature corresponding to each target object, includes:

[0113] Step 2051: The first feature, temporal feature and interaction feature corresponding to each target object are fused to obtain the fused feature.

[0114] The fusion method of the first feature, temporal feature and interaction feature can be set according to actual needs. For example, the three can be spliced ​​together on preset dimensions.

[0115] Step 2052: Use the decoding network in the pre-trained trajectory prediction model to process the fused features and obtain the decoding result.

[0116] The decoding network can be any implementable network, such as a Transformer-based decoding network, a convolutional neural network-based decoding network, etc. No specific limitations are imposed.

[0117] Step 2053: The decoding results are processed using the prediction head network in the trajectory prediction model to obtain the trajectory prediction results corresponding to each target object.

[0118] The prediction head network can include two head networks. One head network predicts the probability that the future trajectory of each target object belongs to each trajectory type, and the other head network regresses the Gaussian distribution parameters or offset values ​​corresponding to the anchor trajectories of the k (k is a positive integer, which can be set according to actual needs) trajectory types with the highest predicted probabilities. In other words, the trajectory prediction result includes the predicted probability that the future trajectory of each target object belongs to each trajectory type, and the Gaussian distribution parameters or offset values ​​corresponding to the k trajectory types with the highest predicted probabilities.

[0119] In an optional embodiment, the Gaussian distribution for trajectory prediction is a two-dimensional Gaussian distribution, and the Gaussian distribution parameters include the mean (μ) of the corresponding anchored trajectory. x ,μ y ), the log value of the standard deviation (logσ) x ,logσ y The correlation coefficient ρ between σ and the covariance matrix. Where, σ x and σ y The standard deviation is represented by x and y, which represent the x-axis and y-axis of the target object's current coordinate system. The mean can be used as the offset value. An anchored trajectory refers to the vehicle's likely trajectory over a future period (e.g., 6 seconds) based on its speed, direction (straight), and turning. Each anchored trajectory can include a preset number (e.g., 12) of trajectory points, and each trajectory corresponds to a trajectory type. Trajectory types can be represented in any feasible way; for example, different numbers can be used to represent different anchored trajectories. Each trajectory point in each anchored trajectory can be a coordinate point in a coordinate system with the target object of the predicted trajectory as the origin (i.e., the aforementioned current coordinate system), representing a coordinate point that the target object might pass through in the future.

[0120] For example, Figure 8 This is a schematic diagram of an anchoring trajectory provided by an exemplary example of this disclosure. This example demonstrates two anchoring trajectories: a straight line and a left turn. In the diagram, the horizontal axis represents the y-axis coordinate of the target object in its current coordinate system, and the vertical axis represents the x-axis coordinate.

[0121] Step 2054: Based on the trajectory prediction results corresponding to each target object, determine the driving trajectory of each target object in the later time frame.

[0122] For any target object, after obtaining the trajectory prediction result corresponding to the target object, the future driving trajectory of the target object can be determined based on the trajectory prediction result. The driving trajectory in the later time frame can include at least one trajectory, and each trajectory can include a set of trajectory points. The number of trajectory points in each set can be set according to actual needs. For example, a trajectory can include 12 trajectory points, that is, 12 future time frames. Taking the first time interval of the time frame as 0.5 seconds as an example, the driving trajectory of the target object in the next 6 seconds can be predicted.

[0123] In one optional embodiment, N driving trajectories with TopN prediction probabilities can be determined based on the trajectory prediction results. The trajectory points for each driving trajectory can be determined based on the trajectory points of the corresponding anchor trajectory and their offset values. For example, the sum of the coordinates and offset values ​​of each trajectory point of the anchor trajectory can be used as the trajectory points of the driving trajectory.

[0124] In one alternative embodiment, Figure 9 This is a schematic diagram of the network structure of a trajectory prediction model provided in an exemplary embodiment of this disclosure. The trajectory prediction model includes a first feature extraction network, a second feature extraction network, a temporal feature extraction network, an attention interaction network, a feature fusion network, a decoding network, and a prediction head network. Cropping is a process outside the model but can also be part of the model; its specific implementation is not limited. Specifically, the first feature extraction network extracts features from the target scene image to obtain first image features. These first image features are then cropped to obtain local image features corresponding to each target object. The second feature extraction network processes these local image features to obtain first features corresponding to each target object. The temporal feature extraction network processes the position sequences corresponding to each target object to obtain temporal features corresponding to each target object. The attention interaction network processes the relative relationship sequences corresponding to each target object to obtain interaction features corresponding to each target object. The first features, temporal features, and interaction features are fused through the feature fusion network to obtain fused features. The decoding network decodes the fused features to obtain a decoding result. The decoding result is then processed by the prediction head network to obtain the trajectory prediction result.

[0125] In an optional embodiment, the feature fusion network may be a Concat network layer, which concatenates the first feature, temporal feature and interaction feature in a preset dimension to obtain fused features.

[0126] For example, the first feature is a 12*128*1*1 feature, the temporal feature is a 12*8*1*1 feature, and the interaction feature is a 12*16*1*1 feature. These features are concatenated along the second dimension to obtain a fused feature of 12*152*1*1. Here, 12 represents the number of target objects, the last 1 represents the number of channels, and the 128*1, 8*1, and 16*1 represent the dimension of the corresponding feature for each target object. Taking 16*1 as an example, the interaction feature of each target object is represented by a 16*1 vector. The representation method of each feature can be set according to actual needs and is not limited to the above method.

[0127] In one alternative embodiment, the attention interaction network can be implemented based on a multi-head self-attention mechanism. For example, Figure 10 This is a schematic diagram of the structure of an attention interaction network provided in an exemplary embodiment of this disclosure. In this example, the attention interaction network includes three independent convolutional layers, a multi-head self-attention layer, and a concatenation layer. The relative relationship sequence of the target object is mapped to a query vector Q, a key vector K, and a value vector V through the three independent convolutional layers, respectively. Q, K, and V undergo self-attention operations in the multi-head self-attention layer to obtain the attention results corresponding to each head. In this example, the multi-head self-attention layer uses three heads as an example. The attention results corresponding to each head are concatenated to obtain the interaction features corresponding to the target object. Specifically, for each head, Q and K are similarity calculated in the form of a dot product to obtain a weight value. Based on the weight value, V is weighted and summed to obtain the context of single-head attention. The single-head attention contexts of multiple heads are concatenated to obtain the interaction features of the target object. This is only an exemplary structural diagram of an attention interaction network. In practical applications, other feasible network structures can also be used for implementation, and no specific limitation is made.

[0128] In an optional embodiment, the trajectory prediction result includes the probability value and offset value of the target object corresponding to each preset trajectory; step 2054, based on the trajectory prediction result corresponding to each target object, determines the driving trajectory of each target object in the subsequent time frame, including:

[0129] For any target object, based on the probability value corresponding to the target object in each preset trajectory, a first number of preset trajectories whose probability values ​​satisfy preset conditions are determined as target preset trajectories; for any target preset trajectory, based on the target preset trajectory and the offset value corresponding to the target preset trajectory, the target trajectory corresponding to the target object in the target preset trajectory is determined; the target trajectory corresponding to the target object in each target preset trajectory is taken as the driving trajectory of the target object in the later time frame.

[0130] The preset trajectory refers to the aforementioned anchored trajectory, which will not be elaborated upon here. Preset conditions can be set according to actual needs, such as sorting by probability value from largest to smallest, with the preset trajectories ranking in the Top N (N being the first number, which can be set according to actual needs) serving as the target preset trajectory. The sum of the target preset trajectory and the offset value becomes the target trajectory of the target object.

[0131] It should be noted that the trajectory prediction model disclosed herein needs to be pre-trained. During the training process, the processing of training sample data is consistent with the application process described above. Based on the comparison between the predicted results obtained during the training process and the label data, the network loss is determined and used to update the network parameters until the training termination condition is met, thus obtaining the trained trajectory prediction model. The label data can be determined based on the real trajectory of the target object in future time frames. Specifically, based on a large amount of collected perception result data, sampling is performed at certain time intervals to obtain the perception results corresponding to each time frame in at least one time frame. Any time frame in each time frame is taken as the current time frame, all time frames before the current time frame are taken as historical time frames, and all time frames after the current time frame are taken as future time frames. Training sample data is determined based on the current time frame and historical time frames, and label data corresponding to the training sample data is determined based on the future time frames. This label data is used to train the trajectory prediction network corresponding to the pre-established trajectory prediction model. The training sample data required for the temporal feature extraction network and the attention interaction network can be determined based on the perception results sampled at smaller time intervals (higher frame rates). The temporal feature extraction network and the attention interaction network can effectively capture more granular temporal features and interaction features between objects. Through supervised learning, the prediction accuracy of the model can be greatly improved, thus obtaining more accurate predicted trajectories in practical applications.

[0132] The trajectory determination method for a target object provided in this disclosure enhances the ability to capture the temporal information of obstacles by combining the first feature, temporal feature, and interaction feature corresponding to the target object to predict its future trajectory. This improves the accuracy of the prediction results and enhances the prediction effect in scenarios such as intersections, turns, and U-turns. Furthermore, it fully utilizes the spatial relationship between obstacles and integrates the influence of the interaction between obstacles on trajectory prediction, further improving the prediction performance in complex road scenarios. This provides more effective obstacle reference information for subsequent planning and control.

[0133] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0134] The trajectory determination method for any target object provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to terminal devices and servers. Alternatively, the trajectory determination method for any target object provided in this disclosure can be executed by a processor, such as by a processor executing the trajectory determination method for any target object mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0135] Exemplary device

[0136] Figure 11 This is a schematic diagram of a target object trajectory determination device provided in an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement corresponding method embodiments of the present disclosure, such as… Figure 11 The device shown includes: a first acquisition module 501, a first processing module 502, a second processing module 503, a third processing module 504, and a fourth processing module 505.

[0137] The first acquisition module 501 is used to acquire the perception results corresponding to each of the at least one time frame and the first area map corresponding to the current time frame. The at least one time frame includes the current time frame, and the perception results include the state information of at least one target object in the first coordinate system.

[0138] The first processing module 502 is used to generate a target scene image corresponding to the current time frame based on the perception results corresponding to each of the time frames and the first area map;

[0139] The second processing module 503 is used to determine, based on the target scene image, the first feature corresponding to each of the target objects in the current time frame;

[0140] The third processing module 504 is used to determine the target features corresponding to each target object in the current time frame based on the perception results corresponding to each of the time frames. The target features include the temporal features and / or the interaction features corresponding to the target object. The interaction features characterize the correlation between the target object and at least one other target object around it.

[0141] The fourth processing module 505 is used to determine the driving trajectory of each target object in a later time frame based on the first feature and the target feature corresponding to each target object.

[0142] Figure 12 This is a schematic diagram of the structure of a target object trajectory determination device provided in another exemplary embodiment of this disclosure.

[0143] In one optional example, the first processing module 502 includes:

[0144] The first processing unit 5021 is used to generate a raster image corresponding to any given time frame based on the perception result of that time frame.

[0145] The second processing unit 5022 is used to fuse the raster images corresponding to each time frame with the first area map to obtain the target scene image.

[0146] In an optional embodiment, the first processing unit 5021 is specifically used for:

[0147] Based on the state information of each target object in the perception result corresponding to the time frame, the position and orientation of each target object in the raster coordinate system corresponding to the raster image are determined; based on the position and orientation of each target object in the raster coordinate system, each target object is rendered in the raster coordinate system according to the preset rendering method to obtain the raster image.

[0148] In an optional embodiment, the second processing unit 5022 is specifically used for:

[0149] The raster images corresponding to each time frame are stitched together with the first region map by channel to obtain the target scene image.

[0150] In an optional embodiment, the second processing module 503 includes:

[0151] The third processing unit 5031 is used to process the target scene image using the first feature extraction network in the pre-trained trajectory prediction model to obtain the first image features.

[0152] The fourth processing unit 5032 is used to determine the local image features corresponding to each target object based on the first image features.

[0153] The fifth processing unit 5033 is used to extract features from the local image features corresponding to each target object using the second feature extraction network in the trajectory prediction model, and obtain the first feature corresponding to each target object.

[0154] In an optional embodiment, the fourth processing unit 5032 is specifically used for:

[0155] For any target object, based on the current state information of the target object in the current time frame, the transformation relationship between the current coordinate system of the target object and the coordinate system corresponding to the first image feature is determined. The current coordinate system takes the current position of the target object as the origin and the current orientation of the target object as the first coordinate axis. Based on the transformation relationship, the first image feature is transformed into the current coordinate system to obtain the target image feature corresponding to the first image feature. Based on the preset cropping rules, the sub-image feature corresponding to the target object is determined from the target image feature. The sub-image feature is used as the local image feature corresponding to the target object.

[0156] In an optional embodiment, at least one time frame further includes at least one historical time frame; the third processing module 504 includes:

[0157] The first determining unit 5041a is used to determine the position sequence corresponding to any target object based on the state information of the target object in each time frame.

[0158] The sixth processing unit 5042a is used to process the position sequences corresponding to each target object by utilizing the temporal feature extraction network in the pre-trained trajectory prediction model, and obtain the temporal features corresponding to each target object.

[0159] In an optional embodiment, the third processing module 504 includes:

[0160] The second determining unit 5041b is used to determine the relative relationship information of each target object in any given time frame based on the state information of each target object in that time frame. The relative relationship information of any target object includes the relative relationship between the target object and other target objects in that time frame.

[0161] The third determining unit 5042b is used to determine the relative relationship sequence corresponding to any target object based on the relative relationship information corresponding to the target object in each time frame.

[0162] The seventh processing unit 5043b uses the attention interaction network in the pre-trained trajectory prediction model to process the relative relationship sequences corresponding to each target object, and obtains the interaction features corresponding to each target object.

[0163] In an optional embodiment, the third processing module 504 is specifically used for:

[0164] Based on the perception results corresponding to each time frame and the perception results corresponding to at least one intermediate frame between time frames, the target features corresponding to each target object are determined.

[0165] In an optional embodiment, the fourth processing module 505 includes:

[0166] The feature fusion unit 5051 is used to fuse the first feature, temporal feature and interaction feature corresponding to each target object to obtain fused features.

[0167] The decoding unit 5052 is used to process the fused features using the decoding network in the pre-trained trajectory prediction model to obtain the decoding result.

[0168] The prediction unit 5053 is used to process the decoding results using the prediction head network in the trajectory prediction model to obtain the trajectory prediction results corresponding to each target object.

[0169] The fourth determining unit 5054 is used to determine the driving trajectory of each target object in the subsequent time frame based on the trajectory prediction results corresponding to each target object.

[0170] In an optional embodiment, the trajectory prediction result includes the probability value and offset value of the target object corresponding to each preset trajectory; the fourth determining unit 5054 is specifically used for:

[0171] For any target object, based on the probability value corresponding to the target object in each preset trajectory, a first number of preset trajectories whose probability values ​​satisfy preset conditions are determined as target preset trajectories; for any target preset trajectory, based on the target preset trajectory and the offset value corresponding to the target preset trajectory, the target trajectory corresponding to the target object in the target preset trajectory is determined; the target trajectory corresponding to the target object in each target preset trajectory is taken as the driving trajectory of the target object in the later time frame.

[0172] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0173] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0174] Exemplary electronic devices

[0175] Figure 13 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure, including at least one processor 11 and a memory 12.

[0176] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0177] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the methods of the various embodiments of this disclosure above and / or other desired functions.

[0178] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0179] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0180] The output device 14 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0181] Of course, for the sake of simplicity, Figure 13 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0182] Exemplary computer program products and computer-readable storage media

[0183] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the trajectory determination method for the target object described in the various embodiments of this disclosure in the "Exemplary Methods" section above.

[0184] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0185] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the trajectory determination method for the target object of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0186] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0187] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0188] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for determining the trajectory of a target object, comprising: Acquire the perception results corresponding to each of the at least one time frame and the first area map corresponding to the current time frame, wherein the at least one time frame includes the current time frame and the perception results include the state information of at least one target object in the first coordinate system; Based on the perception results corresponding to each of the time frames and the first area map, a target scene image corresponding to the current time frame is generated. The target scene image is an image that fuses the object information of the perception results corresponding to each of the time frames and the first area map. Based on the target scene image, determine the first feature corresponding to each target object in the current time frame; The first feature is obtained by feature extraction from the target scene image; Based on the perception results corresponding to each of the time frames, the target features corresponding to each of the target objects corresponding to the current time frame are determined. The target features include the temporal features corresponding to the target object and / or the interaction features corresponding to the target object. The interaction features characterize the correlation between the target object and at least one other target object around it. Based on the first feature and the target feature corresponding to each target object, the driving trajectory of each target object in the later time frame is determined.

2. The method according to claim 1, wherein, The step of determining the target features corresponding to each target object in the current time frame based on the perception results corresponding to each of the time frames includes: Based on the perception results corresponding to each of the time frames and the perception results corresponding to at least one intermediate frame between the time frames, the target features corresponding to each target object are determined.

3. The method according to claim 1, wherein, The at least one time frame further includes at least one historical time frame; the step of determining the target features corresponding to each target object in the current time frame based on the perception results corresponding to each of the time frames includes: For any of the target objects, the position sequence corresponding to the target object is determined based on the state information of the target object in each of the time frames; By utilizing the temporal feature extraction network in the pre-trained trajectory prediction model, the position sequences corresponding to each target object are processed to obtain the temporal features corresponding to each target object.

4. The method according to claim 1, wherein, The step of determining the target features corresponding to each target object in the current time frame based on the perception results corresponding to each of the time frames includes: For any given time frame, based on the state information of each target object in that time frame, the relative relationship information of each target object in that time frame is determined. The relative relationship information corresponding to any target object includes the relative relationship between the target object and other target objects in that time frame. For any of the target objects, a relative relationship sequence corresponding to the target object is determined based on the relative relationship information corresponding to the target object in each of the time frames; By utilizing the attention interaction network in the pre-trained trajectory prediction model, the relative relationship sequences corresponding to each target object are processed to obtain the interaction features corresponding to each target object.

5. The method according to claim 1, wherein, The step of generating a target scene image corresponding to the current time frame based on the perception results corresponding to each of the time frames and the first region map includes: For any given time frame, a raster image corresponding to that time frame is generated based on the perception result corresponding to that time frame; The raster image corresponding to each time frame is fused with the first region map to obtain the target scene image.

6. The method according to claim 5, wherein, The step of generating a raster image corresponding to the time frame based on the perception result corresponding to the time frame includes: Based on the state information of each target object in the perception result corresponding to the time frame, the position and orientation of each target object in the grid coordinate system corresponding to the grid image are determined. Based on the position and orientation of each target object in the grid coordinate system, each target object is rendered in the grid coordinate system according to a preset rendering method to obtain the grid image.

7. The method according to claim 5, wherein, The step of fusing the raster image corresponding to each of the time frames with the first region map to obtain the target scene image includes: The raster image corresponding to each time frame is stitched together with the first region map by channel to obtain the target scene image.

8. The method according to claim 1, wherein, The step of determining the first feature corresponding to each of the target objects in the current time frame based on the target scene image includes: The target scene image is processed using the first feature extraction network in the pre-trained trajectory prediction model to obtain the first image features; Based on the first image features, determine the local image features corresponding to each of the target objects; The second feature extraction network in the trajectory prediction model is used to extract features from the local image features corresponding to each of the target objects to obtain the first feature corresponding to each of the target objects.

9. The method according to claim 8, wherein, The step of determining the local image features corresponding to each of the target objects based on the first image features includes: For any of the target objects, based on the current state information of the target object in the current time frame, the transformation relationship between the current coordinate system of the target object and the coordinate system corresponding to the first image feature is determined, wherein the current coordinate system takes the current position of the target object as the origin and the current orientation of the target object as the first coordinate axis; Based on the transformation relationship, the first image feature is transformed into the current coordinate system to obtain the target image feature corresponding to the first image feature; Based on preset cropping rules, sub-image features corresponding to the target object are determined from the features of the target image; The sub-image features are used as the local image features corresponding to the target object.

10. The method according to claim 1, wherein, The step of determining the driving trajectory of each target object in a later time frame based on the first feature and the target feature corresponding to each target object includes: The first feature, the temporal feature, and the interaction feature corresponding to each of the target objects are fused to obtain a fused feature; The fused features are processed using the decoding network in the pre-trained trajectory prediction model to obtain the decoding result; The decoding results are processed using the prediction head network in the trajectory prediction model to obtain the trajectory prediction results corresponding to each of the target objects. Based on the trajectory prediction results corresponding to each of the target objects, the driving trajectory of each target object in the subsequent time frame is determined.

11. The method according to claim 10, wherein, The trajectory prediction result includes the probability value and offset value of the target object on each preset trajectory; The step of determining the driving trajectory of each target object in a later time frame based on the trajectory prediction results corresponding to each target object includes: For any of the target objects, based on the probability values ​​corresponding to the target object in each of the preset trajectories, a first number of preset trajectories whose probability values ​​satisfy preset conditions are determined as target preset trajectories; For any of the target preset trajectories, the target trajectory corresponding to the target object is determined based on the target preset trajectory and the offset value corresponding to the target preset trajectory; The target trajectory corresponding to each of the preset target trajectories is taken as the driving trajectory of the target object in the later time frame.

12. A trajectory determination device for a target object, comprising: The first acquisition module is used to acquire the perception results corresponding to each of the at least one time frame and the first area map corresponding to the current time frame. The at least one time frame includes the current time frame, and the perception results include the state information of at least one target object in the first coordinate system. The first processing module is used to generate a target scene image corresponding to the current time frame based on the perception results corresponding to each of the time frames and the first area map. The target scene image is an image that fuses the object information of the perception results corresponding to each of the time frames and the first area map. The second processing module is used to determine, based on the target scene image, a first feature corresponding to each of the target objects in the current time frame; the first feature is obtained by feature extraction from the target scene image. The third processing module is used to determine the target features corresponding to each target object in the current time frame based on the perception results corresponding to each of the time frames. The target features include the temporal features and / or the interaction features corresponding to the target object. The interaction features characterize the correlation between the target object and at least one other target object around it. The fourth processing module is used to determine the driving trajectory of each target object in a later time frame based on the first feature and the target feature corresponding to each target object.

13. A computer-readable storage medium storing a computer program for performing the trajectory determination method for a target object according to any one of claims 1-11.

14. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the trajectory determination method for the target object according to any one of claims 1-11.

Images