Training data augmentation method, apparatus, medium, and device for trajectory prediction

Abstract

This disclosure provides a method, apparatus, medium, and device for enhancing training data for trajectory prediction. The method includes: acquiring a set of visual images and a point cloud dataset collected within the same time period for a scene containing at least one dynamic obstacle; determining first trajectory information and second trajectory information corresponding to the dynamic obstacle based on the visual image set and the point cloud dataset; matching first historical trajectory information of each first trajectory information with second historical trajectory information of each second trajectory information to obtain a target matching result; for each pair of trajectory information, constructing first enhanced training data using first historical trajectory information of the first trajectory information as first historical trajectory sample information and second future trajectory information of the second trajectory information as first future trajectory annotation information; and determining an enhanced training dataset based on the first enhanced training data corresponding to each pair of trajectory information.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence technology, deep learning technology, and autonomous driving technology, and in particular to a method, apparatus, storage medium, and device for enhancing training data for trajectory prediction. Background Technology

[0002] In the field of autonomous driving, obstacle trajectory prediction is a crucial step. Trajectory prediction primarily targets dynamic obstacles within a driving scenario, predicting their future trajectories based on information such as the historical location of perceived obstacles. Accurate trajectory prediction ensures the safety of downstream control modules.

[0003] In mainstream trajectory prediction algorithms, deep learning-based trajectory prediction models are commonly used. The performance of these models heavily depends on the quality of the training data. For trajectory prediction models that run on real vehicles (i.e., actual vehicles) rather than through simple offline validation, to ensure consistency between offline and real-vehicle testing performance, the training data used during training is the trajectory data perceived visually during vehicle movement. However, in actual training, trajectory prediction models obtained using this method exhibit poor performance, with predicted trajectories showing significant jitter, lack of smoothness, and low accuracy. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide a method, apparatus, storage medium, and electronic device for enhancing training data for trajectory prediction.

[0005] According to one aspect of the present disclosure, a method for enhancing training data for trajectory prediction is provided, comprising: acquiring a set of visual images and a point cloud dataset collected within the same time period for a scene containing at least one dynamic obstacle; wherein the time period includes a midpoint, historical time before the midpoint, and future time after the midpoint; determining first trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle based on the visual image set, and determining second trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle based on the point cloud dataset; wherein the first trajectory information includes first historical trajectory information corresponding to the midpoint and the historical time, and first future trajectory information corresponding to the future time; and the second trajectory information... The method includes second historical trajectory information corresponding to the intermediate time and the historical time, and second future trajectory information corresponding to the future time; matching the first historical trajectory information of each first trajectory information with the second historical trajectory information of each second trajectory information to obtain a target matching result; wherein, the target matching result includes at least one pair of trajectory information that successfully matches the first trajectory information and the second trajectory information; for each pair of trajectory information, using the first historical trajectory information of the first trajectory information as the first historical trajectory sample information and the second future trajectory information of the second trajectory information as the first future trajectory annotation information, constructing first enhanced training data; and determining an enhanced training dataset based on the first enhanced training data corresponding to each pair of trajectory information.

[0006] According to one aspect of the present disclosure, a method for training a trajectory prediction model is provided, comprising: acquiring an augmented training dataset, the augmented training dataset being obtained by the method described in the present disclosure; obtaining at least one first predicted trajectory information based on at least one first historical trajectory sample information in the augmented training dataset and using an initial trajectory prediction model to be trained; determining a first loss value based on the at least one first predicted trajectory information and first future trajectory annotation information in the augmented training dataset corresponding to each of the at least one historical trajectory information; training the initial trajectory prediction model to be trained based on the first loss value; and determining the trained initial trajectory prediction model as a first trajectory prediction model in response to the trained initial trajectory prediction model satisfying a preset training completion condition.

[0007] According to one aspect of the present disclosure, an augmentation apparatus for training data for trajectory prediction is provided, comprising: a first acquisition module, configured to acquire a set of visual images and a point cloud dataset collected within the same time period for a scene containing at least one dynamic obstacle; wherein the time period includes a midpoint, historical time before the midpoint, and future time after the midpoint; and a first trajectory information determination module, configured to determine first trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle based on the visual image set, and to determine second trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle based on the point cloud dataset; wherein the first trajectory information includes first historical trajectory information corresponding to the midpoint and the historical time, and first future trajectory information corresponding to the future time; and the second trajectory information includes first historical trajectory information corresponding to the midpoint and the historical time, and first future trajectory information corresponding to the future time; and the second trajectory information includes first historical trajectory information corresponding to the midpoint and the historical time, and second future trajectory information corresponding to the future time. The system includes: second historical trajectory information corresponding to the historical time, and second future trajectory information corresponding to the future time; a first trajectory information matching module, configured to match the first historical trajectory information of each first trajectory information with the second historical trajectory information of each second trajectory information to obtain a target matching result; wherein, the target matching result includes: at least one pair of trajectory information that successfully matches the first trajectory information and the second trajectory information; a first enhanced data construction module, configured to construct first enhanced training data for each pair of trajectory information, using the first historical trajectory information of the first trajectory information as first historical trajectory sample information and the second future trajectory information of the second trajectory information as first future trajectory annotation information; and an enhanced training dataset determination module, configured to determine an enhanced training dataset based on the first enhanced training data corresponding to each pair of trajectory information. According to one aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program configured to execute the method described in any of the above embodiments.

[0008] According to one aspect of the present disclosure, a training apparatus for a trajectory prediction model is provided, comprising: a second acquisition module for acquiring an augmented training dataset, the augmented training dataset being obtained by the method described in the embodiments of the present disclosure; a first trajectory information prediction module for obtaining at least one first predicted trajectory information based on at least one first historical trajectory sample information in the augmented training dataset and using an initial trajectory prediction model to be trained; a first loss determination module for determining a first loss value based on the at least one first predicted trajectory information and first future trajectory annotation information in the augmented training dataset corresponding to each of the at least one historical trajectory information; a first training module for training the initial trajectory prediction model to be trained based on the first loss value; and a first model determination module for determining the trained initial trajectory prediction model as a first trajectory prediction model in response to the trained initial trajectory prediction model satisfying a preset training completion condition.

[0009] According to one aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method described above.

[0010] Based on the above embodiments of this disclosure, the method, apparatus, storage medium, and electronic device for enhancing training data for trajectory prediction involve acquiring a set of visual images and a point cloud dataset collected within the same time period for a scene containing at least one dynamic obstacle. Then, based on the visual image set and the point cloud dataset, first trajectory information and second trajectory information corresponding to each dynamic obstacle are determined. The second trajectory information includes second historical trajectory information corresponding to intermediate and historical times, and second future trajectory information corresponding to future times. Next, the first historical trajectory information of each first trajectory information is matched with the second historical trajectory information of each second trajectory information to obtain a target matching result. For each pair of at least one pair of trajectory information, first enhanced training data is constructed using the first historical trajectory information of the first trajectory information as first historical trajectory sample information and the second future trajectory information of the second trajectory information as first future trajectory annotation information. Based on the first enhanced training data corresponding to each pair of at least one pair of trajectory information, an enhanced training dataset is determined. Therefore, the embodiments of this disclosure utilize the second trajectory information obtained from the point cloud dataset to enhance the depth information in the first trajectory information obtained from the visual image set, thereby obtaining an enhanced training dataset. This makes the trajectory prediction model trained on the enhanced training dataset predict the trajectory data of dynamic obstacles more smoothly and more accurately predicts the trajectory of dynamic obstacles.

[0011] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0012] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0013] Figure 1 This is an exemplary application scenario of the training data augmentation method for trajectory prediction provided in this disclosure;

[0014] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present disclosure of a method for enhancing training data for trajectory prediction.

[0015] Figure 3 This is a flowchart illustrating a method for enhancing training data for trajectory prediction, provided in another exemplary embodiment of this disclosure.

[0016] Figure 4 This is a flowchart illustrating step 230 provided in an exemplary embodiment of this disclosure;

[0017] Figure 5 This is a flowchart illustrating step 231 provided in an exemplary embodiment of this disclosure;

[0018] Figure 6 This is a flowchart illustrating a method for enhancing training data for trajectory prediction, provided in yet another exemplary embodiment of this disclosure.

[0019] Figure 7 This is a flowchart illustrating a method for enhancing training data for trajectory prediction, provided in another exemplary embodiment of this disclosure.

[0020] Figure 8 This is a flowchart illustrating a method for enhancing training data for trajectory prediction, provided in another exemplary embodiment of this disclosure.

[0021] Figure 9 This is a flowchart illustrating a method for enhancing training data for trajectory prediction, provided in another exemplary embodiment of this disclosure.

[0022] Figure 10 This is a flowchart illustrating a training method for a trajectory prediction model provided in an exemplary embodiment of this disclosure.

[0023] Figure 11This is a flowchart illustrating a training method for a trajectory prediction model provided in another exemplary embodiment of this disclosure;

[0024] Figure 12 This is a schematic diagram of the structure of an enhancement device for training data for trajectory prediction provided in an exemplary embodiment of the present disclosure;

[0025] Figure 13 This is a schematic diagram of the structure of an enhancement device for training data for trajectory prediction provided in another exemplary embodiment of this disclosure;

[0026] Figure 14 This is a schematic diagram of the structure of a training device for a trajectory prediction model provided in an exemplary embodiment of this disclosure;

[0027] Figure 15 This is a schematic diagram of the structure of a training device for a trajectory prediction model provided in another exemplary embodiment of this disclosure;

[0028] Figure 16 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0029] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0030] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0031] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0032] It should also be understood that in the embodiments disclosed herein, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0033] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0034] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0035] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0036] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0037] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0038] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0039] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0040] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0041] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, standard programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0042] Application Overview

[0043] In the process of realizing this disclosure, the inventors discovered that when training a trajectory prediction model using trajectory data generated from visual images perceived by a visual sensor, the lack of depth information in the visual images and the difficulty in distinguishing nearby obstacles when they are occluded lead to the omission of some obstacle trajectory data. At the same time, due to the large error in the position of obstacles in the visual images along the depth direction and the inaccurate perception angle, the trajectory data obtained through the visual images has large jitter, and the lack of frames results in an uneven trajectory, affecting the training effect of the trajectory prediction model. Consequently, when the trajectory prediction model trained using visually perceived trajectory data is used for trajectory prediction, the output prediction trajectory has poor accuracy.

[0044] Exemplary System

[0045] The technical solution disclosed herein can be applied to autonomous driving, specifically to training trajectory prediction models used in autonomous driving. For example, during the training process of the trajectory prediction model, the training data can be augmented to make the predicted trajectory information obtained by the trajectory prediction model trained with the augmented training data more accurate, thereby better assisting the vehicle in route planning, determining the driving direction of the vehicle or other vehicles, etc.

[0046] Figure 1 This is an exemplary application scenario of the training data augmentation method for trajectory prediction provided in this disclosure. For example... Figure 1As shown, the system includes a visual sensor, a lidar sensor, and a data processing device. The visual sensor can be a monocular camera, a binocular camera, etc., used to acquire a set of visual images. The lidar sensor is used to acquire a point cloud dataset. The data processing device can be a data processor, a server, etc., used to process the input set of visual images and the point cloud dataset to obtain at least one first trajectory information corresponding to the set of visual images and at least one second trajectory information corresponding to the point cloud dataset. Then, the historical trajectory information in the at least one first trajectory information and the historical trajectory information in the at least one second trajectory information are matched to obtain a matching result. The matching result includes at least one pair of trajectory information that matches the historical trajectory information. For each pair of trajectory information, the historical trajectory information in the first trajectory information is used as the first historical trajectory information of the training data, and the future trajectory information in the second trajectory information is used as the annotation information of the first future trajectory of the training data to construct first enhanced training data. Based on the first enhanced training data corresponding to each pair of trajectory information, an enhanced training dataset is obtained. The visual image set and the point cloud dataset are obtained within the same time period for the same scene containing at least one dynamic obstacle.

[0047] When training the trajectory prediction model, the first historical trajectory information in the augmented training dataset is sequentially input into the trajectory prediction model to be trained, and the trajectory prediction model to be trained outputs the first predicted trajectory information corresponding to each first historical trajectory information. Based on each first predicted trajectory information and the first future trajectory annotation information corresponding to each historical trajectory information in the augmented training dataset, a first loss value is determined. Based on the first loss value, the trajectory prediction model to be trained is trained until the prediction model to be trained meets the preset training completion conditions.

[0048] The trained trajectory prediction model can be set in the vehicle's computing platform to predict the trajectory data of obstacles for autonomous vehicles.

[0049] The method disclosed herein uses the second trajectory information obtained from the point cloud dataset to enhance the depth information in the first trajectory information obtained from the visual image set, thereby obtaining an enhanced training dataset. This makes the trajectory prediction model trained on the enhanced training dataset predict the trajectory data of dynamic obstacles more smoothly and the trajectory prediction of dynamic obstacles more accurate.

[0050] Exemplary methods

[0051] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of the present disclosure of a method for enhancing training data for trajectory prediction. This embodiment can be applied to electronic devices, vehicles, etc. Figure 2 As shown, it includes the following steps:

[0052] Step S210: Obtain a set of visual images and a point cloud dataset collected within the same time period for a scene containing at least one dynamic obstacle.

[0053] The aforementioned time periods include: the midpoint, historical time before the midpoint, and future time after the midpoint.

[0054] This time period includes multiple moments. The middle moment can be selected from the time period according to actual needs. Within this time period, all moments before the middle moment are defined as historical time, and all moments after the middle moment are defined as future time. Both historical time and future time include at least one moment.

[0055] The scenarios containing at least one dynamic obstacle can be selected as needed. For example, scenarios including highways, intersections, or T-junctions can be selected. The dynamic obstacle is a moving object, such as, but not limited to, vehicles, drones, pedestrians, and animals.

[0056] A visual image set may include multiple visual images, each containing at least one dynamic obstacle, and each visual image corresponds to a time when it is perceived. A point cloud dataset includes multiple point cloud datasets, each containing at least one dynamic obstacle, and each point cloud dataset corresponds to a time when it is perceived.

[0057] In one implementation, for the same scene within the same time period, visual image sets and point cloud datasets are obtained separately by visual sensors and LiDAR sensors. Any visual image in the visual image set can correspond to the point cloud data at the same time it was perceived.

[0058] Step S220: Based on the visual image set, determine the first trajectory information corresponding to each dynamic obstacle in at least one dynamic obstacle, and based on the point cloud dataset, determine the second trajectory information corresponding to each dynamic obstacle in at least one dynamic obstacle.

[0059] Each first trajectory information includes first historical trajectory information corresponding to the intermediate time and historical time, and first future trajectory information corresponding to the future time; each second trajectory information includes second historical trajectory information corresponding to the intermediate time and historical time, and second future trajectory information corresponding to the future time.

[0060] Both the first and second trajectory information can include: motion data of the dynamic obstacle at each moment within the time period. The motion data of the dynamic obstacle includes: state information, depth information, category information, and size information, etc. The state information includes: position information and orientation information of the dynamic obstacle, etc. Depth information is used to characterize the distance between the dynamic obstacle and the sensor that detects it. Category information indicates the type of dynamic obstacle; for example, category information could be: car, bicycle, pedestrian, etc. Size information indicates the size of the dynamic obstacle; for example, size information can include the length, width, and height of the dynamic obstacle.

[0061] In one implementation, annotation software, such as darklabel or VIBAT, can be used to annotate the trajectory information of the visual image set and the point cloud dataset, to obtain at least one first trajectory information for each dynamic obstacle in the visual image set and at least one second trajectory information for each dynamic obstacle in the point cloud dataset.

[0062] In one implementation, since the time when the visual image is perceived and the time when the point cloud data is perceived may differ, the first trajectory information and the second trajectory information include motion data of dynamic obstacles corresponding to different times. When the first trajectory information and the second trajectory information include motion data of dynamic obstacles corresponding to different times, interpolation can be used to make the first trajectory information and the second trajectory information include motion data of dynamic obstacles corresponding to the same time.

[0063] In one implementation, each moment corresponding to any first trajectory information, or each moment corresponding to any second trajectory information, is used as a time template. That is, the time template includes multiple moments, all of which fall within a time period. Based on the time template, interpolation is used to perform time and data synchronization processing on each first trajectory information or each second trajectory information, so that the moments corresponding to the motion data of the dynamic obstacles included in each first trajectory information correspond to the moments corresponding to the motion data of the dynamic obstacles included in each second trajectory information. For example, the moments corresponding to the first trajectory information are used as time templates. Based on the time templates, the moments corresponding to the second trajectory information are adjusted so that each moment corresponding to the second trajectory information corresponds one-to-one with each moment in the time template. Then, for each adjusted second trajectory information, the newly added moments in the adjusted second trajectory information are determined. For each newly added moment, the state and depth information corresponding to the moments before the newly added moment and the state and depth information corresponding to the moments after the newly added moment can be obtained by interpolation. The category and size information corresponding to the moment with the smallest time difference from the newly added moment can be determined as the category and size information corresponding to the newly added moment, thereby obtaining the motion data of the dynamic obstacle at each newly added moment.

[0064] In a specific example, the time period T includes the times t1, t2, t3, t4, t5, and t6 arranged in chronological order, with a time interval of 1 second between adjacent times. The times corresponding to the first trajectory information are t1, t2, t3, t5, and t6, respectively; the times corresponding to the second trajectory information are t1, t3, t4, and t6, respectively. Using the moments corresponding to the first trajectory information as time templates, the moments of the second trajectory information are adjusted to match the time templates. The adjusted moments of the second trajectory information are: t1, t2, t3, t5, t6. The newly added moments t2 and t5 in the second trajectory information do not correspond to the motion data of dynamic obstacles. Based on the state and depth information corresponding to moments t1 and t3 in the second trajectory information, the state and depth information corresponding to the newly added moment t2 in the second trajectory information is obtained through interpolation. Since the time difference between moments t1 and t3 and the newly added moment t2 is 1 second, the category and size information corresponding to t1 or t3 can be determined as the category and size information corresponding to the newly added moment t2. Similarly, the motion data of the dynamic obstacle corresponding to the newly added moment t5 in the second trajectory information is determined.

[0065] Step S230: Match the first historical trajectory information of each first trajectory information with the second historical trajectory information of each second trajectory information to obtain the target matching result.

[0066] The target matching result includes at least one pair of trajectory information that successfully matches the first trajectory information and the second trajectory information.

[0067] In one implementation, a pair of trajectory information that successfully matches historical trajectory information can be defined as a pair of trajectory information that successfully matches the first trajectory information and the second trajectory information.

[0068] In one implementation, historical trajectory information can be used to obtain the trajectory curvature, trajectory point distribution, etc., corresponding to the historical trajectory information. Based on the trajectory curvature, trajectory point distribution, etc., each first historical trajectory information and each second historical trajectory information are matched to obtain the target matching result. Alternatively, the matching of category information, size information, etc., in the historical trajectory information can be used to determine whether the historical trajectory information of each first trajectory information and the historical trajectory information of each second trajectory information match, thus obtaining the target matching result.

[0069] Step S240: For each pair of trajectory information in at least one pair of trajectory information, the first historical trajectory information of the first trajectory information is used as the first historical trajectory sample information, and the second future trajectory information of the second trajectory information is used as the first future trajectory annotation information to construct the first enhanced training data.

[0070] Each pair of trajectory information can construct a first augmented training data. Each augmented training data includes: first historical trajectory sample information and first future trajectory annotation information. That is, each first augmented training data includes: motion data of dynamic obstacles corresponding to each moment in the time period.

[0071] Step S250: Determine the augmented training dataset based on the first augmented training data corresponding to each pair of trajectory information in at least one pair of trajectory information.

[0072] This augmented training dataset is used to train the trajectory prediction model. The augmented training dataset may include multiple sets of initial augmented training data.

[0073] In this embodiment of the disclosure, the depth information in the first trajectory information obtained from the visual image set is enhanced by using the second trajectory information obtained from the point cloud dataset to obtain the enhanced training dataset. This makes the trajectory prediction model trained with the enhanced training dataset predict the trajectory data of dynamic obstacles more smoothly and the trajectory prediction of dynamic obstacles more accurate.

[0074] In an optional example, the target matching result of this disclosure embodiment also includes at least one first trajectory information that failed to match. For example... Figure 3 As shown, the method for augmenting training data for trajectory prediction according to embodiments of this disclosure may further include the following steps:

[0075] Step S310: Determine at least one first trajectory information that failed to match as trajectory information to be set.

[0076] Among them, the first trajectory information that failed to match the historical trajectory information can be identified as the first trajectory information that failed to match.

[0077] Step S320: For each trajectory information to be set in at least one trajectory information to be set, the historical trajectory information of the trajectory information to be set is used as the second historical trajectory sample information, and the future trajectory information of the trajectory information to be set is used as the second future trajectory annotation information to construct the second enhanced training data.

[0078] Each piece of second enhanced training data includes: second historical trajectory sample information and second future trajectory annotation information. That is, each piece of second enhanced training data includes the motion data of the dynamic obstacles at each moment in the time period corresponding to the trajectory information to be set that forms the second enhanced training data.

[0079] Step S250 of this embodiment may further include: determining an augmented training dataset based on at least one first augmented training data and at least one second augmented training data.

[0080] The augmented training dataset can be constructed using multiple sets of first-level augmented training data and multiple sets of second-level augmented training data. That is, the augmented training dataset can include multiple sets of first-level augmented training data and multiple sets of second-level augmented training data. In one implementation, the number of first-level augmented training data sets in the augmented training dataset is greater than the number of second-level augmented training data sets.

[0081] In this embodiment of the disclosure, by determining the first trajectory information that failed to match as the second enhanced training data and constructing an enhanced training dataset together with the first enhanced training data, not only is the application of the data maximized, but also, since the enhanced training dataset includes both the first and second enhanced training data, the accuracy of the trajectory information predicted by the trajectory prediction model trained using the enhanced training dataset through visual images is improved.

[0082] In an optional example, the first historical trajectory information of this disclosure embodiment includes first sub-historical trajectory information corresponding to the intermediate time, and the second historical trajectory information includes second sub-historical trajectory information corresponding to the intermediate time. For example... Figure 4 As shown, step S230 of this embodiment may further include the following steps:

[0083] Step S231: Match the first sub-historical trajectory information of each first historical trajectory information and the second sub-historical trajectory information of each second historical trajectory information using at least one matching method to determine the first matching result.

[0084] The first matching result includes at least one pair of trajectory information that is paired with the first trajectory information and the second trajectory information.

[0085] The first sub-historical trajectory information includes the motion data of dynamic obstacles at the corresponding intermediate time point in the first trajectory information; the second sub-historical trajectory information includes the motion data of dynamic obstacles at the corresponding intermediate time point in the second trajectory information. Each first sub-historical trajectory information can be matched with each second sub-historical trajectory information using at least one matching method to obtain a first matching result. This matching method can be to use the matching result where the matching degree between the first and second sub-historical trajectories in each pair of paired trajectory information is greater than a preset matching degree as the first matching result. The preset matching degree can be set according to actual needs.

[0086] Step S232: Based on the first matching result, determine the target matching result.

[0087] Among them, the Hungarian Algorithm or the Kuhn-Munkres Algorithm can be used to process the first sub-historical trajectory information and the second sub-historical trajectory information in the first matching result to obtain the target matching result.

[0088] For example, the first trajectory information can be Correspondingly, the first sub-historical trajectory information can be The second trajectory information can be Correspondingly, the second sub-historical trajectory information can be The first matching result can be represented as the first matching degree matrix from the first sub-historical trajectory information to the second sub-historical trajectory information. For example, the first matching degree matrix The form can be as shown in Equation (1), where each element in the first matching degree matrix shown in Equation (1) is the matching degree between the first sub-historical trajectory information and the second sub-historical trajectory information.

[0089] (1)

[0090] Using the Hungarian algorithm, based on the first matching degree matrix The system processes each first sub-historical trajectory information and each second sub-historical trajectory information to obtain at least one pair of trajectory information that successfully matches the first sub-historical trajectory information and the second sub-historical trajectory information. When the first sub-historical trajectory information and the second sub-historical trajectory information successfully match, it can be determined that the first trajectory information corresponding to the first sub-historical trajectory information and the second trajectory information corresponding to the second sub-historical trajectory information have successfully matched. Based on at least one pair of trajectory information that successfully matches the first sub-historical trajectory information and the second sub-historical trajectory information, the target matching result can be obtained.

[0091] The target matching results can be further optimized. For example, the matching can be optimized based on the Hungarian algorithm and the first matching degree matrix. The obtained target matching result is called the first allocation result. From the first allocation result, we can see that each first sub-historical trajectory information corresponds to only one second sub-historical trajectory information. However, it cannot be guaranteed that each second sub-historical trajectory information corresponds to one first sub-historical trajectory information. Therefore, a second matching degree matrix from the second sub-historical trajectory information to the first sub-historical trajectory information can be established based on the first allocation result. For example, the second matching degree matrix The form can be as shown in equation (2), where each element in the second matching degree matrix shown in equation (2) is the matching degree between the second sub-historical trajectory information and the first sub-historical trajectory information.

[0092] (2)

[0093] Based on the Hungarian algorithm and the second matching degree matrix An optimized target matching result is obtained, in which each first trajectory information and each second trajectory information uniquely correspond. In this embodiment of the disclosure, at least one matching method is used to match the first sub-historical trajectory information and the second sub-historical trajectory information to obtain a first matching result. Based on the first matching information, the target matching result is determined, which effectively improves the accuracy of the target matching result, thereby improving the reliability of the first enhanced training data, and making the trajectory predicted by the trajectory prediction model trained on the enhanced training dataset more accurate.

[0094] In one optional example, at least one matching method in this disclosure embodiment includes: polar coordinate distance matching method. For example... Figure 5 As shown, step S231 of this embodiment may further include the following steps:

[0095] Step S2311: If the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is determined to be the preset type information, the size information and depth information of the dynamic obstacle corresponding to each first sub-historical trajectory information are matched with the size information and depth information of the dynamic obstacle corresponding to each second sub-historical trajectory information by means of polar coordinate distance matching, so as to obtain the first polar coordinate distance matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information.

[0096] The type information is used to characterize the type of a dynamic obstacle. This type information can be set according to actual needs; for example, it can be categorized as a large object, small object, dynamic object, or static object. Preset type information can also be set according to actual needs. Specifically, the type information of a dynamic obstacle can be determined based on its category or size information. For example, a dynamic obstacle categorized as a vehicle or drone can be classified as a large object.

[0097] In one embodiment, depth information of dynamic obstacles can be acquired using devices such as LiDAR or binocular cameras. Alternatively, depth information of dynamic obstacles can be obtained using a neural network for detecting depth information, such as CNN (Convolutional Neural Networks), SCNN (Spatial Convolutional Neural Networks), or RNN (Recurrent Neural Networks). The size information of dynamic obstacles can also be obtained using a neural network for detecting their size position information, such as CNN, SCNN, or RNN. In one embodiment, the polar coordinate distance matching method can be as follows: the coordinate system containing the position information of the dynamic obstacle in each first sub-historical trajectory information, and the coordinate system containing the position information of the dynamic obstacle in each second sub-historical trajectory information, are both converted to polar coordinates; the position information corresponding to the position information of the dynamic obstacle in each first sub-historical trajectory information in the polar coordinate system is determined as the first target position information, and the position information corresponding to the position information of the dynamic obstacle in each second sub-historical trajectory information in the polar coordinate system is determined as the second target position information. The polar coordinate system can be divided into multiple sector regions based on a preset center angle. Based on the first target position information and the second target position information, the sector region where each first sub-historical trajectory information and each second sub-historical trajectory information is located is determined. When the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is a preset type, the size and depth information of the dynamic obstacle in the first sub-historical trajectory information located in the same sector region are matched with the size and depth information of the dynamic obstacle in the second sub-historical trajectory information to obtain the first polar coordinate distance matching degree between the first and second sub-historical trajectory information. The first polar coordinate distance matching degree between first and second sub-historical trajectory information that are not located in the same sector region can be determined as 0.

[0098] Specifically, when matching the size and depth information of dynamic obstacles in the first sub-historical trajectory information and the second sub-historical trajectory information located in the same sector area, the weights corresponding to the size matching degree and the depth matching degree can be preset. The initial matching result of the first sub-historical trajectory information and the second sub-historical trajectory information located in the same sector area is obtained by matching the size and depth information of dynamic obstacles in the first sub-historical trajectory information and the second sub-historical trajectory information located in the same sector area. The initial matching result includes the size matching degree corresponding to the size information and the depth matching degree corresponding to the depth information. The size matching degree and the depth matching degree can be multiplied by their corresponding weights and summed to obtain the first polar coordinate distance matching degree corresponding to the first sub-historical trajectory information and the second sub-historical trajectory information located in the same sector area.

[0099] In a specific example, if the preset center angle is 30°, the polar coordinate system can be divided into 12 sector regions. These 12 sector regions are numbered 1 to 12. Based on the first and second target position information, the sector region where each first and second sub-historical trajectory information is located is determined. The size and depth information of the dynamic obstacles in the first sub-historical trajectory information of sector region 1 are matched with the size and depth information of the dynamic obstacles in the second sub-historical trajectory information to obtain the first polar coordinate distance matching degree between the first and second sub-historical trajectory information in sector region 1. Similarly, the size and depth information of dynamic obstacles in the first sub-historical trajectory information of the sector regions numbered 2 to 12 are matched with the size and depth information of dynamic obstacles in the second sub-historical trajectory information to obtain the first polar coordinate distance matching degree between the first sub-historical trajectory information and the second sub-historical trajectory information in the sector regions numbered 2 to 12. The first polar coordinate distance matching degree between the first sub-historical trajectory information and the second sub-historical trajectory information that are not located in the same sector region is determined to be 0. For example, the first polar coordinate distance matching degree between the first sub-historical trajectory information in sector region numbered 1 and the second sub-historical trajectory information in sector region numbered 2 is determined to be 0.

[0100] Step S2312: Based on the first polar coordinate distance matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information, and the first preset threshold condition, determine the first matching result.

[0101] The first preset threshold condition can be set according to the actual situation. For example, the first preset threshold condition may include a first matching degree threshold.

[0102] In one implementation, when the first polar coordinate distance matching degree corresponding to the first sub-historical trajectory information and the second sub-historical trajectory information is greater than or equal to a first preset threshold condition, it can be determined that the first sub-historical trajectory information and the second sub-historical trajectory information are paired. Accordingly, the first trajectory information corresponding to the first sub-historical trajectory information and the second trajectory information corresponding to the second sub-historical trajectory information are paired, and the multiple pairs of paired first trajectory information and second trajectory information constitute the first matching result.

[0103] Step S2313: If the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is determined to be non-preset type information, the size information of the dynamic obstacle corresponding to each first sub-historical trajectory information is matched with the size information of the dynamic obstacle corresponding to each second sub-historical trajectory information by means of polar coordinate distance matching, so as to obtain the second polar coordinate distance matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information.

[0104] In one implementation, when the type information of the dynamic obstacle corresponding to each of the first sub-historical trajectory information is not the preset type information, the size information of the dynamic obstacle in the first sub-historical trajectory information located in the same sector area is matched with the size information of the dynamic obstacle in the second sub-historical trajectory information to obtain the second polar coordinate distance matching degree corresponding to the first sub-historical trajectory information and the second sub-historical trajectory information; the second polar coordinate distance matching degree corresponding to the first sub-historical trajectory information and the second sub-historical trajectory information that are not located in the same sector area can be determined as 0. 。

[0105] It should be noted that steps S2312 and S2313 in the embodiments of this disclosure can be executed in any order, without any restriction on the execution order.

[0106] Step S2314: Based on the matching degree of the second polar coordinate distance between each first sub-historical trajectory information and each second sub-historical trajectory information, and the second preset threshold condition, determine the first matching result.

[0107] The second preset threshold condition can be set according to the actual situation. For example, the second preset threshold condition may include a second matching degree threshold.

[0108] In one implementation, when the matching degree of the second polar coordinate distance corresponding to the first sub-historical trajectory information and the second sub-historical trajectory information is greater than or equal to a second preset threshold condition, it can be determined that the second sub-historical trajectory information and the second sub-historical trajectory information are paired. Correspondingly, the first trajectory information corresponding to the first sub-historical trajectory information and the second trajectory information corresponding to the second sub-historical trajectory information are paired, and the multiple pairs of paired first trajectory information and second trajectory information constitute the first matching result.

[0109] In this embodiment of the disclosure, based on the difference in the type information of dynamic obstacles included in the first sub-historical trajectory information and the second sub-historical trajectory information, the first sub-historical trajectory information and the second sub-historical trajectory information are matched in different ways, which effectively improves the accuracy of the first matching result, thereby improving the accuracy of the target matching result, thereby improving the reliability of the first enhanced training data, and making the trajectory predicted by the trajectory prediction model trained by the enhanced training dataset more accurate.

[0110] In one optional example, at least one matching method in this disclosure embodiment includes: area intersection-to-union ratio matching method. For example... Figure 6 As shown, step S231 of this embodiment may further include the following steps:

[0111] Step S2315: Determine the first position information of the dynamic obstacle corresponding to each first sub-historical trajectory information in the preset coordinate system.

[0112] The preset coordinate system is either a bird's-eye view coordinate system or an image coordinate system. The bird's-eye view coordinate system is the coordinate system within the bird's-eye view image. A bird's-eye view image is a three-dimensional drawing created by viewing the undulations of the ground from a high vantage point using perspective principles, resulting in a more realistic representation than a two-dimensional drawing. Coordinate system transformation can be used to convert the positional information of dynamic obstacles in each first sub-historical trajectory to the preset coordinate system, obtaining the first positional information corresponding to the positional information of dynamic obstacles in each first sub-historical trajectory within the preset coordinate system.

[0113] It should be noted that steps S2314 and S2315 in the embodiments of this disclosure can be executed in any order, without any restriction on the execution order.

[0114] Step S2316: Determine the second position information of the dynamic obstacle corresponding to each second sub-historical trajectory information in the preset coordinate system.

[0115] Specifically, the position information of dynamic obstacles in each second sub-historical trajectory information can be transformed into a preset coordinate system through coordinate system transformation, thereby obtaining the second position information corresponding to the position information of dynamic obstacles in each second sub-historical trajectory information in the preset coordinate system.

[0116] Step S2317: By using the area intersection-union ratio matching method, each first position information is matched with each second position information to obtain the area intersection-union ratio matching degree between the dynamic obstacle corresponding to each first sub-historical trajectory information and the dynamic obstacle corresponding to each second sub-historical trajectory information.

[0117] In one implementation, for each first position information, the coordinate value corresponding to the first position information can be determined as the vertex coordinate value or center point coordinate value of the first detection box. Based on the pre-set length and width information of the first detection box and the vertex coordinate value, the coordinate values ​​of the remaining vertices of the preset detection box are determined. For each second position information, the second position information can be determined as the vertex coordinate value or center point coordinate value of the second detection box. Based on the pre-set length and width information of the second preset detection box and the vertex coordinate value, the coordinate values ​​of the remaining vertices of the second detection box are determined. Using the four vertex coordinate values ​​of the first and second detection boxes, the area intersection-over-union ratio (IoU) of the dynamic obstacles corresponding to each first sub-historical trajectory information and the dynamic obstacles corresponding to each second sub-historical trajectory information is obtained.

[0118] In one implementation, the first trajectory information and the second trajectory information may further include detection boxes of dynamic obstacles corresponding to each time step; the position information of the dynamic obstacle may be the coordinates of the center point and the coordinates of the vertices of the detection box of the dynamic obstacle. The vertex coordinates of the detection boxes of dynamic obstacles in each first sub-historical trajectory information can be converted into the corresponding first vertex coordinates in a preset coordinate system; the vertex coordinates of the detection boxes of dynamic obstacles in each second sub-historical trajectory information can be converted into the corresponding second vertex coordinates in a preset coordinate system. Using the first vertex coordinates of the detection boxes of dynamic obstacles in each first sub-historical trajectory information and the second vertex coordinates of the detection boxes of dynamic obstacles in each second sub-historical trajectory information, the area intersection-union ratio (IUU) matching degree of the dynamic obstacles corresponding to the first and second sub-historical trajectories is obtained.

[0119] Step S2318: The area intersection-union ratio matching degree of each dynamic obstacle corresponding to each first sub-historical trajectory information and the dynamic obstacle corresponding to each second sub-historical trajectory information and its corresponding preset first weight, and the first polar coordinate distance matching degree or the second polar coordinate distance matching degree of each first sub-historical trajectory information and the corresponding preset second weight are weighted to obtain the first weighted matching degree.

[0120] The preset first weight and preset second weight can be set according to actual needs. For example, the first weight and the second weight can be set according to the accuracy or importance of the matching method. For instance, the preset second weight can be greater than the preset first weight. In one embodiment, the sum of the preset first weight and the preset second weight is 1.

[0121] In one implementation, the area intersection-union ratio matching degree of the dynamic obstacle corresponding to each first sub-historical trajectory information and the dynamic obstacle corresponding to each second sub-historical trajectory information is multiplied by a preset first weight to obtain the first matching degree of each first sub-historical trajectory information and each second sub-historical trajectory information.

[0122] When the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is a preset type information, the matching degree of the first polar coordinate distance between each first sub-historical trajectory information and the corresponding second sub-historical trajectory information is multiplied by a preset second weight to obtain the second matching degree of the dynamic obstacle corresponding to each first sub-historical trajectory information and the corresponding second sub-historical trajectory information. When the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is not a preset type information, the matching degree of the second polar coordinate distance between each first sub-historical trajectory information and the corresponding second sub-historical trajectory information is multiplied by a preset second weight to obtain the second matching degree of the dynamic obstacle corresponding to each first sub-historical trajectory information and the corresponding second sub-historical trajectory information.

[0123] The first and second matching degrees of the same second sub-historical trajectory information corresponding to each first sub-historical trajectory information are added together to obtain the first weighted matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information.

[0124] Step S2319: Determine the first matching result based on the first weighted matching degree and the third preset threshold condition.

[0125] The third preset threshold condition can be set according to the actual situation. For example, the third preset threshold condition may include a third matching degree threshold.

[0126] In one implementation, for each first sub-historical trajectory information, when the first weighted matching degree between the first sub-historical trajectory information and the second sub-historical trajectory information is greater than or equal to a third preset threshold condition, it is determined that the first sub-historical trajectory information and the second sub-historical trajectory information are paired. Correspondingly, the first trajectory information corresponding to the first sub-historical trajectory information and the second trajectory information corresponding to the second sub-historical trajectory information are paired, and the multiple pairs of paired first trajectory information and second trajectory information constitute the first matching result.

[0127] In one embodiment, the preset coordinate system may include: a bird's-eye view coordinate system and an image coordinate system, which can determine the first position information of the dynamic obstacle corresponding to each first sub-historical trajectory information in the bird's-eye view coordinate system and the first position information in the image coordinate system; and determine the second position information of the dynamic obstacle corresponding to each second sub-historical trajectory information in the bird's-eye view coordinate system and the second position information in the image coordinate system. The first sub-area intersection-union ratio matching degree can be determined based on the first position information of the dynamic obstacles corresponding to each first sub-historical trajectory information in the bird's-eye view coordinate system and the second position information of the dynamic obstacles corresponding to each second sub-historical trajectory information in the bird's-eye view coordinate system. The second sub-area intersection-union ratio matching degree can then be determined based on the first position information of the dynamic obstacles corresponding to each first sub-historical trajectory information in the image coordinate system and the second position information of the dynamic obstacles corresponding to each second sub-historical trajectory information in the image coordinate system. Then, weights corresponding to the first and second sub-area intersection-union ratio matching degrees are preset. The weight of the first sub-area intersection-union ratio matching degree is set to a preset first weight, and the weight of the second sub-area intersection-union ratio matching degree is set to a preset seventh weight, where the preset seventh weight is less than the preset first weight and the preset second weight. A weighted average is then applied to each first sub-area intersection-union ratio matching degree and its corresponding preset first weight, each second sub-area intersection-union ratio matching degree and its corresponding preset seventh weight, and each first sub-historical trajectory information with the first polar coordinate distance matching degree or the second polar coordinate distance matching degree corresponding to each second sub-historical trajectory information and the preset second weight, to obtain the first weighted matching degree.

[0128] In this embodiment of the disclosure, by comparing the area intersection-union ratio matching degree of dynamic obstacles in the first sub-historical trajectory information and the second sub-historical trajectory information, and combining the first polar coordinate distance matching degree or the second polar coordinate distance matching degree, the first matching result is determined, which effectively improves the reliability of the first matching result, thereby improving the accuracy of the target matching result, and thus improving the reliability of the first enhanced training data, making the trajectory predicted by the trajectory prediction model trained by the enhanced training dataset more accurate.

[0129] In an optional example, the first historical trajectory information in this embodiment includes third sub-historical trajectory information corresponding to historical time, and the second historical trajectory information includes fourth sub-historical trajectory information corresponding to historical time. The method for augmenting training data for trajectory prediction in this embodiment may further include: matching the third sub-historical trajectory information of each first trajectory information with the fourth sub-historical trajectory information of each second trajectory information to determine a second matching result.

[0130] Step S232 of this embodiment may further include: determining the target matching result based on the first matching result and the second matching result.

[0131] Specifically, a third matching result can be determined based on the first matching result using either the Hungarian algorithm or the KM algorithm, and a fourth sub-matching result can be determined based on the second matching result using either the Hungarian algorithm or the KM algorithm. Both the third and fourth matching results include at least one pair of trajectory information that successfully matches the first and second trajectory information.

[0132] In one implementation, when the same first trajectory information is successfully matched with multiple second trajectory information in the third and fourth matching results, the second trajectory information with the highest matching degree with the first trajectory information can be determined as the second trajectory information that matches the first trajectory information.

[0133] In this embodiment of the disclosure, the target matching result is obtained by simultaneously utilizing the first matching result and the second matching result. This not only improves the utilization rate of the data, but also effectively improves the accuracy of the matching result, thereby improving the reliability of the first enhanced training data and making the trajectory predicted by the trajectory prediction model trained through the enhanced training dataset more accurate.

[0134] In an optional example, such as Figure 7 As shown, the method for augmenting training data for trajectory prediction in this embodiment of the present disclosure may further include the following steps:

[0135] Step S410: Determine the third position information of each trajectory point in the third sub-historical trajectory information of each first trajectory information.

[0136] Specifically, the position information of the dynamic obstacle at each moment in each third sub-historical trajectory information can be determined as the third position information of the trajectory point in that third sub-trajectory information. For example, if the moments in the third sub-historical trajectory information are t7, t8, and t9, the position information of the dynamic obstacle corresponding to t7 is P1, the position information of the dynamic obstacle corresponding to t8 is P2, and the position information of the dynamic obstacle corresponding to t9 is P3. Accordingly, the third sub-historical trajectory information includes three trajectory points, and the third position information of these three trajectory points are P1, P2, and P3, respectively.

[0137] Step S420: Determine the fourth position information of each trajectory point in the fourth sub-historical trajectory information of each second trajectory information.

[0138] Specifically, the position information of the dynamic obstacle corresponding to each moment in each fourth sub-historical trajectory information can be determined as the fourth position information of the trajectory point in that fourth sub-trajectory information.

[0139] Step S430: For each third sub-historical trajectory information in each third sub-historical trajectory information, determine the average distance between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information.

[0140] In one implementation, for each third sub-historical trajectory information and each fourth sub-historical trajectory information, the third position information of each trajectory point in the third sub-historical trajectory information can be denoted as... The fourth position information of each trajectory point in the fourth sub-historical trajectory information is denoted as... Based on formula (3), the average distance between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in the fourth sub-historical trajectory information is determined;

[0141] (3)

[0142] Among them, s and Let be the average distance between trajectory points, n be the number of trajectory points, and n and i be positive integers. This refers to the third position information of trajectory point 1a in the third sub-historical trajectory information. This refers to the third position information of the na-th trajectory point in the third sub-historical trajectory information. This refers to the fourth position information of trajectory point 1b in the fourth sub-historical trajectory information. This refers to the fourth position information of the nth trajectory point in the fourth sub-historical trajectory information.

[0143] Step S440: Based on the average distance between the trajectory points in the third sub-historical trajectory information and the fourth position information of each trajectory point in the fourth sub-historical trajectory information, determine the matching degree of the average distance between the trajectory points in the third sub-historical trajectory information and the fourth sub-historical trajectory information.

[0144] The average distance between trajectory points can be normalized by using the normalized exponential function softmax to obtain the matching degree of the average distance between trajectory points between each third sub-historical trajectory information and each fourth sub-historical trajectory information. Specifically, the matching degree of the average distance between trajectory points between each third sub-historical trajectory information and each fourth sub-historical trajectory information can be obtained according to equations (4) and (5).

[0145] (4)

[0146] (5)

[0147] Among them, s jLet be the average distance between the third position information of each trajectory point in the j-th third sub-historical trajectory information and the fourth position information of each trajectory point in the j-th fourth sub-historical trajectory information. For s j The normalized value, Let be the average distance matching degree between the trajectory points of the j-th third sub-historical trajectory information and the j-th fourth sub-historical trajectory information, where j is a positive integer and 1≤j≤n.

[0148] Step S450: Based on the average distance matching degree between trajectory points of each third sub-historical trajectory information and each fourth sub-historical trajectory information, and the fourth preset threshold condition, determine the second matching result.

[0149] The fourth preset threshold condition can be set according to the actual situation. For example, the fourth preset threshold condition may include a fourth matching degree threshold.

[0150] In one implementation, when the average distance matching degree between the trajectory points of the third sub-historical trajectory information and the fourth sub-historical trajectory information is greater than or equal to a fourth preset threshold condition, it is determined that the third sub-historical trajectory information and the fourth sub-historical trajectory information are paired. Correspondingly, the first trajectory information corresponding to the third sub-historical trajectory information and the second trajectory information corresponding to the fourth sub-historical trajectory information are paired, and the multiple pairs of paired first trajectory information and second trajectory information constitute the second matching result.

[0151] In this embodiment, the second matching result is determined based on the average distance matching degree between the trajectory points of each third sub-historical trajectory information and each fourth sub-historical trajectory information, as well as the fourth preset threshold condition. This can effectively ensure the accuracy of the second matching result, and thus ensure the accuracy of the target matching result obtained based on the first matching result and the second matching result. This improves the reliability of the first enhanced training data, making the trajectory predicted by the trajectory prediction model trained through the enhanced training dataset more accurate.

[0152] In an optional example, such as Figure 8 As shown, the method for augmenting training data for trajectory prediction in this embodiment of the present disclosure may further include the following steps:

[0153] Step S510: For each third sub-historical trajectory information, determine the distribution likelihood between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information.

[0154] Among them, the likelihood of the multivariate Gaussian distribution of each trajectory point in each third sub-historical trajectory information and each trajectory point in each fourth sub-historical trajectory information is called the distribution likelihood of the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in the fourth sub-historical trajectory information.

[0155] In one implementation, each trajectory point in the third sub-historical trajectory information can be assumed to have a covariance matrix, and the third position information of each trajectory point in the third sub-historical trajectory information can be defined as follows: The fourth position information of each trajectory point in the fourth sub-historical trajectory information is: The vector of the third position information of the i-th trajectory point in the third sub-historical trajectory information can be represented as: The vector of the fourth position information of the i-th trajectory point in the fourth sub-historical trajectory information can be represented as: , 1≤i≤n; the trajectory points can be determined according to formula (6). At the trajectory point Distribution likelihood .

[0156] (6)

[0157] The higher the likelihood of the above distribution, the more likely the trajectory points are to be found. The closer to the trajectory point The center of the distribution. The likelihood between each trajectory point in the third sub-historical trajectory information and each trajectory point in the fourth sub-historical trajectory information can be determined based on formula (6). Then, based on formula (7), all the obtained distribution likelihoods are summed to obtain the distribution likelihood between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in the fourth sub-historical trajectory information. ;

[0158] (7)

[0159] Step S520: Based on the distribution likelihood between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information, determine the distribution likelihood matching degree between the third sub-historical trajectory information and each fourth sub-historical trajectory information.

[0160] The distribution likelihood can be normalized by using the normalized exponential function softmax to obtain the distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information.

[0161] Step S530: The distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information and its corresponding preset third weight, and the trajectory point average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information and its corresponding preset fourth weight are weighted to obtain the second weighted matching degree.

[0162] The preset third and fourth weights can be set according to actual needs. For example, they can be set based on the accuracy or importance of the matching method. For instance, the preset fourth weight can be greater than the preset third weight. In one implementation, the sum of the preset third and fourth weights is 1.

[0163] In one implementation, the distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information is multiplied by a preset third weight to obtain the third matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information.

[0164] The average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information is multiplied by a preset fourth weight to obtain the fourth matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information.

[0165] The third and fourth matching degrees of the same second sub-historical trajectory information corresponding to each first sub-historical trajectory information are added together to obtain the second weighted matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information.

[0166] Step S540 determines the second matching result based on the second weighted matching degree and the fifth preset threshold condition.

[0167] The fifth preset threshold condition can be set according to the actual situation. For example, the fifth preset threshold condition may include a fifth matching degree threshold.

[0168] In one implementation, for each third sub-historical trajectory information, when the second weighted matching degree between the third sub-historical trajectory information and the fourth sub-historical trajectory information is greater than or equal to a fifth preset threshold condition, it is determined that the third sub-historical trajectory information and the fourth sub-historical trajectory information are paired. Correspondingly, the first trajectory information corresponding to the third sub-historical trajectory information and the second trajectory information corresponding to the fourth sub-historical trajectory information are paired, and the multiple pairs of paired first trajectory information and second trajectory information constitute the second matching result.

[0169] In this embodiment of the disclosure, the second matching result is determined by combining the distribution likelihood between the trajectory points of the third sub-historical trajectory information and the trajectory points of the fourth sub-historical trajectory information with the average distance matching degree of the trajectory points. This effectively improves the accuracy of the second matching result, thereby improving the accuracy of the target matching result obtained through the second matching result. This improves the reliability of the first enhanced training data, making the trajectory predicted by the trajectory prediction model trained with the enhanced training dataset more accurate.

[0170] In an optional example, such as Figure 9 As shown, the method for augmenting training data for trajectory prediction in this embodiment of the present disclosure may further include the following steps:

[0171] Step S610: Based on the third position information of each trajectory point in the third sub-historical trajectory information of each first trajectory information, determine the first historical trajectory curvature of the third sub-historical trajectory information of each first trajectory information.

[0172] Specifically, the curvature of the third sub-historical trajectory information can be calculated using the curvature formula based on the third position information of each trajectory point in each third sub-historical trajectory information. The curvature of each third sub-historical trajectory information is called the first historical trajectory curvature of that third sub-historical trajectory information.

[0173] For example, for each third sub-historical trajectory information, the third position information of each trajectory point of that third sub-historical trajectory information is denoted as... A curvature can be calculated based on the position information of every three trajectory points. Therefore, the curvature corresponding to the third sub-historical trajectory information includes: ; Let be the curvature from trajectory point 1a to trajectory point 3a in the third sub-historical trajectory information. Let the curvature from the (n-2)ath trajectory point to the ath trajectory point in the third sub-historical trajectory information be defined as the curvature of the first historical trajectory of the third sub-historical trajectory information.

[0174] Step S620: Based on the fourth position information of each trajectory point in the fourth sub-historical trajectory information of each second trajectory information, determine the curvature of the second historical trajectory of the fourth sub-historical trajectory information of each second trajectory information.

[0175] Specifically, the curvature corresponding to each fourth sub-historical trajectory information can be calculated using the curvature formula based on the fourth position information of each trajectory point in each fourth sub-historical trajectory information. The curvature corresponding to each fourth sub-historical trajectory information is called the second historical trajectory curvature of that fourth sub-historical trajectory information.

[0176] Step S630: For each third sub-historical trajectory information of the first trajectory information, determine the trajectory curvature matching degree between the first historical trajectory curvature of the third sub-historical trajectory information and the second historical trajectory curvature of each fourth sub-historical trajectory information.

[0177] In one implementation, the point-by-point average curvature of each third sub-historical trajectory information and each fourth sub-historical trajectory information can be determined by formula (8) based on the curvature of the first historical trajectory and the curvature of the second historical trajectory.

[0178] (8)

[0179] in, and The point-by-point average curvature between the third and fourth sub-historical trajectory information. The curvature (first historical trajectory curvature) is the curvature from the ia-th trajectory point to the a-th trajectory point in the third sub-historical trajectory information. The curvature (second historical trajectory curvature) is the curvature between the ibth trajectory point and the bth trajectory point in the fourth sub-historical trajectory information.

[0180] The trajectory curvature matching degree between the first historical trajectory curvature of each third sub-historical trajectory and the second historical trajectory curvature of each fourth sub-historical trajectory can be obtained by normalizing the reciprocal of the point-by-point average curvature of the normalized exponential function softmax. For example, the point-by-point average curvature between the third and fourth sub-historical trajectory information can be denoted as... According to formula (9) and ,get normalized value According to formula (10) and Obtain the matching degree between the historical trajectory information of the third sub-sub and the historical trajectory information of the fourth sub-sub. and will The trajectory curvature matching degree between the third and fourth sub-historical trajectory information is determined.

[0181] (9)

[0182] (10)

[0183] Step S640: The trajectory curvature matching degree between the first historical trajectory curvature of each third sub-historical trajectory information and the second historical trajectory curvature of each fourth sub-historical trajectory information, and its corresponding preset fifth weight; the distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information, and its corresponding preset sixth weight; and the trajectory point average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information, and its corresponding preset seventh weight, are weighted to obtain the third weighted matching degree.

[0184] The preset fifth, sixth, and seventh weights can be set according to actual needs. For example, each weight can be set based on the accuracy or importance of the matching method. For instance, the preset sixth weight is greater than the preset fifth weight, and the preset fifth weight is greater than the preset seventh weight. In one implementation, the sum of the preset fifth, sixth, and seventh weights is 1.

[0185] In one implementation, the distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information is multiplied by a preset sixth weight to obtain the fifth matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information.

[0186] The average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information is multiplied by a preset seventh weight to obtain the sixth matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information.

[0187] The trajectory curvature matching degree between the first historical trajectory curvature of each third sub-historical trajectory information and the second historical trajectory curvature of each fourth sub-historical trajectory information is multiplied by a preset fifth weight to obtain the seventh matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information.

[0188] The fifth, sixth, and seventh matching degrees of the same second sub-historical trajectory information corresponding to each first sub-historical trajectory information are added together to obtain the third weighted matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information.

[0189] Step S650: Determine the second matching result based on the third weighted matching degree and the sixth preset threshold condition.

[0190] The sixth preset threshold condition can be set according to the actual situation. For example, the sixth preset threshold condition may include the sixth matching degree threshold.

[0191] In one implementation, for each third sub-historical trajectory information, when the third weighted matching degree between the third sub-historical trajectory information and the fourth sub-historical trajectory information is greater than or equal to a sixth preset threshold condition, it is determined that the third sub-historical trajectory information and the fourth sub-historical trajectory information are paired. Correspondingly, the first trajectory information corresponding to the third sub-historical trajectory information and the second trajectory information corresponding to the fourth sub-historical trajectory information are paired, and the multiple pairs of paired first trajectory information and second trajectory information constitute the second matching result.

[0192] In this embodiment of the disclosure, the second matching result is determined by comprehensively considering the distribution likelihood matching degree, the average distance matching degree of trajectory points, and the trajectory curvature matching degree between the third and fourth sub-historical trajectory information. This effectively improves the accuracy of the second matching result, thereby improving the accuracy of the matching result obtained through the second matching result. This, in turn, improves the reliability of the first enhanced training data, making the trajectory predicted by the trajectory prediction model trained with the enhanced training dataset more accurate.

[0193] In an optional example, step S232 in this embodiment of the present disclosure may further include: fusing the first matching result and the second matching result to determine the target matching result.

[0194] In one implementation, a third matching result can be determined based on the first matching result using either the Hungarian algorithm or the KM algorithm, and a fourth sub-matching result can be determined based on the second matching result using either the Hungarian algorithm or the KM algorithm. Both the third and fourth matching results include at least one pair of trajectory information that successfully matches the first and second trajectory information.

[0195] Each pair of trajectory information that successfully matches the first and second trajectory information in the third matching result is determined as the first sub-target matching result. The fourth matching result is then checked to determine if there are any successfully matched trajectories of the first and second trajectory information not included in the third matching result. If so, these successfully matched trajectories are selected from the fourth matching result and determined as the second sub-target matching result. The first and second sub-target matching results constitute the target matching result. If the fourth matching result does not contain any successfully matched trajectories of the first and second trajectory information not included in the third matching result, then the first sub-target matching result is determined as the target matching result.

[0196] In this embodiment of the disclosure, the first matching result is supplemented by the second matching result, which effectively improves the success rate of matching the first trajectory information with the second trajectory information, while also ensuring the accuracy of the target matching result.

[0197] Figure 10 This is a flowchart illustrating a training method for a trajectory prediction model provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, vehicles, etc. Figure 10 As shown, it includes the following steps:

[0198] Step S710: Obtain the enhanced training dataset.

[0199] The enhanced training dataset can be obtained through embodiments of this disclosure. The enhanced training dataset may include multiple first enhanced training data sets, each first enhanced training data set including: first historical trajectory sample information and first future trajectory annotation information. In one implementation, the enhanced training dataset may further include at least one second enhanced training data set.

[0200] Step S720: Based on at least one first historical trajectory sample information in the enhanced training dataset, at least one first predicted trajectory information is obtained using the initial trajectory prediction model to be trained.

[0201] The initial trajectory prediction model to be trained can be a neural network, such as CNN, SCNN, RNN, LSTM (Long Short-Term Memory), ResNet, DenseNet, EfficientNet, etc.

[0202] The first historical trajectory sample information of each first augmented training data in the augmented training dataset can be input into the initial trajectory prediction model to be trained. The initial trajectory prediction model to be trained outputs the first predicted trajectory information corresponding to each first augmented training data.

[0203] In one implementation, when the augmented training dataset includes second augmented training data, the second historical trajectory sample information of each second augmented training data in the augmented training dataset can be input into the initial trajectory prediction model to be trained, and the initial trajectory prediction model to be trained outputs the first predicted trajectory information corresponding to each second augmented training data.

[0204] Step S730: Determine the first loss value based on at least one first predicted trajectory information and the first future trajectory annotation information corresponding to at least one historical trajectory information in the enhanced training dataset.

[0205] Specifically, based on the first predicted trajectory information and the first future trajectory annotation information of each first augmented training data, a first loss value is determined using a first loss function. For example, the first loss function can be any implementable loss function, such as the cross-entropy error function or the mean square error function.

[0206] In one implementation, when the augmented training dataset includes second augmented training data, a first loss value can be determined using a loss function based on the first predicted trajectory information and the first future trajectory annotation information of each first augmented training data, as well as the first predicted trajectory information and the second future trajectory annotation information of each second augmented training data.

[0207] Step S740: Train the initial trajectory prediction model to be trained based on the first loss value.

[0208] The first loss value can be reduced by adjusting the parameters of the initial trajectory prediction model to be trained, thereby ensuring that the offset compensation model meets the preset training completion conditions. The adjustment of the parameters of the initial trajectory prediction model can be achieved using any implementable parameter optimizer. For example, the parameter optimizer can be SGD (Stochastic Gradient Descent), Adagrad (Adaptive Gradient Algorithm), Adam (Adaptive Moment Estimation), RMSprop (Root Mean Square Prop), LBFGS (Limited-memory Broyden–Fletcher–Goldfarb–Shanno), etc. For instance, the gradient of each parameter of the initial trajectory prediction model to be trained can be calculated using the parameter optimizer. The gradient represents the direction in which the first loss value decreases the most. The parameters are then adjusted along the gradient direction, and the above steps are repeated until the first loss value no longer decreases, confirming that the preset training completion conditions are met, thus obtaining the first trajectory prediction model.

[0209] Step S750: In response to the initial trajectory prediction model after training meeting the preset training completion conditions, the initial trajectory prediction model after training is determined as the first trajectory prediction model.

[0210] The first trajectory prediction model can be used to predict the trajectory information of dynamic obstacles.

[0211] In this embodiment of the disclosure, an initial trajectory prediction model to be trained is trained using an enhanced training dataset. Since the first enhanced training data in the enhanced training dataset incorporates point cloud data, the initial trajectory prediction model to be trained can better learn the coupling of motion features, thereby effectively improving the accuracy of the trajectory information predicted by the first trajectory prediction model.

[0212] In an optional example, such as Figure 11 As shown, the training method for the trajectory prediction model in this embodiment may further include the following steps:

[0213] Step S810: Obtain a training image set collected within a preset time period for a scene containing at least one dynamic obstacle.

[0214] The preset time period can be selected according to actual needs. The training image set may include multiple training images, each of which includes at least one dynamic obstacle. In one implementation, training images can be acquired using a visual sensor.

[0215] Step S820: Based on the training image set, determine the training trajectory information corresponding to at least one dynamic obstacle.

[0216] Each training trajectory information includes a first sub-trajectory information corresponding to the earlier time segment within a preset time period, and a second sub-trajectory information corresponding to the later time segment within the same preset time period. The first and second sub-trajectory information correspond one-to-one. In other words, each trajectory information includes the first sub-trajectory information within the training trajectory information corresponding to the earlier time segment and the second sub-trajectory information within the training trajectory information corresponding to the later time segment, divided chronologically into earlier and later time segments. The trajectory information of the dynamic obstacle obtained from the training images is called the training trajectory information of that dynamic obstacle. Each training trajectory information includes: the motion data of the dynamic obstacle corresponding to each moment within the preset time period.

[0217] Each training image in the training image set can be input into a neural network used to predict trajectory information. The neural network then outputs training trajectory information corresponding to at least one dynamic obstacle. Alternatively, the training trajectory information corresponding to at least one dynamic obstacle in each training image can be manually labeled.

[0218] Step S830: Take the first sub-trajectory information of at least one training trajectory information as the third historical sample trajectory information, and use the first trajectory prediction model to obtain at least one second predicted trajectory information.

[0219] In this process, the trajectory information of each second historical sample is input into the first trajectory prediction model, and the first trajectory prediction model outputs the second predicted trajectory information of each training trajectory information.

[0220] Step S840: Use the second sub-trajectory information of at least one training trajectory information as the third future trajectory annotation information, and determine the second loss value based on at least one third future trajectory annotation information and at least one second predicted trajectory information.

[0221] Specifically, for each training trajectory information, a second loss value is determined using a second loss function based on the second predicted trajectory information and the third future trajectory annotation information of that training trajectory information. The second loss function can be any applicable loss function, such as the cross-entropy error function or the mean squared error function. The first loss function and the second loss function can be the same or different.

[0222] Step S850: Train the first trajectory prediction model based on the second loss value.

[0223] The second loss value can be reduced by adjusting the parameters of the first trajectory prediction model, thus ensuring that the first trajectory prediction model meets the preset training completion conditions. The adjustment of the parameters of the first trajectory prediction model can be achieved using any implementable parameter optimizer. For example, the parameter optimizer can be SGD, Adagrad, Adam, RMSprop, LBFGS, etc. For instance, the gradient of each parameter of the first trajectory prediction model can be calculated using the parameter optimizer. The gradient represents the direction in which the second loss value decreases the most. The parameters are then adjusted along the gradient direction, and the above steps are repeated until the second loss value no longer decreases, confirming that the preset training completion conditions are met, thus obtaining the second trajectory prediction model.

[0224] Step S860: In response to the first trajectory prediction model after training meeting the preset training completion conditions, the first trajectory prediction model after training is determined as the second trajectory prediction model.

[0225] The second trajectory prediction model can be used to predict the trajectory information of dynamic obstacles.

[0226] In this embodiment of the disclosure, the trajectory prediction model is optimized and trained using a training image set, so that the trained second trajectory prediction model can better predict the trajectory information of dynamic obstacles based on the images collected by the visual sensor, thereby improving the accuracy of the predicted trajectory information.

[0227] Any of the methods for enhancing training data and training trajectory prediction models provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to terminal devices and servers. Alternatively, any of the methods for enhancing training data and training trajectory prediction models provided in this disclosure can be executed by a processor, such as by a processor calling corresponding instructions stored in memory to execute any of the methods for enhancing training data and training trajectory prediction models mentioned in this disclosure. Further details will not be elaborated upon below.

[0228] Exemplary device

[0229] Figure 12 This is a structural block diagram of a device for enhancing training data for trajectory prediction in one embodiment of this disclosure. Figure 12 As shown, the device for enhancing training data for trajectory prediction includes: a first acquisition module 810, a first trajectory information determination module 820, a first trajectory information matching module 830, a first enhanced data construction module 840, and an enhanced training dataset determination module 850.

[0230] The first acquisition module 810 is used to acquire a set of visual images and a set of point cloud data collected within the same time period for a scene containing at least one dynamic obstacle; wherein, the time period includes an intermediate time, historical time before the intermediate time, and future time after the intermediate time;

[0231] The first trajectory information determination module 820 is configured to determine, based on the visual image set, first trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle, and based on the point cloud dataset, second trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle; wherein, the first trajectory information includes first historical trajectory information corresponding to the intermediate time and the historical time, and first future trajectory information corresponding to the future time; the second trajectory information includes second historical trajectory information corresponding to the intermediate time and the historical time, and second future trajectory information corresponding to the future time;

[0232] The first trajectory information matching module 830 is used to match the first historical trajectory information of each first trajectory information with the second historical trajectory information of each second trajectory information to obtain a target matching result; wherein, the target matching result includes at least one pair of trajectory information in which the first trajectory information and the second trajectory information are successfully matched;

[0233] The first augmented data construction module 840 is used to construct first augmented training data for each pair of trajectory information in the at least one pair of trajectory information, using the first historical trajectory information of the first trajectory information as the first historical trajectory sample information and the second future trajectory information of the second trajectory information as the first future trajectory annotation information.

[0234] The augmented training dataset determination module 850 is used to determine the augmented training dataset based on the first augmented training data corresponding to each pair of trajectory information in the at least one pair of trajectory information.

[0235] In one embodiment of this disclosure, the target matching result further includes at least one first trajectory information of unsuccessful matching, such as... Figure 13 The enhancement device for training data for trajectory prediction in another embodiment of this disclosure further includes:

[0236] The second trajectory information determination module 860 is used to take at least one first trajectory information that failed to match as trajectory information to be set.

[0237] The second enhanced data construction module 870 is used to construct second enhanced training data for each of the at least one trajectory information to be set, using the historical trajectory information of the trajectory information to be set as the second historical trajectory sample information and the future trajectory information of the trajectory information to be set as the second future trajectory annotation information.

[0238] The augmented training dataset determination module 850 is further configured to determine the augmented training dataset based on the at least one first augmented training data and the at least one second augmented training data.

[0239] In one embodiment of this disclosure, the first historical trajectory information includes first sub-historical trajectory information corresponding to the intermediate time, and the second historical trajectory information includes second sub-historical trajectory information corresponding to the intermediate time; the first trajectory information matching module 830 includes:

[0240] The first sub-matching module 831 is used to match the first sub-historical trajectory information of each first historical trajectory information and the second sub-historical trajectory information of each second historical trajectory information through at least one matching method to determine the first matching result;

[0241] Sub-determination module 832 is used to determine the target matching result based on the first matching result.

[0242] In one embodiment of this disclosure, the at least one matching method includes: polar coordinate distance matching method; the first sub-matching module 831 includes:

[0243] The first matching unit 8311 is used to respond to the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information being preset type information, and to match the size information and depth information of the dynamic obstacle corresponding to each first sub-historical trajectory information with the size information and depth information of the dynamic obstacle corresponding to each second sub-historical trajectory information through the polar coordinate distance matching method, so as to obtain the first polar coordinate distance matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information respectively;

[0244] The first determining unit 8312 is configured to determine a first matching result based on the first polar coordinate distance matching degree between each of the first sub-historical trajectory information and each of the second sub-historical trajectory information, and a first preset threshold condition; or,

[0245] The second matching unit 8313 is used to respond to the fact that the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is not preset type information, and to match the size information of the dynamic obstacle corresponding to each first sub-historical trajectory information with the size information of the dynamic obstacle corresponding to each second sub-historical trajectory information through the polar coordinate distance matching method, so as to obtain the second polar coordinate distance matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information.

[0246] The second determining unit 8314 is used to determine the first matching result based on the matching degree of the second polar coordinate distance between each of the first sub-historical trajectory information and each of the second sub-historical trajectory information, and the second preset threshold condition.

[0247] In one embodiment of this disclosure, the at least one matching method further includes an area intersection-union ratio matching method; the first sub-matching module 831 includes:

[0248] The first position determination unit 8315 is used to determine the first position information of the dynamic obstacle corresponding to each first sub-historical trajectory information in a preset coordinate system.

[0249] The second position determination unit 8316 is used to determine the second position information of the dynamic obstacle corresponding to each second sub-historical trajectory information in the preset coordinate system.

[0250] The third matching unit 8317 is used to match each first position information with each second position information through the area intersection-union ratio matching method, so as to obtain the area intersection-union ratio matching degree between the dynamic obstacle corresponding to each first sub-historical trajectory information and the dynamic obstacle corresponding to each second sub-historical trajectory information.

[0251] The third determining unit 8318 is used to weight the area intersection-union ratio matching degree of the dynamic obstacle corresponding to each first sub-historical trajectory information and the dynamic obstacle corresponding to each second sub-historical trajectory information and its corresponding preset first weight, and the first polar coordinate distance matching degree or the second polar coordinate distance matching degree of each first sub-historical trajectory information and the preset second weight to obtain a first weighted matching degree.

[0252] The fourth determining unit 8319 is used to determine the first matching result based on the first weighted matching degree and the third preset threshold condition.

[0253] In one embodiment of this disclosure, the first historical trajectory information includes third sub-historical trajectory information corresponding to the historical time, and the second historical trajectory information includes fourth sub-historical trajectory information corresponding to the historical time; the first trajectory information matching module 830 further includes:

[0254] The second sub-matching module 833 is used to match the third sub-historical trajectory information of each first trajectory information and the fourth sub-historical trajectory information of each second trajectory information to determine the second matching result;

[0255] The sub-determination module 832 is further configured to determine the target matching result based on the first matching result and the second matching result.

[0256] In one embodiment of this disclosure, the second sub-matching module 833 includes:

[0257] The third position determination unit 8331 is used to determine the third position information of each trajectory point in the third sub-historical trajectory information of each first trajectory information;

[0258] The fourth position determination unit 8332 is used to determine the fourth position information of each trajectory point in the fourth sub-historical trajectory information of each second trajectory information;

[0259] The fourth matching unit 8333 is used to determine the average distance between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in the fourth sub-historical trajectory information for each third sub-historical trajectory information.

[0260] The fifth determining unit 8334 is used to determine the matching degree of the average distance between the trajectory points in the third sub-historical trajectory information and the fourth sub-historical trajectory information based on the average distance between the trajectory points in the third sub-historical trajectory information and the fourth position information of the trajectory points in the fourth sub-historical trajectory information.

[0261] The sixth determining unit 8335 is used to determine the second matching result based on the average distance matching degree of trajectory points between each third sub-historical trajectory information and each fourth sub-historical trajectory information, and a fourth preset threshold condition.

[0262] In one embodiment of this disclosure, the second sub-matching module 833 includes:

[0263] The seventh determining unit 8336 is used to determine the distribution likelihood between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information for each third sub-historical trajectory information.

[0264] The eighth determining unit 8337 is used to determine the distribution likelihood matching degree between the third sub-historical trajectory information and each fourth sub-historical trajectory information based on the distribution likelihood between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information.

[0265] The ninth determining unit 8338 is used to weight the distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information and its corresponding preset third weight, and the average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information and its corresponding preset fourth weight, to obtain a second weighted matching degree.

[0266] The tenth determining unit 8339 determines the second matching result based on the second weighted matching degree and the fifth preset threshold condition.

[0267] In one embodiment of this disclosure, the second sub-matching module 833 includes:

[0268] The eleventh determining unit 8411 is used to determine the first historical trajectory curvature of the third sub-historical trajectory information of each first trajectory information based on the third position information of each trajectory point in the third sub-historical trajectory information of each first trajectory information.

[0269] The twelfth determining unit 8412 is used to determine the curvature of the second historical trajectory of the fourth sub-historical trajectory information of each second trajectory information based on the fourth position information of each trajectory point in the fourth sub-historical trajectory information of each second trajectory information.

[0270] The thirteenth determining unit 8413 is used to determine the trajectory curvature matching degree between the first historical trajectory curvature of each third sub-historical trajectory information and the second historical trajectory curvature of each fourth sub-historical trajectory information for each first trajectory information.

[0271] The fourteenth determining unit 8414 performs a weighted average of the following: the trajectory curvature matching degree between the first historical trajectory curvature of each third sub-historical trajectory information and the second historical trajectory curvature of each fourth sub-historical trajectory information, and their corresponding preset fifth weight; the distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information, and their corresponding preset sixth weight; and the trajectory point average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information, and their corresponding preset seventh weight, to obtain a third weighted matching degree.

[0272] The fifteenth determining unit 8415 determines the second matching result based on the third weighted matching degree and the sixth preset threshold condition.

[0273] In one embodiment of this disclosure, the sub-determination module 832 is further configured to fuse the first matching result and the second matching result to determine the target matching result.

[0274] Figure 14 This is a structural block diagram of a training device for a trajectory prediction model in one embodiment of this disclosure. Figure 14 As shown, the training device for the trajectory prediction model includes: a second acquisition module 910, a first trajectory information prediction module 920, a first loss determination module 930, a first training module 940, and a first model determination module 950.

[0275] The second acquisition module 910 is used to acquire an enhanced training dataset, which is obtained by the method described in the embodiments of this disclosure.

[0276] The first trajectory information prediction module 920 is used to obtain at least one first predicted trajectory information based on at least one first historical trajectory sample information in the enhanced training dataset and using the initial trajectory prediction model to be trained.

[0277] The first loss determination module 930 is used to determine a first loss value based on the at least one first predicted trajectory information and the first future trajectory annotation information in the enhanced training dataset that corresponds to each of the at least one historical trajectory information.

[0278] The first training module 940 is used to train the initial trajectory prediction model to be trained based on the first loss value.

[0279] The first model determination module 950 is used to determine the trained initial trajectory prediction model as the first trajectory prediction model in response to the training completion condition being met by the trained initial trajectory prediction model.

[0280] In one embodiment of this disclosure, such as Figure 15 As shown, the device further includes:

[0281] The third acquisition module 960 is used to acquire a training image set collected within a preset time period for a scene containing at least one dynamic obstacle;

[0282] The third trajectory information determination module 970 is used to determine the training trajectory information corresponding to each of the at least one dynamic obstacle based on the training image set. Each training trajectory information includes a first sub-trajectory information corresponding to the earlier time period in the preset time period and a second sub-trajectory information corresponding to the later time period in the preset time period. The first sub-trajectory information and the second sub-trajectory information correspond one-to-one.

[0283] The second trajectory information prediction module 980 is used to take the first sub-trajectory information of the at least one training trajectory information as the third historical sample trajectory information, and use the first trajectory prediction model to obtain at least one second predicted trajectory information.

[0284] The second loss determination module 990 is used to take the second sub-trajectory information of the at least one training trajectory information as the third future trajectory annotation information, and determine the second loss value based on the at least one third future trajectory annotation information and at least one second predicted trajectory information.

[0285] The second training module 1000 is used to train the first trajectory prediction model based on the second loss value.

[0286] The second model determination module 1100 is used to determine the trained first trajectory prediction model as the second trajectory prediction model in response to the first trajectory prediction model after training meeting the preset training completion conditions.

[0287] Exemplary electronic devices

[0288] Below, for reference Figure 16 To describe an electronic device according to embodiments of the present disclosure. Figure 16 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Figure 16 As shown, the electronic device includes one or more processors 100 and memory 110.

[0289] The processor 100 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0290] The memory 110 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 100 may execute the program instructions to implement the methods for augmenting training data for trajectory prediction, training methods for trajectory prediction models, and / or other desired functions described in the various embodiments of this disclosure above. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0291] In one example, the electronic device may also include an input device 120 and an output device 130, which are interconnected via a bus system and / or other forms of connection mechanism (not shown). For example, the input device 120 may be the microphone or microphone array described above, used to capture input signals from a sound source.

[0292] In addition, the input device 120 may also include, for example, a keyboard, a mouse, etc.

[0293] The output device 130 can output various information to the outside, including determined distance information, direction information, etc. The output device 130 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0294] Of course, for the sake of simplicity, Figure 16 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0295] Exemplary computer program products and computer-readable storage media

[0296] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods for enhancing training data for trajectory prediction and training a trajectory prediction model according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0297] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0298] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods for enhancing training data for trajectory prediction and training a trajectory prediction model according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0299] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0300] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0301] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0302] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0303] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0304] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0305] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0306] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for augmenting training data for trajectory prediction, comprising: Acquire a set of visual images and a point cloud dataset collected within the same time period for a scene containing at least one dynamic obstacle; wherein the time period includes an intermediate moment, historical time before the intermediate moment, and future time after the intermediate moment; Based on the visual image set, first trajectory information corresponding to each dynamic obstacle in the at least one dynamic obstacle is determined, and second trajectory information corresponding to each dynamic obstacle in the at least one dynamic obstacle is determined based on the point cloud dataset; wherein, the first trajectory information includes first historical trajectory information corresponding to the intermediate time and the historical time, and first future trajectory information corresponding to the future time; the second trajectory information includes second historical trajectory information corresponding to the intermediate time and the historical time, and second future trajectory information corresponding to the future time; The first historical trajectory information of each first trajectory information is matched with the second historical trajectory information of each second trajectory information to obtain a target matching result; wherein, the target matching result includes at least one pair of trajectory information that is successfully matched between the first trajectory information and the second trajectory information; For each pair of trajectory information in the at least one pair of trajectory information, the first historical trajectory information of the first trajectory information is used as the first historical trajectory sample information, and the second future trajectory information of the second trajectory information is used as the first future trajectory annotation information to construct the first enhanced training data; An enhanced training dataset is determined based on the first enhanced training data corresponding to each pair of trajectory information in the at least one pair of trajectory information.

2. The method according to claim 1, wherein, The target matching result also includes at least one piece of the first trajectory information that failed to match, and the method further includes: At least one of the first trajectory information that failed to match is identified as trajectory information to be set; For each trajectory information to be set in at least one trajectory information to be set, the historical trajectory information of the trajectory information to be set is used as the second historical trajectory sample information, and the future trajectory information of the trajectory information to be set is used as the second future trajectory annotation information to construct the second enhanced training data. The step of determining the augmented training dataset based on the first augmented training data corresponding to each pair of trajectory information in the at least one pair of trajectory information includes: The augmented training dataset is determined based on the at least one first augmented training data and the at least one second augmented training data.

3. The method according to claim 1, wherein, The first historical trajectory information includes first sub-historical trajectory information corresponding to the intermediate time, and the second historical trajectory information includes second sub-historical trajectory information corresponding to the intermediate time; The process of matching the first historical trajectory information of each first trajectory information with the second historical trajectory information of each second trajectory information to obtain the target matching result includes: The first sub-historical trajectory information of each first historical trajectory information and the second sub-historical trajectory information of each second historical trajectory information are matched using at least one matching method to determine the first matching result; Based on the first matching result, the target matching result is determined.

4. The method according to claim 3, wherein, The at least one matching method includes polar coordinate distance matching; The step of matching each first sub-historical trajectory information of each first historical trajectory information and each second sub-historical trajectory information of each second historical trajectory information using at least one matching method to determine a first matching result includes: When it is determined that the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is a preset type information, the size information and depth information of the dynamic obstacle corresponding to each first sub-historical trajectory information are matched with the size information and depth information of the dynamic obstacle corresponding to each second sub-historical trajectory information by means of the polar coordinate distance matching method, so as to obtain the first polar coordinate distance matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information. Based on the matching degree of the first polar coordinate distance between each of the first sub-historical trajectory information and each of the second sub-historical trajectory information, and a first preset threshold condition, a first matching result is determined; or, When it is determined that the type information of the dynamic obstacle corresponding to each first sub-historical trajectory information is a non-preset type information, the size information of the dynamic obstacle corresponding to each first sub-historical trajectory information is matched with the size information of the dynamic obstacle corresponding to each second sub-historical trajectory information by means of the polar coordinate distance matching method, so as to obtain the second polar coordinate distance matching degree between each first sub-historical trajectory information and each second sub-historical trajectory information. Based on the matching degree of the second polar coordinate distance between each of the first sub-historical trajectory information and each of the second sub-historical trajectory information, and the second preset threshold condition, the first matching result is determined.

5. The method according to claim 4, wherein, The at least one matching method also includes the area intersection-union ratio matching method; The step of matching the first sub-historical trajectory information of each first historical trajectory information and the second sub-historical trajectory information of each second historical trajectory information using at least one matching method to determine the first matching result includes: Determine the first position information of the dynamic obstacle corresponding to each first sub-historical trajectory information in the preset coordinate system; Determine the second position information of the dynamic obstacle corresponding to each second sub-historical trajectory information in the preset coordinate system; By using the area intersection-union ratio matching method, each first position information is matched with each second position information to obtain the area intersection-union ratio matching degree between the dynamic obstacle corresponding to each first sub-historical trajectory information and the dynamic obstacle corresponding to each second sub-historical trajectory information. The area intersection-union ratio matching degree of each dynamic obstacle corresponding to each first sub-historical trajectory information and the dynamic obstacle corresponding to each second sub-historical trajectory information, and their corresponding preset first weight, and the first polar coordinate distance matching degree or the second polar coordinate distance matching degree of each first sub-historical trajectory information and the preset second weight are weighted to obtain the first weighted matching degree; The first matching result is determined based on the first weighted matching degree and the third preset threshold condition.

6. The method according to any one of claims 3-5, wherein, The first historical trajectory information includes third sub-historical trajectory information corresponding to the historical time, and the second historical trajectory information includes fourth sub-historical trajectory information corresponding to the historical time; The method further includes: The third sub-historical trajectory information of each first trajectory information and the fourth sub-historical trajectory information of each second trajectory information are matched to determine the second matching result; Determining the target matching result based on the first matching result includes: The target matching result is determined based on the first matching result and the second matching result.

7. The method according to claim 6, wherein, The step of matching the third sub-historical trajectory information of each of the first trajectory information and the fourth sub-historical trajectory information of each of the second trajectory information to determine the second matching result includes: Determine the third position information of each trajectory point in the third sub-historical trajectory information of each of the first trajectory information; Determine the fourth position information of each trajectory point in the fourth sub-historical trajectory information of each of the second trajectory information; For each third sub-historical trajectory information in each third sub-historical trajectory information, the average distance between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information is determined respectively. Based on the average distance between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information, the matching degree of the average distance between the trajectory points in the third sub-historical trajectory information and the fourth sub-historical trajectory information is determined. The second matching result is determined based on the average distance matching degree between the trajectory points of each third sub-historical trajectory information and each fourth sub-historical trajectory information, and the fourth preset threshold condition.

8. The method according to claim 7, wherein, The step of matching the third sub-historical trajectory information of each first trajectory information with the fourth sub-historical trajectory information of each second trajectory information to determine the second matching result further includes: For each third sub-historical trajectory information, the distribution likelihood between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information is determined respectively; Based on the distribution likelihood between the third position information of each trajectory point in the third sub-historical trajectory information and the fourth position information of each trajectory point in each fourth sub-historical trajectory information, the distribution likelihood matching degree between the third sub-historical trajectory information and each fourth sub-historical trajectory information is determined. The distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information and its corresponding preset third weight, and the average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information and its corresponding preset fourth weight are weighted to obtain the second weighted matching degree. The second matching result is determined based on the second weighted matching degree and the fifth preset threshold condition.

9. The method according to claim 8, wherein, The step of matching the third sub-historical trajectory information of each first trajectory information with the fourth sub-historical trajectory information of each second trajectory information to determine the second matching result further includes: Based on the third position information of each trajectory point in the third sub-historical trajectory information of each first trajectory information, the first historical trajectory curvature of the third sub-historical trajectory information of each first trajectory information is determined. Based on the fourth position information of each trajectory point in the fourth sub-historical trajectory information of each second trajectory information, the curvature of the second historical trajectory information of the fourth sub-historical trajectory information of each second trajectory information is determined. For each third sub-historical trajectory information of the first trajectory information, the trajectory curvature matching degree between the first historical trajectory curvature of the third sub-historical trajectory information and the second historical trajectory curvature of each fourth sub-historical trajectory information is determined respectively. The trajectory curvature matching degree between the first historical trajectory curvature of each third sub-historical trajectory information and the second historical trajectory curvature of each fourth sub-historical trajectory information, and its corresponding preset fifth weight, the distribution likelihood matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information, and its corresponding preset sixth weight, and the trajectory point average distance matching degree between each third sub-historical trajectory information and each fourth sub-historical trajectory information, and its corresponding preset seventh weight, are weighted together to obtain the third weighted matching degree; The second matching result is determined based on the third weighted matching degree and the sixth preset threshold condition.

10. The method according to claim 6, wherein determining the target matching result based on the first matching result and the second matching result includes: The first matching result and the second matching result are fused to determine the target matching result.

11. A training method for a trajectory prediction model, comprising: Obtain an augmented training dataset, wherein the augmented training dataset is obtained by the method described in any one of claims 1-10; Based on at least one first historical trajectory sample information in the enhanced training dataset, at least one first predicted trajectory information is obtained using the initial trajectory prediction model to be trained. A first loss value is determined based on the at least one first predicted trajectory information and the first future trajectory annotation information in the enhanced training dataset that corresponds to each of the at least one historical trajectory information. Based on the first loss value, the initial trajectory prediction model to be trained is trained; In response to the initial trajectory prediction model after training satisfying the preset training completion conditions, the initial trajectory prediction model after training is determined as the first trajectory prediction model.

12. The method according to claim 11, wherein, After determining the trained initial trajectory prediction model as the first trajectory prediction model, the method further includes: Acquire a training image set collected within a preset time period for a scene containing at least one dynamic obstacle; Based on the training image set, training trajectory information corresponding to each of the at least one dynamic obstacle is determined, wherein each training trajectory information includes a first sub-trajectory information corresponding to the earlier time period in the preset time period, and a second sub-trajectory information corresponding to the later time period in the preset time period, and the first sub-trajectory information and the second sub-trajectory information correspond one-to-one. The first sub-trajectory information of the at least one training trajectory information is used as the third historical sample trajectory information, and the first trajectory prediction model is used to obtain at least one second predicted trajectory information. The second sub-trajectory information of the at least one training trajectory information is used as the third future trajectory annotation information, and the second loss value is determined based on the at least one third future trajectory annotation information and at least one second predicted trajectory information; The first trajectory prediction model is trained based on the second loss value; In response to the first trajectory prediction model after training satisfying the preset training completion conditions, the first trajectory prediction model after training is determined as the second trajectory prediction model.

13. An apparatus for augmenting training data for trajectory prediction, comprising: The first acquisition module is used to acquire a set of visual images and a point cloud dataset collected within the same time period for a scene containing at least one dynamic obstacle; wherein, the time period includes an intermediate moment, historical time before the intermediate moment, and future time after the intermediate moment; The first trajectory information determination module is used to determine, based on the visual image set, first trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle, and based on the point cloud dataset, second trajectory information corresponding to each dynamic obstacle among the at least one dynamic obstacle; wherein, the first trajectory information includes first historical trajectory information corresponding to the intermediate time and the historical time, and first future trajectory information corresponding to the future time; the second trajectory information includes second historical trajectory information corresponding to the intermediate time and the historical time, and second future trajectory information corresponding to the future time; The first trajectory information matching module is used to match the first historical trajectory information of each first trajectory information with the second historical trajectory information of each second trajectory information to obtain a target matching result; wherein, the target matching result includes at least one pair of trajectory information that is successfully matched between the first trajectory information and the second trajectory information; The first augmented data construction module is used to construct first augmented training data for each pair of trajectory information in the at least one pair of trajectory information, using the first historical trajectory information of the first trajectory information as the first historical trajectory sample information and the second future trajectory information of the second trajectory information as the first future trajectory annotation information. An enhanced training dataset determination module is used to determine an enhanced training dataset based on the first enhanced training data corresponding to each pair of trajectory information in the at least one pair of trajectory information.

14. A training device for a trajectory prediction model, comprising: The second acquisition module is used to acquire an augmented training dataset, wherein the augmented training dataset is obtained by the method described in any one of claims 1-10; The first trajectory information prediction module is used to obtain at least one first predicted trajectory information based on at least one first historical trajectory sample information in the enhanced training dataset and using the initial trajectory prediction model to be trained. The first loss determination module is used to determine a first loss value based on the at least one first predicted trajectory information and the first future trajectory annotation information in the enhanced training dataset that corresponds to each of the at least one historical trajectory information. The first training module is used to train the initial trajectory prediction model to be trained based on the first loss value. The first model determination module is used to determine the trained initial trajectory prediction model as the first trajectory prediction model in response to the training completion condition being met by the trained initial trajectory prediction model.

15. A computer-readable storage medium storing a computer program for performing the method according to any one of claims 1-12.

16. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-12.

Images