Trajectory prediction method, apparatus, device, medium, and product

Abstract

The application discloses a trajectory prediction method, device, equipment, medium and product. The method comprises the following steps: obtaining historical state information and environment information of a target obstacle; generating comprehensive environment features based on the historical state information and the environment information; generating at least one predicted trajectory based on the comprehensive environment features; calculating environment constraint loss based on the at least one predicted trajectory and the environment information; training a neural network model through the environment constraint loss to obtain an obstacle trajectory prediction model, so as to perform prediction of a target obstacle trajectory through the obstacle trajectory prediction model. The application can automatically understand complex and diverse traffic scenes by training the neural network model through the environment constraint loss, thereby greatly expanding the coverable scene range of the model, and greatly improving the trajectory prediction effect of the obtained obstacle trajectory prediction model.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and more specifically, to a trajectory prediction method, apparatus, device, medium, and product. Background Technology

[0002] In the field of autonomous driving, accurately predicting the future trajectories of surrounding obstacles is crucial for achieving safe and reliable environmental perception and decision-making. Trajectory prediction needs to comprehensively consider the vehicle's own state, dynamic traffic participants, and static environmental information (such as lane lines, obstacles, and traffic signs) to generate a reasonable trajectory that conforms to traffic rules and physical laws. Especially in complex urban scenarios, obstacle trajectories not only need to meet kinematic constraints but also avoid conflicts with static map elements, stationary vehicles, solid lines, and hard barriers, which places higher demands on the accuracy and generalization ability of the prediction model.

[0003] Currently, mainstream obstacle trajectory prediction technologies are mainly divided into two categories: rule-based post-processing correction methods and neural network-based end-to-end learning methods. Rule-based methods typically generate an initial trajectory from a base model first, and then correct unreasonable trajectories (such as crossing solid lines, colliding with stationary vehicles, etc.) through a series of manually designed rules.

[0004] Currently, the above methods are not applicable to all scenarios. Summary of the Invention

[0005] The main purpose of this application is to provide a trajectory prediction method, device, equipment, medium and product that is not only applicable to various scenarios, but also improves the accuracy of trajectory prediction.

[0006] To achieve the above objectives, firstly, this application provides a trajectory prediction method, comprising: Acquire historical status information and environmental information of the target obstacle, wherein the environmental information includes at least one of static map elements, static vehicles, solid lane lines and hard road barriers; Based on historical state information and environmental information, comprehensive environmental characteristics are generated. Based on comprehensive environmental characteristics, at least one predicted trajectory is generated; Calculate the environmental constraint loss based on at least one predicted trajectory and environmental information; By training a neural network model using environmental constraint loss, an obstacle trajectory prediction model is obtained, which is then used to predict the trajectory of the target obstacle.

[0007] In one embodiment, based on historical state information and environmental information, a comprehensive environmental feature is generated, including: Environmental information is input into a multilayer perceptron network for feature extraction to obtain preliminary environmental features. By using an attention mechanism, preliminary environmental features are fused with historical state information to generate comprehensive environmental features.

[0008] In one embodiment, at least one predicted trajectory is generated based on comprehensive environmental features, including: Multilayer perceptron encoding and pooling are performed on static map elements, static vehicles, solid lane lines and hard road barriers respectively to obtain conditional feature vectors; The conditional feature vectors are concatenated to obtain the concatenated feature vector; The concatenated feature vector is input into a multilayer perceptron network to generate multiple sets of modulation parameters; The comprehensive environmental features are input into the decoder and guided by multiple sets of modulation parameters to generate at least one predicted trajectory.

[0009] In one embodiment, comprehensive environmental features are input into the decoder and guided by multiple sets of modulation parameters to generate at least one predicted trajectory, including: The comprehensive environmental features are input into the decoder. When generating each trajectory point, the decoder applies the modulation parameters from multiple sets of modulation parameters to adjust the parameters of the network layer and outputs multiple position sequences. Each position sequence in the multiple position sequences constitutes a predicted trajectory, and each position sequence consists of multiple trajectory points.

[0010] In one embodiment, based on at least one predicted trajectory and environmental information, the environmental constraint loss is calculated, including: Based on at least one predicted trajectory, the violation loss components for static map elements, static vehicles, solid lane lines, and hard road barriers are calculated respectively to obtain multiple violation loss components; The environmental constraint loss is obtained by weighting and summing multiple violation loss components according to preset weights.

[0011] In one embodiment, violation loss components are calculated based on at least one predicted trajectory for static map elements, static vehicles, solid lane lines, and hard road barriers, resulting in multiple violation loss components, including: For any environmental element among static map elements, static vehicles, solid lane lines, and hard road barriers, perform the following operations: Obtain at least one predicted trajectory; Calculate the geometric distance from each trajectory point in any of at least one predicted trajectory to any environmental element; Based on a preset safe distance threshold, identify violation trajectory points whose geometric distance is less than the safe distance threshold, and obtain all violation trajectory points corresponding to any environmental element; Calculate the violation loss component corresponding to any environmental element based on all violation trajectory points corresponding to any environmental element.

[0012] In one embodiment, calculating the geometric distance from each trajectory point in any of at least one predicted trajectory to any environmental element includes: When the environmental element is a solid lane line or a hard road barrier, the environmental element is represented as a continuous line segment, and the shortest distance from each trajectory point in any of the at least one predicted trajectory to the matching continuous line segment is calculated. Use the shortest distance as the geometric distance.

[0013] In one embodiment, calculating the geometric distance from each trajectory point in any of at least one predicted trajectory to any environmental element includes: When the environment element is a static map element or a static vehicle, the environment element is represented as a point, and the Euclidean distance from each trajectory point in any of the at least one predicted trajectory to the matching point is calculated. Use Euclidean distance as the geometric distance.

[0014] In one embodiment, based on all violation trajectory points corresponding to any environmental element, the violation loss component corresponding to any environmental element is calculated, including: For each violation trajectory point among all violation trajectory points corresponding to any environmental element, obtain the penalty function for each violation trajectory point; Calculate the penalty function value for each violation trajectory point using the penalty function; The penalty function values ​​of each violation trajectory point are aggregated to obtain the violation loss component corresponding to any environmental element.

[0015] Secondly, embodiments of this application provide a trajectory prediction device, comprising: The acquisition module is used to acquire historical state information and environmental information of the target obstacle. The environmental information includes at least one of static map elements, static vehicles, solid lane lines, and hard road barriers. The feature generation module is used to generate comprehensive environmental features based on historical state information and environmental information; The trajectory generation module is used to generate at least one predicted trajectory based on comprehensive environmental features; The loss calculation module is used to calculate the environmental constraint loss based on at least one predicted trajectory and environmental information. The trajectory prediction module is used to train a neural network model through environmental constraint loss to obtain an obstacle trajectory prediction model, which is then used to predict the trajectory of the target obstacle.

[0016] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0019] This application provides a trajectory prediction method, apparatus, device, medium, and product, comprising: first acquiring historical state information and environmental information of a target obstacle; then generating comprehensive environmental features based on the historical state information and environmental information; then generating at least one predicted trajectory based on the comprehensive environmental features; further, calculating environmental constraint loss based on the at least one predicted trajectory and environmental information; thereby training a neural network model through the environmental constraint loss to obtain an obstacle trajectory prediction model; and then performing target obstacle trajectory prediction through the obstacle trajectory prediction model. This application, by training a neural network model through environmental constraint loss, can automatically understand complex and diverse traffic scenarios, eliminating the need to manually formulate cumbersome correction rules for each specific scenario (such as detours, avoidance, and lane keeping), thereby greatly expanding the range of scenarios the model can cover and significantly improving the trajectory prediction performance of the acquired obstacle trajectory prediction model. Attached Figure Description

[0020] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 This is a schematic flowchart of a trajectory prediction method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating another trajectory prediction method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a method for generating a predicted trajectory provided in an embodiment of this application; Figure 4 These are schematic diagrams illustrating the predicted trajectories in different scenarios provided in the embodiments of this application; Figure 5 This is a schematic diagram of the predicted trajectory in a solid line scenario provided in the embodiments of this application; Figure 6This is a schematic diagram of the structure of a trajectory prediction device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0023] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0024] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0025] It should be understood that in this application, "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product or device.

[0026] It should be understood that in this application, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, "and / or B" can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0027] It should be understood that in this application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0028] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0029] The data involved in this application may be data authorized by the tester or fully authorized by all parties. The collection, dissemination, and use of the data shall comply with the relevant laws, regulations and standards of the relevant countries and regions. The implementation methods / executives of this application may be combined with each other.

[0030] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0031] Please see Figure 1 , Figure 1 This is a flowchart illustrating a trajectory prediction method provided in an embodiment of this application. Figure 1 As shown, it includes the following steps: Step S101: Obtain historical state information and environmental information of the target obstacle.

[0032] The environmental information includes at least one of static map elements, static vehicles, solid lane lines, and hard road barriers.

[0033] This embodiment is set in a typical urban road intersection scenario, where the target obstacle is a car traveling in front of the autonomous vehicle. First, the historical state information of the target obstacle is acquired using a multi-sensor fusion system (such as LiDAR, cameras, and millimeter-wave radar) and a vehicle self-localization module mounted on the autonomous vehicle. Specifically, the state data of the target vehicle for 30 time steps over the past 3 seconds is continuously recorded and cached at 100-millisecond intervals. The state information for each time step is a multi-dimensional vector, containing at least the center point coordinates, heading angle, velocity, and acceleration of the target vehicle in the Earth coordinate system or the vehicle's coordinate system at that moment. This series of time-ordered state vectors constitutes a temporal trajectory sequence for prediction.

[0034] Meanwhile, the acquisition of environmental information primarily relies on in-vehicle high-precision maps and real-time perception results. Static map elements are extracted from the high-precision map, including geometric and semantic information such as the boundary lines of the current lane and adjacent lanes, road edges, pedestrian crossings, stop lines, guide lines, and the projection areas of traffic signs on the ground. Static vehicles refer to traffic participants in the scene that are stationary or nearly stationary, excluding target obstacles, such as parked vehicles or broken-down vehicles. Their information (such as outlines, positions, and orientations) is detected and provided in real time by the perception system. Solid lane lines and hard road barriers, as key road constraint elements, are also accurately obtained from the high-precision map. Solid lane lines represent lane boundaries that prohibit lane changes, while hard road barriers represent insurmountable physical barriers such as curbs, central guardrails, or green belts. Both are provided as a series of continuous point sequences or line segments to define their precise geometry and position.

[0035] Step S102: Generate comprehensive environmental features based on historical state information and environmental information.

[0036] To generate comprehensive environmental features based on historical state information and environmental information, the environmental information must first be input into a multilayer perceptron network for feature extraction to obtain preliminary environmental features. Then, the preliminary environmental features are fused with historical state information through an attention mechanism to generate comprehensive environmental features.

[0037] Specifically, such as Figure 2As shown, a parallel multilayer perceptron (MLP) network is first used to extract fine-grained features from environmental information. Specifically, for static map elements (such as lane line geometry and traffic sign areas), their polygon vertex sequences are input into a dedicated three-layer MLP network, outputting a 128-dimensional semantic feature vector. For static vehicle information, the bounding box center coordinates, size, heading angle, and speed information of each vehicle are input into another independent three-layer MLP to generate a 128-dimensional feature vector representing its state and intention. Solid lane lines are represented as a continuous sequence of line segments, and their start-end coordinates and direction vectors are input into a third MLP, encoded as 128-dimensional linear constraint features. Similarly, road hard barriers (such as guardrails and curbs) are also mapped to 128-dimensional features through a fourth MLP network. The feature vectors of all environmental entities of the same category are aggregated through pooling layers, for example, max pooling is performed on the features of multiple static vehicles to obtain a unified 128-dimensional preliminary environmental feature vector for that category, thus forming four sets of structured environmental feature representations.

[0038] Subsequently, these heterogeneous preliminary environmental features are deeply fused with the historical state information of the target obstacle through an attention mechanism. The historical state information (including time series of position, velocity, and heading angle) is first converted into a 256-dimensional historical context vector by a temporal encoder (such as an LSTM or Transformer encoding layer). This vector is used as the attention query, and the four sets of preliminary environmental feature vectors are concatenated as keys and values ​​to calculate the relevance weights between the query and each environmental feature. This mechanism enables the model to dynamically focus on the environmental constraints most relevant to the current driving scenario, such as strengthening the attention to lane boundaries in a curve scenario and focusing on avoiding static vehicles in a congested scenario. Finally, the weighted fused environmental features are concatenated with the historical context vector, and a fully connected layer is used for dimensional integration and information compression to output a 512-dimensional comprehensive environmental feature vector.

[0039] Step S103: Generate at least one predicted trajectory based on comprehensive environmental features.

[0040] Based on comprehensive environmental features, at least one predicted trajectory is generated. This requires multilayer perceptron encoding and pooling of static map elements, static vehicles, solid lane lines, and hard road barriers to obtain conditional feature vectors. These conditional feature vectors are then concatenated to obtain a concatenated feature vector. This concatenated feature vector is then input into a multilayer perceptron network to generate multiple sets of modulation parameters. Finally, the comprehensive environmental features are input into a decoder, which is guided by the multiple sets of modulation parameters to generate at least one predicted trajectory.

[0041] The process involves inputting comprehensive environmental features into the decoder and guiding it with multiple sets of modulation parameters to generate at least one predicted trajectory. This includes: inputting comprehensive environmental features into the decoder; when generating each trajectory point, the decoder applies modulation parameters from multiple sets of modulation parameters to adjust the parameters of the network layer and outputs multiple position sequences. Each position sequence in the multiple position sequences constitutes a predicted trajectory, and each position sequence consists of multiple trajectory points.

[0042] Specifically, such as Figure 3 As shown, four key environmental information categories—static map elements, static vehicles, solid lane lines, and hard road barriers—are independently conditionally encoded. For each category, a three-layer multilayer perceptron (MLP) network is used to map its original geometric and semantic representations into a high-dimensional feature vector. Subsequently, max pooling is used to aggregate the feature vectors of all instances in this category into a unified 128-dimensional conditional feature vector, thus obtaining four sets of conditional features representing map structure, stationary obstacles, lane rules, and physical barriers, respectively.

[0043] Next, these four sets of conditional feature vectors are concatenated to form a 512-dimensional composite conditional feature vector. This vector is then input into a specially designed parameter-generating MLP network, which consists of two fully connected layers and ultimately outputs multiple sets (e.g., 12 sets) of modulation parameters. These modulation parameters are designed as control signals that can dynamically adjust the behavior of the neural network during decoding, such as scaling factors and bias terms.

[0044] Subsequently, the core process of trajectory generation is executed by a decoder based on a recurrent neural network (RNN). The decoder's initial state is initialized by the comprehensive environmental features obtained in step S102. At each time step of decoding (e.g., predicting the next 3 seconds, with 0.1-second intervals for a total of 30 time steps), the decoder unit (such as LSTM or GRU) calculates the position offset of the next trajectory point. Crucially, in each calculation step, the system selects a specific subset of modulation parameters corresponding to this time step (e.g., each set of parameters corresponds to a decoding layer or an attention head) from multiple sets of modulation parameters and incorporates these parameters into the calculation of the decoder's hidden state or the transformation of the output layer by element-wise multiplication (scaling) or addition (biasing). This modulation mechanism allows environmental constraints to influence the trajectory generation direction in real time and with fine granularity. For example, when the predicted trajectory is close to the solid line, the corresponding modulation parameters suppress lateral movement, thereby guiding the trajectory to stay within the lane.

[0045] Ultimately, the decoder outputs 30 consecutive future position coordinates, forming a complete predicted trajectory. By changing the initial state of the decoder or using different combinations of modulation parameters (e.g., introducing randomness through a conditional variational autoencoder), the system can generate multiple (e.g., 6) candidate predicted trajectories in parallel, representing different driving intentions (e.g., going straight, changing lanes to the left, or decelerating and following). Each trajectory consists of a time step sequence and is output as a two-dimensional coordinate sequence.

[0046] Step S104: Calculate the environmental constraint loss based on at least one predicted trajectory and environmental information.

[0047] To calculate the environmental constraint loss based on at least one predicted trajectory and environmental information, it is necessary to first calculate the violation loss components of static map elements, static vehicles, solid lane lines and hard road barriers based on at least one predicted trajectory to obtain multiple violation loss components. Then, the multiple violation loss components are weighted and summed according to preset weights to obtain the environmental constraint loss.

[0048] Specifically, based on at least one predicted trajectory, violation loss components for static map elements, static vehicles, solid lane lines, and hard road barriers are calculated respectively, resulting in multiple violation loss components. This includes: for any environmental element among static map elements, static vehicles, solid lane lines, and hard road barriers, the following operations are performed: obtaining at least one predicted trajectory; calculating the geometric distance from each trajectory point in any of the at least one predicted trajectory to any environmental element; identifying violation trajectory points whose geometric distance is less than a preset safe distance threshold according to a preset safe distance threshold, thus obtaining all violation trajectory points corresponding to any environmental element; and calculating the violation loss component corresponding to any environmental element based on all violation trajectory points corresponding to any environmental element.

[0049] The calculation of the geometric distance from each trajectory point in any of the at least one predicted trajectory to any environmental element includes: when the environmental element is a solid lane line or a hard road barrier, the environmental element is represented as a continuous line segment, and the shortest distance from each trajectory point in any of the at least one predicted trajectory to the matching continuous line segment is calculated; the shortest distance is used as the geometric distance.

[0050] The calculation of the geometric distance from each trajectory point in any of the at least one predicted trajectory to any environmental element includes: when the environmental element is a static map element or a static vehicle, the environmental element is represented as a point, and the Euclidean distance from each trajectory point in any of the at least one predicted trajectory to the matching point is calculated; the Euclidean distance is used as the geometric distance.

[0051] Specifically, the calculation of the violation loss component corresponding to any environmental element based on all violation trajectory points corresponding to any environmental element includes: obtaining the penalty function for each violation trajectory point among all violation trajectory points corresponding to any environmental element; calculating the penalty function value of each violation trajectory point using the penalty function; and aggregating the penalty function values ​​of each violation trajectory point to obtain the violation loss component corresponding to any environmental element.

[0052] Specifically, for each type of environmental element, at least one predicted trajectory (e.g., 6 candidate trajectories) generated in step S103 is traversed. For a single predicted trajectory, the geometric distance from each of its future trajectory points (30 points in total) to all relevant instances in that type of environment is calculated. The specific distance calculation method varies depending on the element type: For solid lane lines and hard road separation, the point sequence obtained from the high-precision map is represented as a series of continuous line segments, and the shortest distance from the trajectory point to each line segment is calculated (e.g., the perpendicular distance from the point to the line segment is calculated using a vector projection method; if the foot of the perpendicular falls outside the line segment, the distance to the endpoint is calculated), and the minimum value among the shortest distances calculated for all line segments is taken as the final geometric distance from the trajectory point to this environmental element. For static map elements (such as traffic islands) and static vehicles, they are simplified to representative points (such as centroids or nearest points), and the Euclidean distance from the trajectory point to that point is directly calculated as the geometric distance.

[0053] Next, violations are determined based on preset safety distance thresholds related to the element type. For example, the safety distance threshold between a vehicle and a stationary vehicle is set to 0.5 meters, and the safety distance threshold between a vehicle and a solid lane line is set to 0.2 meters. If the geometric distance of a trajectory point is less than the corresponding safety threshold, that point is marked as a violation trajectory point for that type of environmental element.

[0054] For each type of environmental element, after identifying all violation trajectory points, its violation loss component is calculated. This calculation applies a penalty function to each violation point; this embodiment uses an exponential penalty function: Penalty value = exp(-λ * actual distance / safe distance) Where λ is the sensitivity coefficient.

[0055] This function ensures that the smaller the distance, the more exponentially the penalty increases, thus strongly deterring serious violations.

[0056] Subsequently, the penalty values ​​of all violation points generated by this type of element on all predicted trajectories are averaged to obtain the violation loss component of this type of element, such as loss_static_vehicle.

[0057] Finally, the four loss components are weighted and summed according to preset importance weights to obtain the final environmental constraint loss (L_constraint). The weight configuration can be set based on prior knowledge, for example: static map element weight 0.2, static vehicle weight 0.3, solid lane line weight 0.1, and road hard barrier weight 0.3.

[0058] The calculation formula is: L_constraint=0.2*loss_map+0.3*loss_vehicle+0.1*loss_solid_line+0.3*loss_curb This total loss, along with other loss terms such as trajectory regression loss, will participate in the training of the neural network model. Through backpropagation, the model parameters will be adjusted, driving the model to generate more reasonable trajectories that conform to environmental physical constraints and traffic rules in the future.

[0059] Step S105: Train the neural network model using environmental constraint loss to obtain an obstacle trajectory prediction model, and then use the obstacle trajectory prediction model to predict the trajectory of the target obstacle.

[0060] Specifically, the environmental constraint loss (L_constraint) calculated in step S104 is combined with the regression loss of the trajectory prediction task itself (L_regression, such as using the Huber loss function to measure the deviation between the predicted trajectory points and the actual future trajectory points) to form the overall objective function for model optimization: L_total = L_regression +β* L_constraint Here, β is a hyperparameter that balances the importance of the two losses, used to adjust the tradeoff between fitting historical data and adhering to environmental rules.

[0061] During the training phase, a large amount of driving datasets containing rich scenarios (data samples include historical trajectories of target obstacles, environmental information, and corresponding real future trajectories) are used. Following the forward computation process described in steps S101 to S104 above, the total loss L_total is obtained. Subsequently, using the backpropagation algorithm and the Adam optimizer, all parameters in the network are iteratively updated, including the weights of each MLP encoder, attention fusion module, parameter generation MLP, and trajectory decoder. This process enables the neural network not only to learn the mapping rules from historical states to future motion, but also to internalize a deep understanding of constraints such as static maps, vehicles, lane lines, and medians, thereby embedding traffic rules and physical feasibility at the parameter level.

[0062] After training, a deployable obstacle trajectory prediction model is obtained. In practical inference applications, this model operates as a complete forward computation system: First, the perception and localization system provides the historical state sequence of the target obstacle and the current environmental information in real time (step S101); then, the encoding and fusion module within the model automatically generates comprehensive environmental features (step S102); next, a soft constraint mechanism is triggered, generating modulation parameters based on the environmental information and guiding the decoder to generate multiple reasonable candidate prediction trajectories (step S103). The key difference is that loss calculation (S104) and parameter updates are no longer performed during the inference stage. The model outputs the finally generated multiple candidate trajectories and their confidence scores to the downstream planning and control module.

[0063] The planning module can select an optimal trajectory based on the highest confidence level or risk utility assessment, such as... Figure 4 As shown, in the three scenarios of circling around static map elements, circling around stationary vehicles, and crossing hard barriers, the line with arrows on the left represents the optimal trajectory for each scenario. Figure 5 When crossing the solid line, the line with the arrow in the middle represents the optimal trajectory.

[0064] This well-trained model, through explicit training with environmental constraint loss, ensures the rationality and safety of its trajectory generation in unknown scenarios, ultimately achieving accurate, reliable, and compliant prediction of the future movement of target obstacles on an embedded platform.

[0065] This application provides a trajectory prediction method, comprising: first, acquiring historical state information and environmental information of a target obstacle; then, generating comprehensive environmental features based on the historical state information and environmental information; further, generating at least one predicted trajectory based on the comprehensive environmental features; and then calculating environmental constraint loss based on the at least one predicted trajectory and environmental information. A neural network model is then trained using the environmental constraint loss to obtain an obstacle trajectory prediction model, which is used to predict the trajectory of the target obstacle. This application trains a neural network model using environmental constraint loss, enabling it to automatically understand complex and diverse traffic scenarios without requiring the manual formulation of cumbersome correction rules for each specific scenario (such as detours, avoidance, and lane keeping). This significantly expands the range of scenarios the model can cover, greatly improving the trajectory prediction performance of the acquired obstacle trajectory prediction model.

[0066] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0067] The following are device embodiments of this application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0068] Figure 6 The diagram illustrates the structure of a trajectory prediction device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. The trajectory prediction device includes an acquisition module 601, a feature generation module 602, a trajectory generation module 603, a loss calculation module 604, and a trajectory prediction module 605, as detailed below: The acquisition module 601 is used to acquire historical state information and environmental information of the target obstacle, wherein the environmental information includes at least one of static map elements, static vehicles, solid lane lines and hard road barriers; The feature generation module 602 is used to generate comprehensive environmental features based on historical state information and environmental information; The trajectory generation module 603 is used to generate at least one predicted trajectory based on comprehensive environmental features; The loss calculation module 604 is used to calculate the environmental constraint loss based on at least one predicted trajectory and environmental information; The trajectory prediction module 605 is used to train a neural network model through environmental constraint loss to obtain an obstacle trajectory prediction model, so as to perform the prediction of the target obstacle trajectory through the obstacle trajectory prediction model.

[0069] In one embodiment, the feature generation module 602 is further configured to input environmental information into a multilayer perceptron network for feature extraction to obtain preliminary environmental features; By using an attention mechanism, preliminary environmental features are fused with historical state information to generate comprehensive environmental features.

[0070] In one embodiment, the trajectory generation module 603 is further configured to perform multi-layer perceptron encoding and pooling on static map elements, static vehicles, solid lane lines and hard road barriers respectively to obtain conditional feature vectors; The conditional feature vectors are concatenated to obtain the concatenated feature vector; The concatenated feature vector is input into a multilayer perceptron network to generate multiple sets of modulation parameters; The comprehensive environmental features are input into the decoder and guided by multiple sets of modulation parameters to generate at least one predicted trajectory.

[0071] In one embodiment, the trajectory generation module 603 is further configured to input comprehensive environmental features into the decoder. When generating each trajectory point, the decoder applies modulation parameters from multiple sets of modulation parameters to adjust the parameters of the network layer and outputs multiple position sequences. Each position sequence in the multiple position sequences constitutes a predicted trajectory, and each position sequence consists of multiple trajectory points.

[0072] In one embodiment, the loss calculation module 604 is further configured to calculate the violation loss components of static map elements, static vehicles, solid lane lines and hard road barriers based on at least one predicted trajectory, thereby obtaining multiple violation loss components. The environmental constraint loss is obtained by weighting and summing multiple violation loss components according to preset weights.

[0073] In one embodiment, the loss calculation module 604 is further configured to perform the following operations for any of the environmental elements among static map elements, static vehicles, solid lane lines, and hard road barriers: Obtain at least one predicted trajectory; Calculate the geometric distance from each trajectory point in any of at least one predicted trajectory to any environmental element; Based on a preset safe distance threshold, identify violation trajectory points whose geometric distance is less than the safe distance threshold, and obtain all violation trajectory points corresponding to any environmental element; Calculate the violation loss component corresponding to any environmental element based on all violation trajectory points corresponding to any environmental element.

[0074] In one embodiment, the loss calculation module 604 is further configured to characterize the environmental element as a continuous line segment when the environmental element is a solid lane line or a hard road barrier, and to calculate the shortest distance from each trajectory point in any of the at least one predicted trajectory to the matching continuous line segment. Use the shortest distance as the geometric distance.

[0075] In one embodiment, the loss calculation module 604 is further configured to, when the environmental element is a static map element or a static vehicle, represent the environmental element as a point and calculate the Euclidean distance from each trajectory point in any of the at least one predicted trajectory to the matching point. Use Euclidean distance as the geometric distance.

[0076] In one embodiment, the loss calculation module 604 is further configured to obtain a penalty function for each violation trajectory point among all violation trajectory points corresponding to any environmental element; Calculate the penalty function value for each violation trajectory point using the penalty function; The penalty function values ​​of each violation trajectory point are aggregated to obtain the violation loss component corresponding to any environmental element.

[0077] This application provides a trajectory prediction device, specifically used for: first acquiring historical state information and environmental information of a target obstacle; then generating comprehensive environmental features based on the historical state information and environmental information; generating at least one predicted trajectory based on the comprehensive environmental features; and then calculating environmental constraint loss based on the at least one predicted trajectory and environmental information. The device then trains a neural network model using the environmental constraint loss to obtain an obstacle trajectory prediction model, which is used to predict the trajectory of the target obstacle. This application trains a neural network model using environmental constraint loss, enabling it to automatically understand complex and diverse traffic scenarios without requiring the manual formulation of cumbersome correction rules for each specific scenario (such as detours, avoidance, and lane keeping). This significantly expands the range of scenarios the model can cover, greatly improving the trajectory prediction performance of the acquired obstacle trajectory prediction model.

[0078] This application Figure 7 A schematic diagram of a computer device is provided. (Example) Figure 7 As shown, the computer device 7 in this embodiment includes a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program 703, it implements the steps in the various trajectory prediction method embodiments described above, for example... Figure 1 Steps 101 to 105 are shown. Alternatively, when processor 701 executes computer program 703, it implements the functions of each module / unit in the above-described trajectory prediction device embodiments, for example... Figure 6 The functions of modules / units 601 to 605 shown.

[0079] This application also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the trajectory prediction methods provided in the various embodiments described above.

[0080] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0081] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the execution of the executable instructions by the at least one processor causes the device to implement the trajectory prediction methods provided in the various embodiments described above.

[0082] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0083] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A trajectory prediction method, characterized in that, include: The historical state information and environmental information of the target obstacle are obtained, wherein the environmental information includes at least one of static map elements, static vehicles, solid lane lines and hard road barriers; Based on the historical state information and the environmental information, a comprehensive environmental feature is generated; Based on the comprehensive environmental characteristics, at least one predicted trajectory is generated; Based on the at least one predicted trajectory and the environmental information, calculate the environmental constraint loss; The neural network model is trained using the environmental constraint loss to obtain an obstacle trajectory prediction model, which is then used to predict the trajectory of the target obstacle.

2. The trajectory prediction method as described in claim 1, characterized in that, The generation of comprehensive environmental features based on the historical state information and the environmental information includes: The environmental information is input into a multilayer perceptron network for feature extraction to obtain preliminary environmental features. The preliminary environmental features are fused with the historical state information through an attention mechanism to generate the comprehensive environmental features.

3. The trajectory prediction method as described in claim 1, characterized in that, The generation of at least one predicted trajectory based on the comprehensive environmental features includes: Multilayer perceptron encoding and pooling are performed on the static map elements, the static vehicles, the solid lane lines, and the hard road barriers to obtain conditional feature vectors. The conditional feature vectors are concatenated to obtain the concatenated feature vector; The concatenated feature vector is input into a multilayer perceptron network to generate multiple sets of modulation parameters; The comprehensive environmental features are input into the decoder and guided by the multiple sets of modulation parameters to generate at least one predicted trajectory.

4. The trajectory prediction method as described in claim 3, characterized in that, The step of inputting the comprehensive environmental features into the decoder and generating the at least one predicted trajectory under the guidance of the multiple sets of modulation parameters includes: The comprehensive environmental features are input into the decoder. When generating each trajectory point, the decoder applies the modulation parameters from the multiple sets of modulation parameters to adjust the parameters of the network layer and outputs multiple position sequences. Each of the multiple position sequences constitutes a predicted trajectory, and each position sequence consists of multiple trajectory points.

5. The trajectory prediction method as described in claim 1, characterized in that, The calculation of environmental constraint loss based on the at least one predicted trajectory and the environmental information includes: Based on the at least one predicted trajectory, the violation loss components of the static map elements, the static vehicles, the solid lane lines, and the hard road barriers are calculated respectively to obtain multiple violation loss components; The environmental constraint loss is obtained by weighting and summing the multiple violation loss components according to preset weights.

6. The trajectory prediction method as described in claim 5, characterized in that, The violation loss components are calculated based on the at least one predicted trajectory for the static map elements, the static vehicles, the solid lane lines, and the hard road barriers, resulting in multiple violation loss components, including: For any of the environmental elements among the static map elements, the static vehicles, the solid lane lines, and the hard road barriers, perform the following operations: Obtain at least one predicted trajectory; Calculate the geometric distance from each trajectory point in any of the at least one predicted trajectory to any of the environmental elements; Based on a preset safe distance threshold, identify violation trajectory points whose geometric distance is less than the safe distance threshold, and obtain all violation trajectory points corresponding to any environmental element. Based on all violation trajectory points corresponding to any one of the environmental elements, calculate the violation loss component corresponding to any one of the environmental elements.

7. The trajectory prediction method as described in claim 6, characterized in that, The calculation of the geometric distance from each trajectory point in any of the at least one predicted trajectory to any environmental element includes: When the environmental element is a solid lane line or a hard road barrier, the environmental element is represented as a continuous line segment, and the shortest distance from each trajectory point in any of the at least one predicted trajectory to the matching continuous line segment is calculated. The shortest distance is taken as the geometric distance.

8. The trajectory prediction method as described in claim 6, characterized in that, The calculation of the geometric distance from each trajectory point in any of the at least one predicted trajectory to any environmental element includes: When the environmental element is a static map element or a static vehicle, the environmental element is represented as a point, and the Euclidean distance from each trajectory point in any of the at least one predicted trajectory to the matching point is calculated. The Euclidean distance is used as the geometric distance.

9. The trajectory prediction method as described in claim 6, characterized in that, The step of calculating the violation loss component corresponding to any environmental element based on all violation trajectory points corresponding to any environmental element includes: For each violation trajectory point among all violation trajectory points corresponding to any environmental element, obtain the penalty function for each violation trajectory point; The penalty function is used to calculate the penalty function value for each violation trajectory point; The penalty function values ​​of each violation trajectory point are aggregated to obtain the violation loss component corresponding to any environmental element.

10. A trajectory prediction device, characterized in that, include: The acquisition module is used to acquire historical state information and environmental information of the target obstacle, wherein the environmental information includes at least one of static map elements, static vehicles, solid lane lines, and hard road barriers; The feature generation module is used to generate comprehensive environmental features based on the historical state information and the environmental information; The trajectory generation module is used to generate at least one predicted trajectory based on the comprehensive environmental features. The loss calculation module is used to calculate the environmental constraint loss based on the at least one predicted trajectory and the environmental information; The trajectory prediction module is used to train a neural network model through the environmental constraint loss to obtain an obstacle trajectory prediction model, so as to perform the prediction of the trajectory of the target obstacle through the obstacle trajectory prediction model.

11. A computer device, characterized in that, Includes a memory, and one or more processors communicatively connected to the memory; The memory stores instructions that can be executed by the one or more processors to cause the one or more processors to implement the trajectory prediction method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, Includes a program or instructions that, when run on a computer, implement the trajectory prediction method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the trajectory prediction method according to any one of claims 1 to 9.

Images