Rule and model combined end-to-end trajectory generation method and device

Abstract

The invention relates to the technical field of trajectory planning, in particular to a rule and model combined end-to-end trajectory generation method and device, and the method comprises the steps: generating a plurality of candidate transverse paths of a vehicle based on a rule; acquiring environment information; generating a space-time interaction feature by using a first cross attention mechanism; generating a longitudinal behavior query vector based on a data-driven longitudinal behavior model; generating a scenarized longitudinal behavior query vector by using a second cross attention mechanism; and generating a final trajectory planning result by using a third cross attention mechanism. According to the invention, the candidate path generated by the rule is combined with the data driving model; the security compliance of the basic path is ensured through the rule, and the intelligent coping capability in an unknown or complex scene is improved by using the model, so that the overall performance and reliability of the system are remarkably enhanced while the code complexity is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of trajectory planning technology, specifically to an end-to-end trajectory generation method and apparatus that combines rules and models. Background Technology

[0002] In autonomous driving technology, trajectory planning is a core element that determines how a vehicle can drive safely and comfortably. Especially in complex urban scenarios with dense targets and frequent interactions, it places extremely high demands on the robustness and adaptability of the technology.

[0003] Currently, mainstream solutions fall into two main categories: the first is trajectory planning methods based on pre-defined rules, which use numerous "if-else" statements to handle different scenarios. However, as scenarios become more segmented, the rule base becomes increasingly bloated, leading to difficult code maintenance and susceptibility to failure in unknown and complex scenarios. The second is data-driven end-to-end model methods, which, while possessing strong generalization capabilities, suffer from insufficient understanding of physical indicators such as trajectory safety and comfort, posing inherent safety risks and heavily relying on massive amounts of high-quality labeled data. Therefore, designing a trajectory generation method that can flexibly handle complex scenarios while strictly ensuring safety and comfort, thereby significantly improving the passability, safety, and reliability of autonomous driving systems in complex urban environments, has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, this disclosure provides an end-to-end trajectory generation method and apparatus that combines rules and models to solve the problem of how to design a trajectory generation method that can flexibly cope with complex scenarios while strictly ensuring safety and comfort, thereby significantly improving the passability, safety and reliability of autonomous driving systems in complex urban environments.

[0005] This disclosure provides an end-to-end trajectory generation method that combines rules and models, the method including: Multiple candidate lateral paths for the vehicle are generated based on rules; Acquire environmental information, which includes map information and status information of other traffic participants; Environmental information is interactively encoded to obtain traffic participant behavior query vectors; multiple candidate lateral paths are feature-encoded to obtain path query vectors. By utilizing the first cross-attention mechanism, the traffic participant behavior query vector and the path query vector are fused to generate spatial-temporal interaction features; Generate vertical behavior query vectors based on a data-driven vertical behavior model; By utilizing the second cross-attention mechanism, the vertical behavior query vector and the path query vector are fused to generate a contextualized vertical behavior query vector; By utilizing the third cross-attention mechanism, the contextualized vertical behavior query vector is fused with spatial-temporal interaction features to generate the final trajectory planning result.

[0006] This disclosure also provides an end-to-end trajectory generation apparatus that combines rules and models, the apparatus comprising: The lateral path generation module is used to generate multiple candidate lateral paths for the vehicle based on rules; The environmental information acquisition module is used to acquire environmental information, including map information and status information of other traffic participants. The environmental information encoding module is used to interactively encode environmental information to obtain traffic participant behavior query vectors; and to perform feature encoding on multiple candidate lateral paths to obtain path query vectors. The first cross-fusion module is used to fuse traffic participant behavior query vectors and path query vectors using the first cross-attention mechanism to generate spatial-temporal interaction features. The vertical behavior generation module is used to generate vertical behavior query vectors based on a data-driven vertical behavior model. The second cross-fusion module is used to fuse the vertical behavior query vector and the path query vector using the second cross-attention mechanism to generate a contextualized vertical behavior query vector. The final trajectory generation module utilizes a third cross-attention mechanism to fuse the contextualized longitudinal behavior query vector with spatial-temporal interaction features to generate the final trajectory planning result.

[0007] This disclosure also provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the aforementioned end-to-end trajectory generation method that combines rules and models.

[0008] This disclosure also provides a computer-readable storage medium storing computer instructions for enabling a computer to implement the aforementioned end-to-end trajectory generation method that combines rules and models.

[0009] This disclosure also provides a computer program product, including computer instructions for causing a computer to execute the end-to-end trajectory generation method combining the aforementioned rules and models.

[0010] The end-to-end trajectory generation method and apparatus of the above embodiments of this disclosure, which combines rules and models, integrates rule-generated interpretable and highly secure candidate paths with the powerful learning and generalization capabilities of data-driven models. Rules ensure the safety and compliance of the basic path, while models enhance intelligent response capabilities in unknown or complex scenarios, thereby significantly improving the overall performance and reliability of the system while reducing code complexity. Through a multi-layered cross-attention interaction mechanism, the final trajectory is not merely a simple superposition of lateral and longitudinal behaviors, but an intelligent decision based on in-depth verification and fusion of dynamic environmental information and path geometric information. This mechanism ensures a high degree of synergy between longitudinal acceleration / deceleration and lateral path selection, significantly improving the safety and rationality of trajectory planning.

[0011] Furthermore, because the model deeply integrates the prediction and interaction of multimodal behaviors of surrounding traffic participants during the planning process, the generated trajectory can be highly adapted to typical complex scenarios such as narrow-gap passage and continuous cutting in by multiple vehicles, thereby greatly enhancing the passability and interaction capabilities in complex traffic scenarios. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating an end-to-end trajectory generation method combining rules and models provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the specific process of an end-to-end trajectory generation method combining rules and models provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the architecture of an end-to-end trajectory generation device that combines rules and models, provided in an embodiment of this disclosure. Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0015] As autonomous driving technology continues to penetrate complex urban environments, extremely high demands are placed on vehicles' real-time, safe, and comfortable trajectory planning capabilities in dense, dynamic environments. To achieve intelligent navigation, related technologies require the use of onboard sensors to acquire environmental information such as maps, lane lines, and traffic participants. Based on this, behavioral decision-making and trajectory generation algorithms are employed to ultimately output an executable motion trajectory.

[0016] In practical applications, high-quality trajectory planning heavily relies on the depth of the planning method's understanding of the environment and the rationality of its decisions. Currently, mainstream planning paradigms are mainly divided into rule-based optimization methods and data-driven end-to-end learning methods. While both types of methods can be effective in specific scenarios, they often have the following inherent limitations when dealing with real-world, complex urban driving tasks: 1. Traditional rule-based approaches heavily rely on a large number of pre-defined "if-else" logic statements or cost functions to address different scenarios. As urban scenarios become increasingly segmented, the rule base becomes bloated, and the code becomes difficult to maintain. More importantly, this method struggles to enumerate all scenarios. When faced with unknown or undefined scenarios, its planning performance drops sharply, potentially leading to system failure, severely limiting the generalization ability of autonomous driving systems in real-world open environments.

[0017] 2. While pure end-to-end data-driven methods achieve strong scenario generalization capabilities through training on large amounts of data, the model, as a "black box," struggles to explain and constrain the inherent logic of key physical indicators such as trajectory safety, comfort, and traffic rule compliance. This leads to generated trajectories potentially containing dangerous maneuvers and posing potential safety hazards. Furthermore, the model's performance is highly dependent on massive amounts of high-quality training data, resulting in high development costs and difficulty in guaranteeing reliability across all edge scenarios.

[0018] 3. Related technologies typically involve simply linking rules and models together or only supplementing them locally, failing to form an organic whole. This loose combination cannot achieve the complementary advantages between the controllability of rules and the generalization ability of models. Rule-based systems struggle to utilize the inference capabilities of models, while model decisions lack robust safety boundary guidance. This results in generated trajectories that may be neither intelligent nor safe in complex interactive scenarios, failing to meet the increasingly stringent performance and reliability requirements of modern autonomous driving systems and hindering their large-scale commercial application.

[0019] To address the aforementioned issues, various embodiments of this disclosure provide an end-to-end trajectory generation method combining rules and models. The method includes: generating multiple candidate lateral paths for a vehicle based on rules; acquiring environmental information, including map information and status information of other traffic participants; interactively encoding the environmental information to obtain traffic participant behavior query vectors; performing feature encoding on the multiple candidate lateral paths to obtain path query vectors; using a first cross-attention mechanism to fuse the traffic participant behavior query vectors and path query vectors to generate spatial-temporal interaction features; generating a longitudinal behavior query vector based on a data-driven longitudinal behavior model; using a second cross-attention mechanism to fuse the longitudinal behavior query vectors and path query vectors to generate a contextualized longitudinal behavior query vector; and using a third cross-attention mechanism to fuse the contextualized longitudinal behavior query vector with the spatial-temporal interaction features to generate the final trajectory planning result.

[0020] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an end-to-end trajectory generation method combining rules and models provided in this disclosure. The method may include the following steps: Step S101: Generate multiple candidate lateral paths for the vehicle based on the rules.

[0021] In this embodiment, these candidate lateral paths generated by preset rules (such as multinomial programming) constitute the set of ego paths for subsequent processing (also known as ego-paths).

[0022] Rule-based generation can refer to a method that systematically generates a set of paths based on deterministic logic or mathematical constraints pre-defined by engineers, without relying on data-driven model learning and / or inference.

[0023] Here, the advantages of rule-based generation can include process transparency, controllable behavior, and computational efficiency.

[0024] The term "autonomous vehicle" can refer to the intelligent vehicle served in this embodiment that requires automated trajectory planning.

[0025] Candidate lateral paths can refer to a series of continuous and smooth position sequences that a vehicle may take in the lateral direction of the road over a period of time. Specifically, "candidate" can refer to multiple geometrically feasible lateral paths generated based on rules in this step, to construct a path strategy library covering various possible driving behaviors (such as lane keeping, fine-tuning to the left or right).

[0026] For example, the rules followed in generating multiple candidate lateral paths may include, but are not limited to: spatial sampling rules based on lane geometry, smoothness constraint rules based on vehicle dynamics, etc.

[0027] Step S102: Obtain environmental information, which includes map information and status information of other traffic participants.

[0028] In this embodiment, environmental information can refer to the collection of static and dynamic data about the driving scene around the vehicle that the vehicle needs to perceive in order to perform trajectory planning.

[0029] Map information, also known as static environmental information, refers to prior knowledge of road structures that do not change frequently or change slowly over time. Map information defines the legal space for vehicles to travel and traffic rule constraints, providing a basic framework for trajectory planning.

[0030] For example, map information may include, but is not limited to: the geometric topology of lanes (such as lane center lines and boundary lines), and the semantic information of traffic signs and markings (such as speed limits and stop lines).

[0031] The status information of other traffic participants, also known as dynamic environment information, refers to the real-time movement status of other vehicles, pedestrians, non-motorized vehicles, and other objects around the white vehicle. This information can describe the instantaneous situation of a dynamic scene.

[0032] For example, state information may include, but is not limited to, core parameters used to describe the motion state of traffic participants, such as their position, speed, and heading angle.

[0033] Step S103: Interactively encode the environmental information to obtain the traffic participant behavior query vector; perform feature encoding on multiple candidate lateral paths to obtain the path query vector.

[0034] In this embodiment, the purpose of this step may be to convert the heterogeneous raw information obtained in the above steps into a unified, high-dimensional feature representation that can be effectively processed by subsequent deep learning models and used for deep interaction.

[0035] Interactive encoding of environmental information can refer to using a data-driven model to fuse and reason about static map information and dynamic traffic participant status information in order to extract high-dimensional features that are rich in interactive semantics, beyond the original data itself.

[0036] The process aims to understand the interactions between traffic participants and between traffic participants and the map environment, and to predict their possible future behaviors.

[0037] Furthermore, the traffic participant behavior query vector (also known as the Agent-Motion Query), as one of the outputs of this step, can be one or a set of high-dimensional feature vectors. Here, this vector encapsulates the encoded environmental interaction semantics, and its representation can include, but is not limited to, an implicit expression of the possible multimodal motion trajectories of surrounding traffic participants (also known as Agents) in the future and their probability of occurrence.

[0038] The traffic participant behavior query vector will serve as the query representing the environmental intent in subsequent steps.

[0039] Furthermore, feature encoding of multiple candidate lateral paths can refer to converting the geometry-based path data generated in step S101 into high-dimensional features that can characterize their intrinsic attributes and driving strategies through a model.

[0040] The path query vector (also known as the Ego-path Query), as another output of this step, can be one or a set of high-dimensional feature vectors. Here, the vector encapsulates the abstract features of multiple candidate lateral paths to form a path feature library representing the possible behaviors of the vehicle.

[0041] The path query vector will serve as the knowledge base (Key and Value) for subsequent queries.

[0042] Step S104: Using the first cross-attention mechanism, the traffic participant behavior query vector and the path query vector are fused to generate spatial-temporal interaction features.

[0043] In this embodiment, this step achieves feature-level fusion between traffic participant behavior information and vehicle path information through a first cross-attention mechanism.

[0044] The first cross-attention mechanism can be a standard deep learning operator in which the traffic participant behavior query vector is defined as the query input and the path query vector is defined as both the key and value input.

[0045] The computation process of the first cross-attention mechanism follows the standard cross-attention procedure and may include the following steps: Calculate the similarity between the query and the key, and generate an attention weight matrix; The values ​​are then summed using this weight matrix. Output the new feature vector obtained after weighted summation.

[0046] In the specific application of this embodiment, the physical meaning of the above calculation process can be: reconstructing the features of multiple candidate lateral paths based on the dynamic characteristics of the environment.

[0047] The output space-time interaction feature (also known as ST Query) can be a high-dimensional feature vector, which can be a conditionally weighted combination of the original path features in the dynamic feature space of the environment. This feature can integrate the geometric properties of the path with the constraints and influences of the environment on the path.

[0048] For example, the first cross-attention mechanism can calculate that an environmental feature representing "vehicle cutting in from the left" and a path feature representing "fine-tuning to the left" have high attention weights, so that the representation of the path in the fused features will more significantly contain information related to this dynamic risk.

[0049] Step S105: Generate a vertical behavior query vector based on a data-driven vertical behavior model.

[0050] In this embodiment, the core of this step may be to utilize behavioral patterns learned from real driving data to generate feature representations that characterize reasonable longitudinal driving behavior.

[0051] A data-driven longitudinal behavior model can refer to a computational model that extracts longitudinal speed control patterns from massive amounts of actual driving data from human drivers using machine learning methods. This model does not rely on manually preset rules, but rather autonomously discovers and summarizes typical driving behavior patterns through data learning.

[0052] The longitudinal behavior query vector (also known as the V-line query), as the output of this step, can be a high-dimensional feature vector used to abstractly represent one or more typical longitudinal driving strategies, such as smooth following, active acceleration, or deceleration to yield. This vector will serve as the input for the query representing "speed intent" in subsequent steps.

[0053] For example, the process of building the model may include: firstly, performing cluster analysis on the longitudinal velocity profile in the historical trajectory to identify a limited number of typical behavior patterns; and then encoding these patterns into dense feature vectors through a feature extraction network.

[0054] For example, typical behavioral patterns may include, but are not limited to: smooth following, aggressive following, emergency braking, and smooth acceleration.

[0055] In practical applications, the vertical behavior model activates or generates vertical behavior query vectors that match the current scene context.

[0056] Step S106: Using the second cross-attention mechanism, the vertical behavior query vector and the path query vector are fused to generate a scenario-based vertical behavior query vector.

[0057] In this embodiment, this step can be a key step in achieving the initial fusion of longitudinal behavior and lateral path. Its purpose is to match and conditionally apply the data-driven, general longitudinal behavior strategy with the specific candidate lateral paths of the vehicle, generating a more targeted longitudinal planning representation.

[0058] The second cross-attention mechanism is implemented similarly to the first cross-attention mechanism. This step may include: using a longitudinal behavior query vector as query input, representing multiple possible longitudinal speed control strategies (such as following, acceleration, and deceleration) suggested by the data-driven model.

[0059] The path query vector, as the key and value input, represents the geometric and strategic features of multiple candidate lateral paths for the vehicle.

[0060] The fusion process of the second cross-attention mechanism can be described as follows: using different vertical speed strategies (queries) as conditions, the features (keys / values) of multiple candidate horizontal paths are filtered and weighted and fused to output a scenario-based vertical behavior query vector (also known as V line query).

[0061] The physical meaning of the contextualized vertical behavior query vector lies in the fact that it encapsulates the semantic information of "what vertical strategy should be adopted for a specific path".

[0062] For example, the mechanism might learn that for a path that involves "slightly adjusting to the right to avoid a collision," the most relevant longitudinal strategy is "gentle following" or "conservative deceleration"; while for a path that involves "keeping the lane centered," it might be more associated with the "active acceleration" strategy.

[0063] Through this step, vertical behavior can be transformed from a general pattern into scenario-based decision-making that is tied to a specific path.

[0064] Step S107: Using the third cross-attention mechanism, the scenario-based longitudinal behavior query vector is fused with the spatial-temporal interaction features to generate the final trajectory planning result.

[0065] In this embodiment, this step can be the final deep fusion of horizontal and vertical behaviors. Its purpose is to finally synthesize the vertical planning that has undergone path conditionalization with the intelligent path planning that already includes environmental interaction semantics.

[0066] The third cross-attention mechanism can be the last cross-attention operation in this process.

[0067] In this step, the third cross-attention mechanism may include the following: The contextualized vertical behavior query vector serves as the query input, representing the vertical speed intent after initial path matching.

[0068] Spatial-temporal interaction features serve as key and value inputs, representing a set of intelligent path strategies that have been validated and weighted by dynamic environmental information (behavior of other traffic participants, map constraints).

[0069] The fusion process of the third cross-attention mechanism can be described as follows: using a scenario-based vertical strategy (query) as the final condition, path strategies (key / value) rich in environmental interaction information are ultimately filtered and fused. This process ensures that vertical acceleration / deceleration behavior and horizontal path selection can be highly coordinated in complex dynamic environments.

[0070] The final trajectory planning result is a multimodal trajectory combining horizontal and vertical axes.

[0071] Each trajectory can be composed of a series of path points and their corresponding motion state information such as velocity and acceleration. It can be the optimized result obtained after the whole process of "rule path generation → environmental interaction understanding → vertical and horizontal strategy matching → dynamic scene fusion", which can be directly executed by downstream control modules.

[0072] The end-to-end trajectory generation method and apparatus of the above embodiments of this disclosure, which combines rules and models, integrates rule-generated interpretable and highly secure candidate paths with the powerful learning and generalization capabilities of data-driven models. Rules ensure the safety and compliance of the basic path, while models enhance intelligent response capabilities in unknown or complex scenarios, thereby significantly improving the overall performance and reliability of the system while reducing code complexity. Through a multi-layered cross-attention interaction mechanism, the final trajectory is not merely a simple superposition of lateral and longitudinal behaviors, but an intelligent decision after in-depth verification and fusion of environmental dynamic information and path geometric information. This mechanism ensures a high degree of coordination between longitudinal acceleration / deceleration behavior and lateral path selection, significantly improving the safety and rationality of trajectory planning. Because the model deeply integrates the prediction and interaction of multimodal behaviors of surrounding traffic participants during the planning process, the generated trajectory can highly adapt to typical complex scenarios such as narrow-gap passage and continuous multi-vehicle cutting in, thereby greatly enhancing the passability and interaction capabilities in complex traffic scenarios.

[0073] In one possible implementation of step S101 above, multiple candidate lateral paths for the vehicle are generated based on rules, including: Obtain one or more candidate lanes based on navigation information; Based on the lane centerline of the candidate lane, the path endpoint state is sampled; the path endpoint state includes different lateral offsets and longitudinal displacements. Based on the current state of the vehicle and the sampled path endpoint state, a multinomial fitting algorithm is used to generate multiple smooth candidate lateral paths.

[0074] In this embodiment, candidate lanes can refer to all lanes in which the vehicle can legally drive now and in the future, based on the global navigation path.

[0075] Path endpoint state sampling can be a systematic sampling process performed in the Frenet coordinate system. The lateral offset can refer to the offset value in the direction normal to the lane centerline. By setting multiple discrete lateral offsets (e.g., -0.5 meters, 0.0 meters, +0.5 meters), different lateral driving strategies such as lane keeping, leftward fine-tuning, and rightward fine-tuning can be covered.

[0076] Longitudinal displacement refers to different planned lengths along the lane centerline. By setting multiple discrete longitudinal displacements (e.g., 10 meters, 20 meters, 40 meters, 80 meters), different planning needs such as short-distance emergency avoidance, medium-distance lane keeping, and long-distance path planning can be covered.

[0077] Furthermore, the polynomial fitting algorithm can be used to generate a smooth path connecting the current state of the vehicle with the sampling endpoint state.

[0078] In a specific implementation, a fifth-order polynomial can be used for fitting, and its general form is:

[0079] in, It can represent lateral offset. It can represent longitudinal displacement. The fifth-order polynomial can ensure that the position, direction (i.e., first derivative) and curvature (i.e., second derivative) of the generated path are continuous at the start and end points, thus meeting the requirements of vehicle dynamics for path smoothness.

[0080] By iterating through all combinations of lateral offsets and longitudinal displacements (e.g., 3 lateral offsets × 4 longitudinal displacements = 12 combinations) and performing polynomial fitting on each combination, a set of candidate lateral paths covering various possible driving strategies can be systematically generated.

[0081] Here, these paths can collectively form the basic path set for subsequent trajectory optimization and decision-making.

[0082] The end-to-end trajectory generation method and apparatus of the above embodiments of this disclosure, which combines rules and models, utilizes deterministic rules of discrete sampling and traversal combination to cover multiple core driving behaviors such as lane keeping and left / right fine-tuning with only a small amount of computation (e.g., generating 12 paths). This significantly reduces code volume and maintenance costs while ensuring the systematicity and integrity of the strategy, fundamentally avoiding planning failures caused by rule omissions. Sampling based on lane centerlines ensures that all candidate paths are within the legal lane space. Fitting with a fifth-order polynomial rigorously guarantees the continuity and smoothness of the path in terms of position, direction, and curvature, satisfying the actual dynamic constraints of the vehicle and fundamentally ensuring the compliance, smoothness, and vehicle dynamics feasibility of the generated path.

[0083] In one possible implementation of step S101 above, path endpoint state sampling is performed, including: Multiple discrete lateral offset values ​​and multiple discrete longitudinal displacement values ​​are defined; the lateral offset values ​​include negative, zero, and positive values, which are used to characterize deviation to the left of the lane, driving along the lane centerline, and deviation to the right of the lane; the longitudinal displacement is a different planned length along the lane centerline direction. By traversing all combinations of lateral offset values ​​and longitudinal displacement values, the endpoint states of multiple paths are determined.

[0084] In this embodiment, the specific implementation of the path endpoint state sampling method used in this embodiment has been included in the corresponding description of the above embodiments, and will not be repeated here.

[0085] In one possible implementation of step S103 above, the environmental information is interactively encoded to obtain a traffic participant behavior query vector, including: Feature extraction is performed on map information and the status information of other traffic participants to obtain map features and traffic participant features; Map features and traffic participant features are input into the interaction layer to perform interaction processing between the map and traffic participants, as well as among traffic participants themselves. Based on the output of the interactive processing, a traffic participant behavior query vector is generated; wherein, the traffic participant behavior query vector contains the predicted multimodal trajectory of each traffic participant and the corresponding probability; Feature encoding is performed on multiple candidate lateral paths to obtain a path query vector, including: Multiple candidate lateral paths are input into a neural network model for processing, and the corresponding path query vectors are output.

[0086] In this embodiment, the feature extraction process can refer to processing map information and traffic participant status information through a multilayer perceptron (MLP), converting them from raw data in different formats into unified, numerical map features and traffic participant features.

[0087] Furthermore, the interaction layer can be a module specifically designed to handle the relationships between different features.

[0088] For example, the interaction layer can be a module built on a cross-attention mechanism, which can specifically perform the following two core interactions: Interaction 1: Map - Interaction with traffic participants.

[0089] Specifically, it allows map features to interact with the features of each traffic participant.

[0090] Here, the objective may be to understand how the road environment affects and constrains the behavior of traffic participants (e.g., vehicles must slow down at curves and may turn at intersections).

[0091] Interaction 2: Traffic participant-to-traffic participant interaction.

[0092] Specifically, it allows all traffic participants to interact with each other based on their characteristics.

[0093] Here, the purpose may be to deduce the mutual influence between them (for example, a car slowing down may cause the car behind to slow down as well, and the two cars may have a potential conflict at the intersection).

[0094] Feature encoding of multiple candidate lateral paths is similar to the process described above. It can be achieved by processing the candidate lateral paths through another MLP and converting them into path query vectors in numerical form.

[0095] The end-to-end trajectory generation method and apparatus that combine rules and models according to the above embodiments of this disclosure generate more accurate traffic participant behavior predictions that include multimodal trajectories and their probabilities through cross-attention interactions between map-participants and participants-participants. This allows for more accurate prediction of multiple possible trajectories for each participant and the probability of each trajectory under specific road structures and traffic interactions, providing more reliable environmental dynamic information for subsequent planning.

[0096] In one possible implementation of step S104 above, the first cross-attention mechanism is used to fuse the traffic participant behavior query vector and the path query vector to generate spatial-temporal interaction features, including: Use traffic participant behavior query vectors as queries and path query vectors as keys and values; The first cross-attention mechanism is used to calculate the correlation between the query and the key, and the values ​​are weighted and fused according to the correlation to generate spatial-temporal interaction features.

[0097] In this embodiment, the specific calculation process of the first cross-attention mechanism can be as follows: Calculate the similarity between each vector in the traffic participant behavior query vector and the path query vector, and generate attention weights; The path query vectors are weighted and summed based on attention weights; high-risk paths are assigned low weights during the fusion process, and their impact is suppressed; while low-risk paths are assigned high weights, and their characteristics are preserved and enhanced in the output features. The new feature vector after weighted summation is output as the space-time interaction feature.

[0098] As an example, suppose the vehicle is traveling in the middle lane of a three-lane road, and there is a vehicle accelerating in the left lane (referred to as Agent A). The traffic participant behavior query vector output in step S103 above contains the predicted information of Agent A, such as: high probability trajectory 1: going straight and gradually merging into the vehicle's lane (probability 70%) and low probability trajectory 2: staying in the original lane and going straight (probability 30%).

[0099] Meanwhile, the path query vector generated in step S101 corresponds to the following three representative candidate lateral paths: Path 1 (P1): maintaining the center of the current lane (lateral offset 0.0m), Path 2 (P2): slightly adjusting to the right lane (lateral offset +0.5m), and Path 3 (P3): slightly adjusting to the left lane (lateral offset -0.5m).

[0100] At this point, the workflow of the first cross-attention mechanism can be as follows: The query vector representing "Agent A entry" is matched and calculated with the three path vectors.

[0101] Here, path P1 has the highest correlation with the "cutting in" intention because the two directly overlap in space, indicating a high risk of collision; path P3 also has a high correlation with the "cutting in" intention because a slight adjustment to the left would actively approach the cutting vehicle, increasing the risk; path P2 has the lowest correlation with the "cutting in" intention because a slight adjustment to the right would move away from the conflict area in space.

[0102] Based on the aforementioned relevance, a fusion weight is assigned to each path. Typically, paths with high relevance (e.g., P1, P3) receive lower weights (due to their higher risk), while paths with low relevance (e.g., P2) receive higher weights. These weights are then used to perform a weighted summation of the original path query vectors (P1, P2, P3). During this process, the features of path P2 dominate the fused features, while the feature contributions of paths P1 and P3 are significantly suppressed.

[0103] The final output space-time interaction features can strongly characterize the semantics of "fine-tuning to the right (P2) is the safest strategy at present" because it is dominated by safe P2 path features and incorporates environmental interaction information of "avoiding conflict with vehicles cutting in from the left".

[0104] The end-to-end trajectory generation method and apparatus of the above embodiments of this disclosure, which combines rules and models, uses the predicted behavior of traffic participants as a query and performs cross-attention calculation with candidate paths. This mechanism can automatically and quantitatively assess the risk level of each path conflicting with the dynamic environment. By deeply integrating the two through a standard, learnable attention mechanism, it ensures that the final trajectory planning is based on both "path geometric feasibility" and "environmental interaction safety," fundamentally improving the rationality of the system's decision-making in complex scenarios.

[0105] In one possible implementation of step S105 above, generating a vertical behavior query vector based on a data-driven vertical behavior model includes: The K-means algorithm was used to cluster the longitudinal behavior of expert trajectories in the road survey data to obtain several typical longitudinal behavior clusters. The vertical behavior clusters are input into the neural network model for feature extraction, generating vertical behavior query vectors.

[0106] In this embodiment, longitudinal behavior can refer to the movement pattern of a vehicle along the road direction during driving, which may include, but is not limited to, acceleration, deceleration, cruising, etc. Longitudinal behavior can usually be represented by a speed-time curve.

[0107] Expert trajectories in road data collection refer to trajectory data generated by experienced drivers in real road environments. These data can reflect the decision-making wisdom of human drivers when facing different traffic scenarios.

[0108] Furthermore, by analyzing the longitudinal behavior of the aforementioned expert trajectories using the K-means clustering algorithm, the massive amount of driving behavior data can be summarized into a limited number of typical patterns, namely, longitudinal behavior clusters. Each behavior cluster represents a typical driving strategy.

[0109] For example, longitudinal behavior clusters may include, but are not limited to: smooth following clusters, aggressive acceleration clusters, gentle deceleration clusters, or emergency braking clusters.

[0110] Furthermore, the vertical behavior clusters are feature-extracted using an MLP, transforming them from raw numerical sequences into high-dimensional feature representations, i.e., vertical behavior query vectors. These vectors retain the essential features of the corresponding vertical behavior patterns while possessing a format suitable for subsequent deep learning model processing.

[0111] The end-to-end trajectory generation method and apparatus of the above embodiments of this disclosure, which combines rules and models, clusters massive amounts of expert longitudinal behaviors using the K-means algorithm. This summarizes infinite, continuous driving behaviors (such as various acceleration and deceleration curves) into finite, representative longitudinal behavior clusters. This process precipitates human drivers' experiential knowledge into quantifiable data templates, providing high-quality, structured prior information for data-driven models. By extracting features from these behavior clusters to generate query vectors, subsequent fusion models can efficiently invoke these typical driving strategies, rather than blindly searching in the raw data, thereby significantly improving the efficiency and reliability of decision-making.

[0112] In one possible implementation of step S106 above, the vertical behavior query vector and the path query vector are fused to generate a contextualized vertical behavior query vector, including: Use the vertical behavior query vector as the query and the path query vector as the key and value; The second cross-attention mechanism is used to calculate the correlation between the query and the key, and the values ​​are weighted and fused according to the correlation to generate a contextualized vertical behavior query vector.

[0113] In this embodiment, the second cross-attention mechanism is a key step in achieving the initial fusion of longitudinal behavior and lateral path. Its purpose is to match and conditionally apply the data-driven, general longitudinal behavior strategy with the specific candidate lateral paths of the vehicle, generating a more targeted longitudinal planning representation.

[0114] The specific calculation process of this mechanism can be as follows: calculate the similarity between the vertical behavior query vector and each vector in the path query vector to generate attention weights; perform weighted summation on the path query vectors based on the attention weights; and output the new feature vector after weighted summation, which is the contextualized vertical behavior query vector.

[0115] As a concrete example, suppose the vertical behavior query vector generated in step S105 contains the following three typical vertical strategies: Strategy A (Gentle Following): Maintain a steady speed and a safe distance.

[0116] Strategy B (Active Acceleration): Moderately increase vehicle speed and actively pass through the scenario.

[0117] Strategy C (Conservative Deceleration): Reduce vehicle speed in advance to allow for a greater safety margin.

[0118] Meanwhile, the path query vector generated and encoded in step S101 represents multiple candidate lateral paths, such as: Path X: Represents the path characteristic of "driving in the center of the current lane".

[0119] Path Y: Represents the path characteristics of "minor adjustment to the right to avoid".

[0120] At this point, the working process of the second cross-attention mechanism can be described as follows: Each vertical strategy (as a query) is matched against two path features (as keys and values). The physical meaning of this is to evaluate the compatibility and appropriateness of each vertical strategy with each horizontal path.

[0121] For example, strategy A (gentle following) may be most associated with path X (driving in the center), which represents a stable driving tendency to "stay in the current state".

[0122] Strategy B (active acceleration) is likely to be most closely associated with path Y (avoiding to the right), which represents a decision-making tendency to "actively change the state to pass through the scenario".

[0123] Strategy C (conservative deceleration) may have a relatively balanced correlation with multiple paths, reflecting a cautious driving style that leaves room for safety.

[0124] Based on the aforementioned correlation, different fusion weights are assigned to each vertical strategy, and the path query vectors are weighted and summed.

[0125] For strategy A, when fusing to generate its corresponding contextualized vector, the features of path X will dominate.

[0126] For strategy B, when fusing to generate its corresponding contextualized vector, the features of path Y will dominate.

[0127] The output of this process is no longer the original, generic vertical behavior query vector, but a scenario-specific vertical behavior query vector. This vector encapsulates semantic information about "what vertical strategy should be adopted for a specific path or type of path," laying a precise and conditional foundation for the subsequent final integration with the environment.

[0128] The end-to-end trajectory generation method and apparatus of the above embodiments of this disclosure, which combines rules and models, intelligently matches general longitudinal driving behaviors with specific lateral paths through a second cross-attention mechanism, generating a scenario-based representation of longitudinal behavior. This step transforms longitudinal behavior from an independent speed curve into a driving decision tightly coupled with the spatial path. This conditionalization process of longitudinal behavior greatly improves the accuracy and efficiency of subsequent final lateral-longitudinal fusion, and is a key preliminary step in generating reasonable and collaborative multimodal trajectories.

[0129] In one possible implementation of step S107 above, a third cross-attention mechanism is used to fuse the contextualized longitudinal behavior query vector with spatial-temporal interaction features to generate the final trajectory planning result, including: Use the contextualized vertical behavior query vector as the query and the spatial-temporal interaction features as the key and value; The third cross-attention mechanism is used to calculate the correlation between the query and the key, and the values ​​are weighted and fused according to the correlation to generate the final trajectory planning result.

[0130] In this embodiment, the third cross-attention mechanism is the step that achieves the final deep fusion of horizontal and vertical behaviors. Its purpose is to synthesize the vertical planning that has undergone path conditionalization with the intelligent path planning that already includes environmental interaction semantics, and generate a multimodal trajectory that coordinates horizontal and vertical behavior.

[0131] The specific calculation process of this mechanism can be shown below: Calculate the similarity between the contextualized vertical behavior query vector and each vector in the spatial-temporal interaction features to generate attention weights; perform weighted summation on the spatial-temporal interaction features based on the attention weights; output the new feature vector after weighted summation, and obtain the final trajectory planning result after decoding.

[0132] As a concrete example, following the output of step S106, the scenario-based vertical behavior query vector may contain the following vertical strategies after path conditionalization: Strategy A' (Center-Gentle Following): This strategy is tied to the "Center-Drive" path and emphasizes maintaining a steady speed.

[0133] Strategy B' (Avoidance-Active Acceleration): This strategy is tied to the "Avoid to the Right" path and emphasizes appropriately increasing vehicle speed to pass through.

[0134] Strategy C' (Center-Conservative Deceleration): This strategy is tied to the "Center Driving" path and emphasizes decelerating in advance to allow for more space.

[0135] Meanwhile, the space-time interaction features (ST Query) output in step S104 represent the set of intelligent path strategies after being verified by environmental interactions (such as other vehicles cutting in, intersection conflicts, etc.).

[0136] At this point, the working process of the third cross-attention mechanism can be described as follows: Each scenario-based vertical strategy (as a query) is then matched with path features rich in environmental information (as keys and values). The physical meaning of this is to evaluate which "path-speed" combination strategy is the safest, most reasonable, and most effective under real-world dynamic constraints.

[0137] For example, strategy B' (avoidance-proactive acceleration) may be most strongly associated with the feature in the ST Query that represents "the avoidance path is safe and the path ahead is clear", and thus be strengthened.

[0138] Conversely, if ST Query also contains a strong risk signal of "a vehicle rapidly approaching from the right", then strategy B' may be given a lower weight, because actively accelerating to the right to avoid the vehicle may create a new conflict.

[0139] Strategy A' (centering out and following gently) is likely to be highly correlated with the features in ST Query that characterize "current lane safety", making it a stable alternative.

[0140] Strategy C' (centering-conservative deceleration) may be highly correlated with the feature in ST Query that represents "potential risks exist and space needs to be reserved", and will be highlighted when the risk is high.

[0141] Based on the final correlation coefficients described above, a weighted fusion is performed. The output feature vector is then processed by a trajectory decoder to generate a multimodal trajectory planning result that combines horizontal and vertical axes. For example: Trajectory 1: Driven by strategy A', outputting a trajectory that is centered along the lane and maintains a stable speed.

[0142] Trajectory 2: Driven by strategy B', it outputs a trajectory that slightly adjusts to the right to avoid an obstacle, accompanied by moderate acceleration.

[0143] Trajectory 3: Driven by strategy C', it outputs a conservative trajectory that travels in the center but decelerates in advance.

[0144] The end-to-end trajectory generation method and apparatus combining rules and models, as described in the above embodiments of this disclosure, achieves end-to-end intelligent decision-making from raw information to final planning through a third cross-attention mechanism. This mechanism ensures high coordination between longitudinal acceleration / deceleration and lateral path selection in complex dynamic environments. The final generated trajectory is no longer a simple superposition of path and speed, but a unified decision optimized through the entire process of "rule-based path generation → environmental interaction understanding → longitudinal and lateral strategy matching → dynamic scene fusion," thereby achieving higher levels of safety, compliance, comfort, and intelligence, and flexibly coping with complex urban scenarios such as narrow road traffic and multiple vehicles cutting in line.

[0145] In one possible implementation of the above embodiments, after generating the final trajectory planning result, the method further includes: The trajectory planning results are scored and ranked to select the optimal trajectory; the scoring and ranking includes at least one of the following verifications: safety verification, compliance verification, comfort verification, and continuity verification.

[0146] In this embodiment, the scoring and ranking process quantifies and evaluates various indicators of multiple trajectories, calculates a comprehensive score according to a preset weight allocation, and finally selects the trajectory with the highest score as the optimal trajectory output. The weights for safety verification, compliance verification, comfort verification, and continuity verification can be allocated as 5:2:2:1.

[0147] The specific implementation of safety verification can be as follows: perform collision detection, calculate the minimum distance between the trajectory and static and dynamic obstacles in the environment; evaluate the safety margin of each point on the trajectory, and identify potential collision risks.

[0148] The specific implementation of compliance verification can be as follows: check whether the trajectory complies with traffic rules, verify whether the trajectory points are always within the lane boundaries, and ensure that the speed value does not exceed the road speed limit.

[0149] The specific implementation method of comfort verification can be as follows: assess whether the acceleration (Jerk) of the trajectory is within the comfortable range of ±0.6 to ±1.2 m / s³, and verify whether the turning radius corresponding to the trajectory curvature is not less than 5 meters, thereby ensuring that the acceleration changes smoothly and providing a comfortable riding experience.

[0150] The specific implementation of continuity verification can be as follows: ensure smooth connection between the current trajectory and the historical trajectory, control the deviation between adjacent planning cycle trajectory points to within 0.3 meters, thereby avoiding trajectory jumps and ensuring the stability of control execution.

[0151] Here, it can be understood that the specific thresholds (acceleration range, minimum turning radius, trajectory deviation) in the aforementioned comfort and continuity checks can be occupant safety and comfort thresholds set based on a large amount of road test driving data. Through this weighted scoring mechanism based on actual driving data, the system can select the trajectory with the best overall performance in terms of safety, compliance, comfort, and continuity from multimodal trajectories, providing the best execution plan for vehicle control.

[0152] The end-to-end trajectory generation method and apparatus of the above embodiments of this disclosure, which combines rules and models, achieves a scientific and comprehensive quantitative evaluation of multimodal trajectories by establishing a multi-objective weighted scoring system based on road test data, ensuring the comprehensive optimality of the final output trajectory across multiple key dimensions. Through collision detection, compliance review, comfort filtering, and smoothness assurance, trajectories that may pose safety risks, violate traffic regulations, cause discomfort, or experience sudden control changes are effectively filtered out, ultimately outputting a safe, legal, comfortable, and stable optimal solution, ensuring the reliability and direct executability of the final output trajectory.

[0153] In one specific embodiment, please refer to Figure 2 , Figure 2 This is a schematic diagram of a specific process for an end-to-end trajectory generation method combining rules and models, provided in an embodiment of this disclosure. The process may include the following steps: First, multiple candidate lateral paths for the vehicle are generated based on preset rules (such as multinomial fitting). These paths are then feature-encoded using a multilayer perceptron to obtain path query vectors.

[0154] Simultaneously, the environmental information encoding module acquires map information and status information of other traffic participants. This information is then processed for feature extraction using a multilayer perceptron and input into the interaction layer. In the interaction layer, interactions between the map and traffic participants, as well as among traffic participants themselves, are processed to ultimately generate a traffic participant behavior query vector. This vector encapsulates the multimodal trajectories and probabilities predicted by surrounding agents.

[0155] Next, the first cross-attention mechanism is executed. In this mechanism, the traffic participant behavior query vector is used as the query, and the path query vector is used as the key and value, and they are deeply fused. The output of this mechanism is a space-time interaction feature, which characterizes a series of intelligent path strategies under the dynamic environmental interaction constraints.

[0156] In parallel, the vertical behavior generation module is based on a data-driven model. It uses the K-means algorithm to cluster the vertical behaviors of expert trajectories to obtain vertical behavior clusters, and then uses a multilayer perceptron to extract features and generate vertical behavior query vectors.

[0157] Subsequently, a second cross-attention mechanism is executed. In this mechanism, the longitudinal behavior query vector is used as the query, and the path query vector is used as the key and value, and then fused together. The output of this mechanism is a contextualized longitudinal behavior query vector, which contains longitudinal behavior information after interacting with the vehicle's lateral path.

[0158] Then, a third cross-attention mechanism is executed. In this mechanism, the contextualized vertical behavior query vector is used as the query, and the spatial-temporal interaction features are used as the key and value to perform the final fusion of horizontal and vertical behaviors. The output of this mechanism is the multimodal trajectory combining horizontal and vertical behaviors.

[0159] Finally, the multimodal trajectories are scored and ranked. The trajectories are comprehensively scored based on verification rules such as safety, compliance, comfort, and continuity, and an optimal trajectory is selected and output to the vehicle control module for execution.

[0160] In one embodiment, an end-to-end trajectory generation device 300 combining rules and models is provided. This end-to-end trajectory generation device 300 corresponding one-to-one with the end-to-end trajectory generation method combining rules and models described in the above embodiments is provided. Figure 3 As shown, the end-to-end trajectory generation device 300 that combines the rule with the model includes: The lateral path generation module 301 is used to generate multiple candidate lateral paths for the vehicle based on rules; The environmental information acquisition module 302 is used to acquire environmental information, including map information and status information of other traffic participants. The environmental information encoding module 303 is used to interactively encode environmental information to obtain traffic participant behavior query vectors; and to perform feature encoding on multiple candidate lateral paths to obtain path query vectors. The first cross-fusion module 304 is used to fuse the traffic participant behavior query vector and the path query vector using the first cross-attention mechanism to generate spatial-temporal interaction features. Vertical behavior generation module 305 is used to generate vertical behavior query vectors based on a data-driven vertical behavior model; The second cross-fusion module 306 is used to fuse the vertical behavior query vector and the path query vector using the second cross-attention mechanism to generate a scenario-based vertical behavior query vector. The final trajectory generation module 307 is used to utilize the third cross-attention mechanism to fuse the scenario-based longitudinal behavior query vector with spatial-temporal interaction features to generate the final trajectory planning result.

[0161] In one embodiment, the lateral path generation module 301 is specifically used to obtain one or more candidate lanes based on navigation information; Based on the lane centerline of the candidate lane, the path endpoint state is sampled; the path endpoint state includes different lateral offsets and longitudinal displacements. Based on the current state of the vehicle and the sampled path endpoint state, a multinomial fitting algorithm is used to generate multiple smooth candidate lateral paths.

[0162] In one embodiment, the lateral path generation module 301 is specifically used to set multiple discrete lateral offset values ​​and multiple discrete longitudinal displacement values; wherein, the lateral offset includes negative values, zero values ​​and positive values, used to represent offset to the left of the lane, travel along the center line of the lane and offset to the right of the lane; the longitudinal displacement is different planned lengths along the direction of the center line of the lane. By traversing all combinations of lateral offset values ​​and longitudinal displacement values, the endpoint states of multiple paths are determined.

[0163] In one embodiment, the environmental information encoding module 303 is specifically used to extract features from map information and the state information of other traffic participants to obtain map features and traffic participant features. Map features and traffic participant features are input into the interaction layer to perform interaction processing between the map and traffic participants, as well as among traffic participants themselves. Based on the output of the interactive processing, a traffic participant behavior query vector is generated; wherein, the traffic participant behavior query vector contains the predicted multimodal trajectory of each traffic participant and the corresponding probability; Feature encoding is performed on multiple candidate lateral paths to obtain a path query vector, including: Multiple candidate lateral paths are input into a neural network model for processing, and the corresponding path query vectors are output.

[0164] In one embodiment, the first cross-fusion module 304 is used to use the traffic participant behavior query vector as a query and the path query vector as a key and value; The first cross-attention mechanism is used to calculate the correlation between the query and the key, and the values ​​are weighted and fused according to the correlation to generate spatial-temporal interaction features.

[0165] In one embodiment, the longitudinal behavior generation module 305 is specifically used to cluster the longitudinal behavior of expert trajectories in the road sampling data using the K-means algorithm to obtain multiple typical longitudinal behavior clusters; The vertical behavior clusters are input into the neural network model for feature extraction, generating vertical behavior query vectors.

[0166] In one embodiment, the second cross-fusion module 306 is specifically used to use the vertical behavior query vector as a query and the path query vector as a key and value; The second cross-attention mechanism is used to calculate the correlation between the query and the key, and the values ​​are weighted and fused according to the correlation to generate a contextualized vertical behavior query vector.

[0167] In one embodiment, the final trajectory generation module 307 is specifically used to use the scenario-based longitudinal behavior query vector as the query and the spatial-temporal interaction features as the key and value; The third cross-attention mechanism is used to calculate the correlation between the query and the key, and the values ​​are weighted and fused according to the correlation to generate the final trajectory planning result.

[0168] In one embodiment, the system further includes a scoring and sorting module 308, wherein: The scoring and sorting module 308 is specifically used to score and sort the trajectory planning results in order to select the optimal trajectory; wherein, the scoring and sorting includes at least one of the following verifications: safety verification, compliance verification, comfort verification, and continuity verification.

[0169] In one embodiment, the lateral path generation module 301 is specifically used to generate multiple smooth candidate lateral paths using a fifth-order polynomial; and / or, Generate the final trajectory planning result, including generating a multimodal trajectory that combines lateral and longitudinal directions.

[0170] It should be noted that the end-to-end trajectory generation device for combining corresponding rules and models provided in the above embodiments is only illustrated by the division of the above program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the above system can be divided into different program modules to complete all or part of the processing described above. In addition, the system provided in the above embodiments and the corresponding Figure 1 The embodiments of the methods shown belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0171] This disclosure also provides an electronic device having the above-described features. Figure 3 The end-to-end trajectory generation device shown combines rules and models.

[0172] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure.

[0173] The following is a detailed reference. Figure 4The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 402 or a program loaded from memory 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of the electronic device. The processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0174] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0175] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from memory 408, or installed from ROM 402. When the computer program is executed by processor 401, it performs the functions defined in the network data stream hardware offloading method for heterogeneous descriptor unified processing of embodiments of this disclosure.

[0176] Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0177] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the network data stream hardware offloading method for unified processing of heterogeneous descriptors shown in the above embodiments is implemented.

[0178] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0179] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An end-to-end trajectory generation method combining rules and models, characterized in that, The method includes: Multiple candidate lateral paths for the vehicle are generated based on rules; Acquire environmental information, including map information and status information of other traffic participants; The environmental information is interactively encoded to obtain a traffic participant behavior query vector; the multiple candidate lateral paths are feature-encoded to obtain a path query vector. By utilizing the first cross-attention mechanism, the traffic participant behavior query vector and the path query vector are fused to generate spatial-temporal interaction features; Generate vertical behavior query vectors based on a data-driven vertical behavior model; By utilizing the second cross-attention mechanism, the vertical behavior query vector and the path query vector are fused to generate a contextualized vertical behavior query vector; By utilizing the third cross-attention mechanism, the scenario-based vertical behavior query vector is fused with the spatial-temporal interaction features to generate the final trajectory planning result.

2. The method according to claim 1, characterized in that, The rule-based generation of multiple candidate lateral paths for the vehicle includes: Obtain one or more candidate lanes based on navigation information; Based on the lane centerline of the candidate lane, the path endpoint state is sampled; wherein, the path endpoint state includes different lateral offsets and longitudinal displacements; Based on the current state of the vehicle and the sampled path endpoint state, a multinomial fitting algorithm is used to generate multiple smooth candidate lateral paths.

3. The method according to claim 2, characterized in that, The step of sampling the path endpoint status includes: Multiple discrete lateral offset values ​​and multiple discrete longitudinal displacement values ​​are defined; wherein, the lateral offset includes negative, zero, and positive values, used to characterize deviation to the left of the lane, travel along the lane centerline, and deviation to the right of the lane; the longitudinal displacement is a different planned length along the direction of the lane centerline; By traversing all combinations of the lateral offset value and the longitudinal displacement value, multiple path endpoint states are determined.

4. The method according to claim 1, characterized in that, The interactive encoding of the environmental information to obtain the traffic participant behavior query vector includes: Feature extraction is performed on map information and the status information of other traffic participants to obtain map features and traffic participant features; The map features and traffic participant features are input into the interaction layer to perform interaction processing between the map and traffic participants, as well as among traffic participants themselves. Based on the output of the interaction processing, the traffic participant behavior query vector is generated; wherein, the traffic participant behavior query vector includes the predicted multimodal trajectory of each traffic participant and the corresponding probability; The step of feature encoding the multiple candidate lateral paths to obtain a path query vector includes: The multiple candidate lateral paths are input into a neural network model for processing, and the corresponding path query vectors are output.

5. The method according to claim 4, characterized in that, The method of using a first cross-attention mechanism to fuse the traffic participant behavior query vector with the path query vector to generate spatial-temporal interaction features includes: Use the traffic participant behavior query vector as the query and the path query vector as the key and value; The correlation between the query and the key is calculated using a first cross-attention mechanism, and the values ​​are weighted and fused based on the correlation to generate the spatial-temporal interaction features.

6. The method according to claim 1, characterized in that, The data-driven vertical behavior model generates vertical behavior query vectors, including: The K-means algorithm was used to cluster the longitudinal behavior of expert trajectories in the road survey data to obtain several typical longitudinal behavior clusters. The vertical behavior cluster is input into a neural network model for feature extraction to generate the vertical behavior query vector.

7. The method according to claim 1, characterized in that, The step of using a second cross-attention mechanism to fuse the vertical behavior query vector with the path query vector to generate a contextualized vertical behavior query vector includes: Use the vertical behavior query vector as the query and the path query vector as the key and value; The second cross-attention mechanism is used to calculate the correlation between the query and the key, and the value is weighted and fused according to the correlation to generate the scenario-based vertical behavior query vector.

8. The method according to claim 7, characterized in that, The third cross-attention mechanism is used to fuse the contextualized vertical behavior query vector with the spatial-temporal interaction features to generate the final trajectory planning result, including: Use the scenario-based vertical behavior query vector as the query and the space-time interaction features as the key and value; The correlation between the query and the key is calculated using a third cross-attention mechanism, and the values ​​are weighted and fused based on the correlation to generate the final trajectory planning result.

9. The method according to claim 8, characterized in that, After generating the final trajectory planning result, the method further includes: The trajectory planning results are scored and ranked to select the optimal trajectory; wherein the scoring and ranking includes at least one of the following verifications: safety verification, compliance verification, comfort verification, and continuity verification.

10. The method according to claim 2, characterized in that, The method of generating multiple smooth candidate lateral paths using a polynomial fitting algorithm includes using a fifth-order polynomial; and / or, The generation of the final trajectory planning result includes generating a multimodal trajectory that combines horizontal and vertical axes.

11. An end-to-end trajectory generation device combining rules and models, characterized in that, The device includes: The lateral path generation module is used to generate multiple candidate lateral paths for the vehicle based on rules; An environmental information acquisition module is used to acquire environmental information, including map information and status information of other traffic participants. The environmental information encoding module is used to interactively encode the environmental information to obtain a traffic participant behavior query vector; and to perform feature encoding on the multiple candidate lateral paths to obtain a path query vector. The first cross-fusion module is used to fuse the traffic participant behavior query vector with the path query vector using the first cross-attention mechanism to generate spatial-temporal interaction features. The vertical behavior generation module is used to generate vertical behavior query vectors based on a data-driven vertical behavior model. The second cross-fusion module is used to fuse the vertical behavior query vector with the path query vector using the second cross-attention mechanism to generate a contextualized vertical behavior query vector. The final trajectory generation module is used to fuse the scenario-based longitudinal behavior query vector with the spatial-temporal interaction features using a third cross-attention mechanism to generate the final trajectory planning result.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the end-to-end trajectory generation method combining rules and models as described in any one of claims 1 to 9 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the end-to-end trajectory generation method combining rules and models as described in any one of claims 1 to 9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the end-to-end trajectory generation method that combines rules and models as described in any one of claims 1 to 9.

Images