Training method of driving model, vehicle control method, device, vehicle and equipment

Abstract

The present disclosure provides a driving model training method, a vehicle control method, a device, a vehicle and equipment, and relates to the application of artificial intelligence technology in the field of vehicles. The training method comprises: obtaining scene information of a driving scene, and a labeled trajectory and a negative sample trajectory associated with the scene information; the spatial distance between the negative sample trajectory and the labeled trajectory is the closest, and the safety level of the labeled trajectory is higher than that of the negative sample trajectory; a driving model is used to plan a trajectory according to the scene information to obtain a predicted trajectory; and the driving model is trained according to the predicted trajectory, the labeled trajectory and the negative sample trajectory. Thus, the driving model can not only reproduce the real driving behavior corresponding to the labeled trajectory, but also effectively identify and actively avoid adjacent behaviors that may appear reasonable and feasible in terms of spatial geometry but actually have high risks, while maintaining the naturalness of the planned trajectory, significantly improving the safety index and enhancing the decision reliability in safety asymmetric scenes.

Description

Technical Field

[0001] This disclosure relates to the application of artificial intelligence technology in the vehicle field, and in particular to a method for training a driving model, a vehicle control method, a device, a vehicle, and equipment. Background Technology

[0002] Intelligent driving models in related technologies typically rely solely on imitation learning mechanisms to learn driving strategies from expert demonstration trajectories. This makes it difficult to establish precise and effective safety boundaries in intelligent driving planning tasks, thus limiting the generalization ability of intelligent driving models to scenarios with safety asymmetry. Summary of the Invention

[0003] This disclosure proposes a method for training a driving model, a vehicle control method, a device, a vehicle, and equipment to at least partially solve one of the technical problems in the related art.

[0004] One embodiment of this disclosure proposes a method for training a driving model, comprising: acquiring first scene information of a first driving scenario, and a first labeled trajectory and a negative sample trajectory associated with the first scene information; wherein the negative sample trajectory is closest in spatial distance to the first labeled trajectory, and the first labeled trajectory has a higher safety level than the negative sample trajectory; using a driving model to perform trajectory planning based on the first scene information to obtain a predicted trajectory; and training the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

[0005] In summary, by introducing negative sample trajectories that are geometrically very close to the first labeled trajectory but have lower safety (i.e., although the negative sample trajectory is highly similar to the first labeled trajectory in path shape, it has defects in the safety dimension, such as intruding into the oncoming lane, deviating from the legal driving area (such as crossing the solid lane line or driving off the curb), potentially colliding with static obstacles or dynamic traffic participants, and performing illegal operations in the no-lane-change area), and by combining the first labeled trajectory, the negative sample trajectory, and the predicted trajectory output by the driving model, the driving model is trained through comparative optimization. This allows the driving model to explicitly learn fine-grained safety decision boundaries around the first labeled trajectory. As a result, the driving model can not only reproduce the real driving behavior corresponding to the first labeled trajectory, but also effectively identify and actively avoid those nearby behaviors that seem reasonable and feasible in spatial geometry but actually have high risks. While maintaining the naturalness and smoothness of the planned trajectory, it significantly improves the safety index and enhances the decision reliability in scenarios with safety asymmetry.

[0006] As one possible implementation, the method for obtaining the negative sample trajectory includes: using a scene encoding network in the negative sample generation model to encode the first scene information to obtain a first scene feature; using a decoding network in the negative sample generation model to decode the first scene feature based on a first noise trajectory to obtain multiple candidate trajectories; and determining the negative sample trajectory that is spatially closest to the first labeled trajectory from the multiple candidate trajectories.

[0007] Therefore, by encoding the first scene information using a negative sample generation model and generating multiple candidate trajectories based on a decoding process that introduces noise, and then selecting negative sample trajectories that are spatially close to the first labeled trajectory from these candidate trajectories, it is possible to ensure that the constructed negative sample trajectories are highly similar to real driving behavior in terms of geometric shape and contextual semantics, while possessing clear safety defects or unsafe factors (such as deviation from legal drivable areas, collision risks, etc.). This "high similarity, low safety" negative sample trajectory effectively simulates the error-prone decisions of human drivers in critical states, allowing comparative training to focus on subtle differences near the safety boundary, thereby significantly improving the driving model's sensitivity and discrimination ability to dangerous trajectory areas, and avoiding training ineffectiveness or overfitting caused by negative sample trajectories that are too simple or deviate from reality.

[0008] As one possible implementation, determining the negative sample trajectory that is spatially closest to the first labeled trajectory from the plurality of candidate trajectories includes: performing a safety assessment on the plurality of candidate trajectories based on a set safety scoring strategy to obtain a safety score for the plurality of candidate trajectories; wherein the safety score is used to indicate the safety level of the vehicle traveling along the candidate trajectory; filtering the plurality of candidate trajectories to obtain retained trajectories whose safety scores are lower than a set scoring threshold; and determining the negative sample trajectory from the retained trajectories based on the spatial distance between the retained trajectories and the first labeled trajectory.

[0009] Therefore, by introducing a trajectory selection mechanism based on a safety scoring strategy, multiple candidate trajectories are quantitatively evaluated for safety. Candidate trajectories with safety scores higher than a set threshold (i.e., relatively safe candidate trajectories) are filtered out, retaining only those with clear safety hazards or unsafe factors (such as deviation from legal drivable areas, collision risks, etc.). Then, the trajectory closest in spatial distance to the first labeled trajectory is selected from the retained trajectories as the final negative sample trajectory. This ensures that the selected negative sample trajectory not only closely approximates the first labeled trajectory geometrically but also exhibits significant safety defects. This construction method of "high similarity, low safety" negative sample trajectories allows the driving model to effectively focus on adversarial regions near the safety decision boundary. During comparative training, it can accurately perceive safety abrupt changes caused by minute trajectory deviations, thus more reliably distinguishing between safe and dangerous behaviors in complex or critical scenarios, significantly improving the driving model's safety robustness and decision-making ability.

[0010] As one possible implementation, the method of using the decoding network in the negative sample generation model to decode the first scene features based on the first noise trajectory to obtain multiple candidate trajectories includes: amplifying a set noise distribution parameter; sampling noise based on the amplified noise distribution parameter to obtain the first noise trajectory; and using the decoding network to decode the first scene features based on a set classifier-free guided CFG scale parameter and the first noise trajectory to obtain the multiple candidate trajectories; wherein the CFG scale parameter is less than 1.

[0011] In summary, by introducing a CFG scale parameter less than 1 during the generation of negative sample trajectories and combining it with the amplification of noise distribution parameters, the decoding network can intentionally weaken its strong dependence on the features of the first scene during conditional generation. This significantly improves the randomness and behavioral diversity of the generated trajectories while maintaining semantic consistency with the first driving scene. As a result, the multiple candidate trajectories generated by the negative sample generation model can closely surround the first labeled trajectory (expert demonstration trajectory) in space. At the same time, due to noise disturbances and low CFG guidance, unsafe factors such as deviation from the legal drivable area and intrusion into the conflict area are naturally introduced. Furthermore, by combining the safety scoring strategy to select negative sample trajectories that are "geometrically close but significantly less safe than the first labeled trajectory", high-resolution supervision signals can be provided for comparative training. This enables the driving model to accurately capture subtle differences near the safety boundary and effectively enhance its ability to perceive and avoid critical risks, thereby achieving a higher level of safety and decision robustness in complex and dynamic real driving scenarios.

[0012] As one possible implementation, the negative sample generation model discards the first scene features with a set probability. The decoding network decodes the first scene features based on a set classifier-free guided CFG scale parameter and the first noise trajectory to obtain multiple candidate trajectories. This includes: using the decoding network to determine a first velocity field at each time step based on a first intermediate state and the first scene features; wherein the first intermediate state is generated based on the first noise trajectory and the first labeled trajectory; using the decoding network to determine a second velocity field at each time step based only on the first intermediate state; fusing the first velocity field and the second velocity field according to the CFG scale parameter to obtain a third velocity field; and generating the multiple candidate trajectories based on the third velocity field.

[0013] In summary, by randomly discarding the first scene features with a set probability in the negative sample generation model, and fusing conditional and unconditional trajectory generation paths based on the CFG mechanism—at each time step, the decoding network uses a CFG scale parameter less than 1 to weight and fuse the first velocity field (the conditional velocity field calculated by combining the first noisy trajectory, the first labeled trajectory, and the first scene features) and the second velocity field (the unconditional velocity field calculated by relying only on intermediate states and ignoring the first scene features) to generate the final third velocity field, and calculates multiple candidate trajectories accordingly. This allows the trajectory generation process to retain some scene semantic constraints while actively introducing controllable semantic deviations, thereby efficiently exploring neighborhood regions in the trajectory space that are geometrically similar to the real driving behavior corresponding to the first labeled trajectory, but produce unsafe behaviors (such as lane departure or intrusion into obstacle areas) due to weakened conditions. This results in the generated negative sample trajectories having both high spatial similarity and clear safety defects, providing fine decision boundary supervision signals for subsequent comparative training, significantly enhancing the driving model's ability to discriminate safety critical states and its robust avoidance capabilities, and effectively improving its safety and generalization performance in complex and uncertain traffic environments.

[0014] As one possible implementation, the training method of the negative sample generation model includes: acquiring second scene information of a second driving scenario and a second labeled trajectory associated with the second scene information; encoding the second scene information using the scene encoding network to obtain second scene features, and determining the target velocity field to be fitted by the decoding network based on the difference between the second labeled trajectory and the second noise trajectory; using the decoding network to determine a fourth velocity field at each time step based on a second intermediate state at each time step and the second scene features; wherein the second intermediate state is generated based on the second noise trajectory and the second labeled trajectory; and training the negative sample generation model based on the difference between the fourth velocity field and the target velocity field.

[0015] In summary, by using the second labeled trajectory of the second driving scenario as the supervision signal during the training of the negative sample generation model, a target velocity field evolving from the second noisy trajectory to the second labeled trajectory is constructed. This drives the decoding network to learn and predict a fourth velocity field aligned with the intermediate state and second scene features. This enables the negative sample generation model to accurately model the dynamic evolution of conditional trajectories. It ensures that even when noise perturbations or weakened scene conditions are introduced during the inference phase (e.g., discarding scene features or using a CFG scale parameter <1), the decoding network can still generate candidate trajectories that closely approximate real driving behavior in terms of geometric structure and semantic plausibility. This provides a high-fidelity, highly diverse pool of candidate trajectories for subsequent negative sample trajectory selection. Consequently, the generated negative sample trajectories possess both spatial proximity to the first labeled trajectory and effectively embed unsafe factors, significantly improving the accuracy of safety boundary characterization during comparative training, thereby enhancing the risk discrimination capability and decision robustness of the driving model.

[0016] As one possible implementation, the scene encoding network includes a first visual encoding network and a first text encoding network. The step of using the scene encoding network in the negative sample generation model to encode the first scene information to obtain the first scene feature includes: using the first visual encoding network to encode visual data from at least one perspective in the first scene information to obtain a first visual feature from a bird's-eye view; using the first text encoding network to encode text data in the first scene information to obtain a first text feature; and fusing the first visual feature and the first text feature to obtain the first scene feature.

[0017] Therefore, the first visual encoding network can extract structured environmental features (such as lane lines and obstacle distribution) from a bird's-eye view, while the first text encoding network can extract semantic or contextual features (such as traffic rules and navigation prompts) from the first driving scenario. By fusing the two, more comprehensive first-scenario features can be generated. This allows the generation process of negative sample trajectories to not only rely on visual geometric information but also incorporate high-level semantic constraints, thereby improving the spatial rationality of the generated negative sample trajectories and conforming to the semantic logic of the first driving scenario (for example, it will not generate a left-turn trajectory under the "No Left Turn" text prompt). This ensures that the negative sample trajectories are both realistic and targeted, enhancing the driving model's ability to distinguish between safe and unsafe behaviors in complex driving scenarios and improving the effectiveness of training and generalization performance.

[0018] As one possible implementation, the driving model includes a second visual encoding network, a second text encoding network, and a trajectory prediction network. The step of using the driving model to perform trajectory planning based on the first scene information to obtain a predicted trajectory includes: using the second visual encoding network to encode visual data from at least one perspective in the first scene information to obtain a second visual feature from a bird's-eye view; using the second text encoding network to encode text data in the first scene information to obtain a second text feature; fusing the second visual feature and the second text feature to obtain a scene fusion feature; and using the trajectory prediction network to perform trajectory planning based on the scene fusion feature to obtain a predicted trajectory.

[0019] In summary, by using a second visual encoding network and a second text encoding network to independently encode the visual and text data in the first scene information and then fusing them to generate a unified scene fusion feature, this dual-modal fusion mechanism significantly enhances the driving model's comprehensive understanding of complex driving scenarios, avoids the limitations of single-modal perception, and generates predictive trajectories that are more consistent with real traffic logic, safer, and semantically more accurate. This effectively improves the planning reliability and generalization performance of the intelligent driving system in open and dynamic environments.

[0020] As one possible implementation, the negative sample trajectory is generated using a negative sample generation model based on the first scene information; the second visual encoding network shares model parameters with the first visual encoding network in the negative sample generation model; and / or, the second text encoding network shares model parameters with the first text encoding network in the negative sample generation model.

[0021] This allows the negative sample generation model and the driving model to have a consistent understanding of the same driving scenario, thus improving the accuracy of trajectory planning.

[0022] As one possible implementation, training the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory includes: determining an imitation learning loss based on a first difference between the predicted trajectory and the first labeled trajectory; wherein the imitation learning loss is positively correlated with the first difference; determining a negative distance loss based on a second difference between the predicted trajectory and the negative sample trajectory; wherein the negative distance loss is negatively correlated with the second difference; and training the driving model based on the imitation learning loss and the negative distance loss.

[0023] Therefore, by simultaneously constructing imitation learning loss (measuring the deviation between the predicted trajectory and the first labeled trajectory) and negative distance loss (encouraging the predicted trajectory to stay away from unsafe negative sample trajectories), the driving model not only learns "how to act" but also explicitly learns "which trajectory areas should not be approached". Among them, imitation learning loss ensures that the driving model reproduces high-quality driving behavior, while negative distance loss maintains maximum separability with spatially adjacent but unsafe negative sample trajectories, explicitly shaping the safety decision boundary around the first labeled trajectory. The synergistic optimization of the two effectively alleviates the problem that pure imitation learning mechanisms in related technologies are prone to getting trapped in local suboptimal or insensitive to critical risks, significantly improving the driving model's discrimination ability and risk avoidance initiative in safety asymmetry scenarios, thereby enhancing the safety and robustness of overall trajectory planning while maintaining the naturalness of the trajectory.

[0024] As one possible implementation, the driving model further includes an auxiliary task network; the auxiliary task network is used to perform map reconstruction based on the scene fusion features to obtain map reconstruction results, and / or to perform target detection based on the scene fusion features to obtain target detection results; wherein the map reconstruction results are used to indicate the semantic information and / or vector information of map elements in the first driving scene; wherein the target detection results are used to indicate the location information and / or category information of obstacles in the first driving scene.

[0025] Therefore, by introducing an auxiliary task network based on scene fusion features into the driving model and simultaneously performing perception tasks such as map reconstruction and object detection, the backbone network can obtain richer supervision signals during training, promoting the deep fusion of visual and textual features and improving representation capabilities. At the same time, the semantic and vector information provided by the map reconstruction results can enhance the driving model's prior knowledge of static scenes, while the object detection results strengthen the real-time perception of dynamic risks. Together, they provide multi-granular and multi-dimensional contextual support for trajectory planning, thereby significantly improving the environmental consistency, safety, and robustness of predicted trajectories, especially demonstrating stronger generalization ability and reliability in highly complex and partially observable driving scenarios.

[0026] As one possible implementation, training the driving model based on the imitation learning loss and the negative distance loss includes: acquiring map annotation information and obstacle annotation information associated with the first scene information; determining an auxiliary task loss based on a third difference between the map annotation information and the map reconstruction result, and / or a fourth difference between the obstacle annotation information and the target detection result; determining a target loss based on the imitation learning loss, the negative distance loss, and the auxiliary task loss; and adjusting the model parameters in the driving model based on the target loss.

[0027] Therefore, an auxiliary task loss based on map reconstruction and object detection is introduced—that is, utilizing the geometric-semantic differences between map annotation information and map reconstruction results, and the localization-classification differences between obstacle annotation information and object detection results—to provide a structured environmental perception supervision signal for the driving model. This auxiliary task loss not only promotes the visual-text encoding network to learn more discriminative and generalizable scene fusion features, but also enables the driving model to implicitly align high-order semantic maps with dynamic obstacle understanding during end-to-end training, thus providing more accurate and complete contextual support for trajectory planning. Within the framework of the above multi-task joint optimization, the main task (trajectory planning task) and the auxiliary task mutually enhance each other, improving the driving model's prediction accuracy and adaptability to complex, partially observable scenarios.

[0028] Another embodiment of this disclosure proposes a vehicle control method, including: acquiring actual scene information and vehicle information of the driving scenario in which the vehicle is located; using a driving model to control the vehicle based on the actual scene information and the vehicle information; wherein the driving model is trained using the driving model training method described in the foregoing embodiment.

[0029] In summary, by using a trained driving model to control the vehicle based on multi-source scene information and fine-grained vehicle information, the accumulation of errors and information loss in the perception-planning-control link of the traditional modular architecture can be avoided, significantly improving the real-time response, behavioral rationality and operational safety of the intelligent driving system in open road environments.

[0030] Another embodiment of this disclosure proposes a training device for a driving model, comprising: a first acquisition module, configured to acquire first scene information of a first driving scenario, and a first labeled trajectory and a negative sample trajectory associated with the first scene information; wherein the negative sample trajectory is closest in spatial distance to the first labeled trajectory, and the first labeled trajectory has a higher safety level than the negative sample trajectory; a planning module, configured to use the driving model to perform trajectory planning based on the first scene information to obtain a predicted trajectory; and a first training module, configured to train the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

[0031] Therefore, by introducing negative sample trajectories that are highly similar to the first labeled trajectory in spatial geometry but have lower safety (i.e., although the negative sample trajectory is highly similar to the first labeled trajectory in path shape, it has defects in the safety dimension, such as intruding into the oncoming lane, deviating from the legal driving area (such as crossing the solid lane line or driving off the curb), potentially colliding with static obstacles or dynamic traffic participants, and performing illegal operations in the no-lane-change area), and by combining the first labeled trajectory, the negative sample trajectory and the predicted trajectory output by the driving model, the driving model is trained by comparison and optimization. This allows the driving model to explicitly learn fine-grained safety decision boundaries around the first labeled trajectory. As a result, the driving model can not only reproduce the real driving behavior corresponding to the first labeled trajectory, but also effectively identify and actively avoid those nearby behaviors that seem reasonable and feasible in spatial geometry but actually have high risks. While maintaining the naturalness and smoothness of the planned trajectory, it significantly improves the safety index and enhances the decision reliability in scenarios with safety asymmetry.

[0032] As one possible implementation, the first training module is configured to: determine an imitation learning loss based on a first difference between the predicted trajectory and the first labeled trajectory; wherein the imitation learning loss is positively correlated with the first difference; determine a negative distance loss based on a second difference between the predicted trajectory and the negative sample trajectory; wherein the negative distance loss is negatively correlated with the second difference; and train the driving model based on the imitation learning loss and the negative distance loss.

[0033] Therefore, by simultaneously constructing imitation learning loss (measuring the deviation between the predicted trajectory and the first labeled trajectory) and negative distance loss (encouraging the predicted trajectory to stay away from unsafe negative sample trajectories), the driving model not only learns "how to act" but also explicitly learns "which trajectory areas should not be approached". Among them, imitation learning loss ensures that the driving model reproduces high-quality driving behavior, while negative distance loss maintains maximum separability with spatially adjacent but unsafe negative sample trajectories, explicitly shaping the safety decision boundary around the first labeled trajectory. The synergistic optimization of the two effectively alleviates the problem that pure imitation learning mechanisms in related technologies are prone to getting trapped in local suboptimal or insensitive to critical risks, significantly improving the driving model's discrimination ability and risk avoidance initiative in safety asymmetry scenarios, thereby enhancing the safety and robustness of overall trajectory planning while maintaining the naturalness of the trajectory.

[0034] As one possible implementation, the driving model further includes an auxiliary task network; the auxiliary task network is used to perform map reconstruction based on the scene fusion features to obtain map reconstruction results, and / or to perform target detection based on the scene fusion features to obtain target detection results; wherein the map reconstruction results are used to indicate the semantic information and / or vector information of map elements in the first driving scene; wherein the target detection results are used to indicate the location information and / or category information of obstacles in the first driving scene.

[0035] Therefore, by introducing an auxiliary task network based on scene fusion features into the driving model and simultaneously performing perception tasks such as map reconstruction and object detection, the backbone network can obtain richer supervision signals during training, promoting the deep fusion of visual and textual features and improving representation capabilities. At the same time, the semantic and vector information provided by the map reconstruction results can enhance the driving model's prior knowledge of static scenes, while the object detection results strengthen the real-time perception of dynamic risks. Together, they provide multi-granular and multi-dimensional contextual support for trajectory planning, thereby significantly improving the environmental consistency, safety, and robustness of predicted trajectories, especially demonstrating stronger generalization ability and reliability in highly complex and partially observable driving scenarios.

[0036] As one possible implementation, the first training module is configured to: acquire map annotation information and obstacle annotation information associated with the first scene information; determine an auxiliary task loss based on a third difference between the map annotation information and the map reconstruction result, and / or a fourth difference between the obstacle annotation information and the target detection result; determine a target loss based on the imitation learning loss, the negative distance loss, and the auxiliary task loss; and adjust the model parameters in the driving model based on the target loss.

[0037] Therefore, an auxiliary task loss based on map reconstruction and object detection is introduced—that is, utilizing the geometric-semantic differences between map annotation information and map reconstruction results, and the localization-classification differences between obstacle annotation information and object detection results—to provide a structured environmental perception supervision signal for the driving model. This auxiliary task loss not only promotes the visual-text encoding network to learn more discriminative and generalizable scene fusion features, but also enables the driving model to implicitly align high-order semantic maps with dynamic obstacle understanding during end-to-end training, thus providing more accurate and complete contextual support for trajectory planning. Within the framework of the above multi-task joint optimization, the main task (trajectory planning task) and the auxiliary task mutually enhance each other, improving the driving model's prediction accuracy and adaptability to complex, partially observable scenarios.

[0038] In another aspect of this disclosure, a vehicle control device is proposed, comprising: a second acquisition module for acquiring actual scene information and vehicle information of the driving scenario in which the vehicle is located; and a control module for controlling the vehicle using a driving model based on the actual scene information and the vehicle information; wherein the driving model is trained using the driving model training device described in the foregoing embodiment.

[0039] In summary, by using a trained driving model to control the vehicle based on multi-source scene information and fine-grained vehicle information, the accumulation of errors and information loss in the perception-planning-control link of the traditional modular architecture can be avoided, significantly improving the real-time response, behavioral rationality and operational safety of the intelligent driving system in open road environments.

[0040] This disclosure also provides an embodiment of a vehicle, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: implement a training method for a driving model as described in the foregoing aspect, and / or implement a vehicle control method as described in the foregoing aspect.

[0041] In another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the driving model training method as described in the foregoing aspect, and / or the vehicle control method as described in the foregoing aspect.

[0042] Another aspect of this disclosure provides a chip including an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is configured to perform a driving model training method as described in the foregoing aspect, and / or to perform a vehicle control method as described in the foregoing other aspect.

[0043] In another aspect of this disclosure, a non-transitory computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the driving model training method as described in the foregoing aspect, and / or, when executed, implement the vehicle control method as described in the foregoing aspect.

[0044] Another aspect of this disclosure provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the driving model training method as described in the foregoing aspect, and / or, when executed, implements the vehicle control method as described in the foregoing aspect.

[0045] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0046] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which: Figure 1 A flowchart illustrating a method for training a driving model provided as an exemplary embodiment of this disclosure; Figure 2 A flowchart illustrating another method for training a driving model provided as an exemplary embodiment of this disclosure; Figure 3 A schematic flowchart illustrating another method for training a driving model provided as an exemplary embodiment of this disclosure; Figure 4 A flowchart illustrating another method for training a driving model provided as an exemplary embodiment of the present disclosure; Figure 5 A flowchart illustrating a method for training a driving model provided as an exemplary embodiment of this disclosure; Figure 6 A schematic flowchart of a vehicle control method provided for an exemplary embodiment of the present disclosure; Figure 7 A schematic diagram illustrating the application process of a driving model and a negative sample generation model provided for exemplary embodiments of this disclosure; Figure 8 A schematic diagram of the structure of a training device for a driving model provided as an exemplary embodiment of the present disclosure; Figure 9 A schematic diagram of the structure of a vehicle control device provided for an exemplary embodiment of the present disclosure; Figure 10 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present disclosure; Figure 11 This is a block diagram illustrating a vehicle according to an exemplary embodiment; Figure 12 This is a schematic diagram of the structure of a chip proposed for an exemplary embodiment of the present disclosure. Detailed Implementation

[0047] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0048] It should be noted that the acquisition, storage, use, and processing of data in this disclosed technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0049] It should also be noted that all data processed in this disclosure is data that has been explicitly authorized by users or relevant parties, and has been de-identified or anonymized before collection and use, and does not contain any personally identifiable information or user privacy content; all data is used only for model training and vehicle control purposes, ensuring that data security and user privacy rights are fully protected while achieving technical effects.

[0050] In view of at least one of the problems existing in the above-mentioned related technologies, this disclosure proposes a driving model training method, a vehicle control method, a device, a vehicle, and an equipment.

[0051] The following description, with reference to the accompanying drawings, describes a method for training a driving model, a vehicle control method, an apparatus, a vehicle, and a device according to embodiments of the present disclosure.

[0052] Figure 1 This is a flowchart illustrating a method for training a driving model, provided as an exemplary embodiment of the present disclosure.

[0053] It should be noted that the driving model training method of this disclosure can be applied to a driving model training device. In some possible embodiments, the driving model training device can be configured in an electronic device so that the electronic device can perform the driving model training function. Additionally, in some possible embodiments, the driving model training device can also be software within an electronic device.

[0054] Among them, electronic devices include, but are not limited to: terminals, servers (or cloud, servers, etc.). Terminals can be cars with communication functions, smart cars, mobile phones, wearable devices, computers, tablets (Pads), computers with wireless transceiver functions, supercomputers, etc.

[0055] like Figure 1 As shown, the training method for this driving model may include the following steps S101 to S103: Step S101: Obtain the first scene information of the first driving scenario, as well as the first labeled trajectory and negative sample trajectory associated with the first scene information; wherein, the spatial distance between the negative sample trajectory and the first labeled trajectory is the shortest, and the safety level of the first labeled trajectory is higher than that of the negative sample trajectory.

[0056] The first driving scenario can be any driving scenario in which a vehicle is located; the vehicle can be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles; the vehicle can be an assisted driving vehicle, a semi-assisted driving vehicle, or a non-assisted driving vehicle.

[0057] Among them, assisted driving refers to the technology that uses sensors, algorithms and artificial intelligence to perceive the environment, make decisions, plan and execute control commands of the vehicle in order to assist the driver to drive more safely and efficiently.

[0058] The scene information of the first driving scenario (referred to as the first scene information in this disclosure) includes at least one of the visual data, text data and point cloud data in the first driving scenario.

[0059] The visual data in the first driving scenario includes, but is not limited to, images and videos. This visual data can be collected by the vehicle's onboard cameras. For example, the onboard cameras can be cameras located on the exterior of the vehicle and can have surround-view functionality. There can be at least one onboard camera; however, to improve the comprehensiveness and accuracy of image information collection, multiple onboard cameras can be used, distributed in different locations within the vehicle as needed.

[0060] In the first driving scenario, text data refers to textual information related to vehicle driving, including but not limited to at least one of the following: vehicle driving instructions, vehicle kinematic state, vehicle driving task information, vehicle historical driving trajectory, and weather information (or meteorological information) of the vehicle's location. Therefore, the driving model can understand driving needs and the driving environment from multiple dimensions and levels, thereby planning more realistic, accurate, and reliable trajectories, effectively improving the vehicle's driving performance and safety in complex driving scenarios.

[0061] Navigation instructions are used to direct the vehicle's driving plan at the current moment. For example, navigation instructions can be commands output by a navigation system (such as map software) based on the destination location at the current moment. These instructions can be used to instruct the driving model on the macro-level operations it needs to perform, including but not limited to: left turn, right turn, left lane change, right lane change, straight ahead, and driving along the lane. In other words, navigation instructions include, but are not limited to: left turn instructions, right turn instructions, left lane change instructions, right lane change instructions, straight ahead instructions, and driving along the lane instructions.

[0062] Among them, the kinematic state can be obtained through the vehicle's relevant sensors, including but not limited to: driving speed, driving acceleration, steering angle, braking status, etc.

[0063] Among them, driving task information refers to text information related to the driving task that the vehicle needs to complete, which clarifies the goal and requirements of the vehicle's current driving.

[0064] Weather information (or meteorological information) can be used to characterize the degree of impact of the vehicle's environment on perception and driving. For example, rain and snow may cause slippery roads, reduced visibility, or decreased sensor performance; high temperatures or strong light may cause overexposure of vehicle cameras; and smog will significantly weaken the detection capabilities of vehicle radar and vehicle cameras.

[0065] In the first driving scenario, point cloud data can be collected by the vehicle's onboard radar, which includes, but is not limited to, lidar, millimeter-wave radar, and ultrasonic radar.

[0066] The first labeled trajectory includes the actual trajectory obtained by the vehicle based on the first scenario information and the expert demonstration trajectory (or expert trajectory).

[0067] Among them, the spatial distance between the negative sample trajectory and the first labeled trajectory is close, and the safety level of the first labeled trajectory is higher than that of the negative sample trajectory.

[0068] For example, the spatial distance between the negative sample trajectory and the first labeled trajectory is less than a set distance threshold, and the negative sample trajectory contains unsafe factors. These unsafe factors include, but are not limited to, deviations from the legal drivable area (such as crossing a solid lane line or driving off the curb), collision risk, sharp turns, and other dangerous factors.

[0069] For example, a negative sample trajectory with a relatively low level of security and the closest spatial distance to the first labeled trajectory can be determined from multiple candidate trajectories.

[0070] Step S102: The driving model is used to plan the trajectory based on the first scene information to obtain the predicted trajectory.

[0071] The driving model is an artificial intelligence (AI) model to be trained. This disclosure does not impose specific restrictions on the model structure of the driving model. For example, the driving model includes, but is not limited to: a vision-language model (VLM), a vision-language-action model (VLA), a unimodal model, a multimodal model, a world model, etc.

[0072] The predicted trajectory can be a differential trajectory, a non-differential final trajectory, a single-modal trajectory, or a multi-modal trajectory; this disclosure does not impose any limitations on this. Different modes correspond to different driving behaviors.

[0073] For example, taking the predicted trajectory as a difference trajectory, the predicted trajectory can be represented as: Where K refers to the number of trajectory points. h represents the displacement during the k-th time interval (which can be equivalent to the average vehicle speed multiplied by the length of the time interval). k This represents the vehicle's orientation radian or heading angle within the k-th time interval. The sine and cosine codes for the heading angle are used to avoid discontinuities in angle regression.

[0074] For example, taking the predicted trajectory as the final trajectory, the predicted trajectory can be represented as: Among them, T f For predicting the time domain (e.g., 8 time steps, assuming the time interval between adjacent time steps is 0.5s, then it corresponds to the next 4 seconds), h represents the coordinates of the i-th time step. i Let represent the heading angle at the i-th time step.

[0075] In this embodiment of the disclosure, a driving model can perform trajectory planning based on the first scene information to obtain a predicted trajectory. For example, the driving model can encode the first scene information to obtain scene fusion features of the first driving scene, and perform trajectory planning based on these scene fusion features to obtain a predicted trajectory.

[0076] Step S103: Train the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

[0077] For example, the model parameters in the driving model can be adjusted based on a first difference between the predicted trajectory and the first labeled trajectory, and based on a second difference between the predicted trajectory and the negative sample trajectory, so as to minimize the first difference and / or maximize the second difference.

[0078] It should be noted that this disclosure only uses the first difference minimization and / or the second difference maximization as examples of the termination conditions for model training, but this disclosure is not limited to these. For example, the termination conditions may also include: the training time reaches a set time, the training rounds reach a set number of rounds, etc.

[0079] The driving model training method of this disclosure introduces negative sample trajectories that are highly similar to the first labeled trajectory in spatial geometry but have lower safety (i.e., although the negative sample trajectory is highly similar to the first labeled trajectory in path shape, it has defects in safety dimension, such as intruding into the oncoming lane, deviating from the legal drivable area (such as crossing the solid lane line or driving off the curb), having potential collisions with static obstacles or dynamic traffic participants, or performing illegal operations in the prohibited lane-changing area, etc.). The driving model is then compared and optimized by integrating the first labeled trajectory, the negative sample trajectory and the predicted trajectory output by the driving model. This allows the driving model to explicitly learn fine-grained safety decision boundaries around the first labeled trajectory. As a result, the driving model can not only reproduce the real driving behavior corresponding to the first labeled trajectory, but also effectively identify and actively avoid those nearby behaviors that seem reasonable and feasible in spatial geometry but actually have high risks. While maintaining the naturalness and smoothness of the planned trajectory, it significantly improves the safety index and enhances the decision reliability in safety asymmetry scenarios.

[0080] As one possible implementation method, Figure 2 A flowchart illustrating another method for training a driving model provided as an exemplary embodiment of this disclosure.

[0081] It should be noted that the training method of the driving model can be executed alone, or it can be executed together with any embodiment of this disclosure or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this disclosure do not limit this.

[0082] like Figure 2 As shown, the training method for this driving model may include the following steps S201 to S206: Step S201: Obtain the first scene information of the first driving scenario and the first labeled trajectory associated with the first scene information.

[0083] It should be noted that the explanation of step S201 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0084] Step S202: The scene encoding network in the negative sample generation model is used to encode the first scene information to obtain the first scene features.

[0085] The negative sample generation model is a trained AI model, and this disclosure does not impose specific restrictions on the model structure of the negative sample generation model. The negative sample generation model includes an encoding network (referred to as the scene encoding network in this disclosure) and a decoding network.

[0086] In this application embodiment, no specific restrictions are placed on the network structure of the scene encoding network and the decoding network.

[0087] In this embodiment of the disclosure, the scene encoding network in the negative sample generation model can be used to encode the first scene information to obtain the scene features of the first driving scene, which are referred to as the first scene features in this disclosure.

[0088] As one possible implementation, the scene encoding network includes a first visual encoding network and a first text encoding network. The encoding method of the first scene information is as follows: using the first visual encoding network, visual data from at least one perspective in the first scene information is visually encoded to obtain the first visual features of the bird's-eye view; using the first text encoding network, text data in the first scene information is text-encoded to obtain the first text features; and using feature fusion technology, the first visual features and the first text features are fused to obtain the first scene features.

[0089] This disclosure does not impose any restrictions on the network structure of the first visual encoding network, nor on the network structure of the first text encoding network.

[0090] Understandably, the first visual encoding network can extract structured environmental features from a bird's-eye view (such as lane lines and obstacle distribution), while the first text encoding network can extract semantic or contextual features from the first driving scenario (such as traffic rules and navigation prompts). By fusing the two, more comprehensive first-scenario features can be generated. This allows the generation of negative sample trajectories to not only rely on visual geometric information but also incorporate high-level semantic constraints, improving the spatial rationality of the generated negative sample trajectories and conforming to the semantic logic of the first driving scenario (for example, it will not generate a left-turn trajectory under the "No Left Turn" text prompt). This ensures that the negative sample trajectories are both realistic and targeted, enhancing the driving model's ability to distinguish between safe and unsafe behaviors in complex driving scenarios and improving the effectiveness of training and generalization performance.

[0091] Step S203: The decoding network in the negative sample generation model is used to decode the features of the first scene based on the first noise trajectory to obtain multiple candidate trajectories.

[0092] The first noise trajectory can be a noise trajectory obtained by sampling noise based on noise distribution parameters, wherein the noise distribution parameters include, but are not limited to, at least one of the following: noise mean, noise standard deviation, noise disturbance direction, and noise intensity.

[0093] The safety level of the candidate trajectory can be lower than that of the first labeled trajectory.

[0094] In this embodiment of the disclosure, a decoding network in the negative sample generation model can be used to decode the first scene features based on the first noise trajectory to obtain multiple candidate trajectories.

[0095] Step S204: From multiple candidate trajectories, determine the negative sample trajectory that is spatially closest to the first labeled trajectory.

[0096] As one possible approach, the spatial distance between multiple candidate trajectories and the first labeled trajectory can be calculated separately, and the candidate trajectory with the smallest spatial distance can be used as the negative sample trajectory.

[0097] For example, the spatial distance between any candidate trajectory and the first labeled trajectory can be calculated in the following way: calculate the spatial distance between corresponding trajectory points in the candidate trajectory and the first labeled trajectory, and take the mean, cumulative value, weighted sum, or maximum value of the spatial distances of each trajectory point as the spatial distance between the candidate trajectory and the first labeled trajectory.

[0098] As another possible implementation, a safety assessment can be performed on multiple candidate trajectories based on a set safety scoring strategy to obtain safety scores for multiple candidate trajectories. The safety score is used to indicate the degree of safety of the vehicle traveling along the candidate trajectory. Based on the safety scores of multiple candidate trajectories, multiple candidate trajectories are filtered to obtain retained trajectories with safety scores lower than a set scoring threshold. Negative sample trajectories are determined from the retained trajectories based on the spatial distance between the retained trajectories and the first labeled trajectory.

[0099] The security scoring strategy can be implemented based on one or more pre-defined security scoring functions, where the security scoring function is a calculation rule or algorithm model used to generate a quantitative security scoring index.

[0100] For example, the safety scoring function is calculated based on one or more safety evaluation indicators, which include, but are not limited to: collision risk, following distance, lane departure, crossing / overriding lines, comfort, violation of driving zone rules, and prediction of dangerous scenarios.

[0101] The calculation method for the spatial distance between each retained trajectory and the first labeled trajectory is similar to the calculation method for the spatial distance between the candidate trajectory and the first labeled trajectory, and will not be elaborated here.

[0102] For example, the trajectory with the smallest spatial distance from the first labeled trajectory can be used as the negative sample trajectory.

[0103] For example, the following formula can be used to filter negative sample trajectories from multiple candidate trajectories: (1) in, This refers to the pool of unsafe candidate trajectories, containing N unsafe candidate trajectories. ; This is a safety scoring function; the lower the value, the stronger the candidate trajectory. The less safe; This refers to setting a scoring threshold to filter out unsafe trajectories; This refers to the first labeled trajectory (or the expert demonstration trajectory); This refers to the hard negative sample trajectory; It refers to Euclidean distance, which represents the sum of spatial distances over all points on the trajectory.

[0104] In summary, by introducing a trajectory selection mechanism based on a safety scoring strategy, multiple candidate trajectories are quantitatively evaluated for safety. Candidate trajectories with safety scores exceeding a set threshold (i.e., relatively safe candidate trajectories) are filtered out, retaining only those with clear safety hazards or unsafe factors (such as deviation from legal drivable areas, collision risks, etc.). Then, the trajectory closest in spatial distance to the first labeled trajectory is selected from the retained trajectories as the final negative sample trajectory. This ensures that the selected negative sample trajectory not only closely approximates the first labeled trajectory geometrically but also exhibits significant safety defects. This construction method of "high similarity, low safety" negative sample trajectories allows the driving model to effectively focus on adversarial regions near the safety decision boundary. During comparative training, it can accurately perceive safety abrupt changes caused by minor trajectory deviations, thus more reliably distinguishing between safe and dangerous behaviors in complex or critical scenarios, significantly improving the driving model's safety robustness and decision-making ability.

[0105] Step S205: The driving model is used to plan the trajectory based on the first scene information to obtain the predicted trajectory.

[0106] Step S206: Train the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

[0107] It should be noted that the explanations of steps S205 to S206 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0108] The driving model training method of this disclosure employs a negative sample generation model to encode first scene information and generates multiple candidate trajectories based on a decoding process that introduces noise. Then, it selects negative sample trajectories from these candidate trajectories that are spatially close to the first labeled trajectory. This ensures that the constructed negative sample trajectories are highly similar to real driving behavior in terms of geometry and contextual semantics, while possessing clear safety defects or unsafe factors (such as deviation from legal drivable areas, collision risks, etc.). This "high similarity, low safety" negative sample trajectory effectively simulates the error-prone decisions of human drivers in critical states, allowing comparative training to focus on subtle differences near safety boundaries. This significantly improves the driving model's sensitivity and discrimination ability to dangerous trajectory areas, avoiding ineffective training or overfitting due to overly simplistic or unrealistic negative sample trajectories.

[0109] As one possible implementation method, Figure 3 A flowchart illustrating another method for training a driving model provided as an exemplary embodiment of this disclosure.

[0110] It should be noted that the training method of the driving model can be executed alone, or it can be executed together with any embodiment of this disclosure or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this disclosure do not limit this.

[0111] like Figure 3 As shown, the training method for this driving model may include the following steps S301 to S308: Step S301: Obtain the first scene information of the first driving scenario and the first labeled trajectory associated with the first scene information.

[0112] Step S302: The scene encoding network in the negative sample generation model is used to encode the first scene information to obtain the first scene features.

[0113] It should be noted that the explanations of steps S301 to S302 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0114] Step S303: Amplify the set noise distribution parameters.

[0115] The noise distribution parameters include, but are not limited to, at least one of the following: noise mean, noise standard deviation, noise disturbance direction, and noise intensity.

[0116] In this embodiment of the disclosure, a set amplification factor can be used to amplify the noise distribution parameters; the specific value of the amplification factor is not limited and can be configured based on actual application requirements. For example, the amplification factor can be 2, 3, etc.

[0117] As an example, taking a magnification factor of 2, the noise standard deviation in the noise distribution parameter can be magnified using the following formula: (2) in, To widen the previous noise standard deviation, This represents the expanded noise standard deviation.

[0118] Step S304: Sampling is performed based on the expanded noise distribution parameters to obtain the first noise trajectory.

[0119] In this embodiment of the disclosure, noise sampling can be performed based on the amplified noise distribution parameters to obtain a first noise trajectory. For example, a target noise distribution can be constructed based on the noise distribution parameters, and random sampling can be performed on the target noise distribution to obtain a first noise sample.

[0120] The target noise distribution includes, but is not limited to, any parameterizable distribution such as Gaussian, Laplace, or mixture distribution, used to characterize the statistical properties of the noise.

[0121] Step S305: Using the decoding network in the negative sample generation model, based on the set Classifier-Free Guidance (CFG) scale parameter and the first noise trajectory, the first scene features are decoded to obtain multiple candidate trajectories; wherein, the CFG scale parameter is less than 1.

[0122] The CFG scale parameter controls the trade-off between diversity and conditional fidelity in the generated trajectories. For example, denoted as w, when w < 1, the negative sample generation model tends to generate more diverse trajectories (i.e., weakens the dependence on input scene conditions (such as first scene features), enhancing randomness); when w > 1, the negative sample generation model enhances the adherence to input scene conditions (such as first scene features), improving the consistency between the generated trajectory and the input context (i.e., improving conditional fidelity).

[0123] Understandably, by setting w to a value less than 1, the distribution range of candidate trajectories can be effectively expanded while ensuring semantic rationality. This allows for a more comprehensive coverage of potential unsafe behavior patterns, providing a rich and challenging pool of candidate trajectories for subsequent negative sample trajectory screening. The candidate trajectory pool includes multiple candidate trajectories.

[0124] In this embodiment of the disclosure, a decoding network in the negative sample generation model can be used to decode the features of the first scene based on the set CFG scale parameters and the first noise trajectory to obtain multiple candidate trajectories.

[0125] As one possible implementation, when the negative sample generation model discards the first scene features with a set probability, the decoding method of the first scene features is as follows: using a decoding network, the first velocity field at each time step is determined based on the first intermediate state and the first scene features at each time step; using a decoding network, the second velocity field at each time step is determined based only on the first intermediate state and the corresponding time step at each time step; the first velocity field and the second velocity field are fused according to the CFG scale parameters to obtain the third velocity field; and multiple candidate trajectories are generated based on the third velocity field.

[0126] The first intermediate state is generated based on the first noise trajectory and the first labeled trajectory.

[0127] For example, the first noise trajectory is labeled x0, and it is assumed that the time step t follows a uniform distribution U on [0,1], that is: Then the first intermediate state at time step t It can be represented as: ,in, This refers to the vectorized representation of the first labeled trajectory. For example, the first labeled trajectory is labeled as... ,but vec is a pointer to vectorization; the first velocity field at time step t can be expressed as: Where C represents the scene features input to the decoding network (set to null with a set probability, such as 0.1). The second velocity field at time step t can be expressed as: The third velocity field at time step t can be expressed as: .

[0128] Where w is the CFG scaling parameter, and its value is less than 1. For example, w=0.5 to increase the diversity of candidate trajectories output by the negative sample generation model and explore unsafe regions near the first labeled trajectory manifold.

[0129] For example, an Ordinary Differential Equation (ODE) system is constructed based on the third velocity field, and the ODE solver is used to numerically integrate the ODE system to generate multiple candidate trajectories.

[0130] Understandably, by randomly discarding the first scene features with a set probability in the negative sample generation model, and fusing conditional and unconditional trajectory generation paths based on the CFG mechanism—at each time step, the decoding network uses a CFG scale parameter less than 1 to weight and fuse the first velocity field (the conditional velocity field calculated by combining the first noisy trajectory, the first labeled trajectory, and the first scene features) and the second velocity field (the unconditional velocity field calculated by relying only on intermediate states and ignoring the first scene features) to generate the final third velocity field, and calculates multiple candidate trajectories accordingly. Thus, the trajectory generation process can retain some scene semantic constraints while actively introducing controllable semantic deviations, thereby efficiently exploring neighborhood regions in the trajectory space that are geometrically similar to the real driving behavior corresponding to the first labeled trajectory, but produce unsafe behaviors (such as lane departure or intrusion into obstacle areas) due to weakened conditions. This results in the generated negative sample trajectories having both high spatial similarity and clear safety defects, providing fine decision boundary supervision signals for subsequent comparative training, significantly enhancing the driving model's ability to discriminate safety critical states and robust avoidance capabilities, and effectively improving its safety and generalization performance in complex and uncertain traffic environments.

[0131] Step S306: From multiple candidate trajectories, determine the negative sample trajectory that is spatially closest to the first labeled trajectory.

[0132] Step S307: The driving model is used to plan the trajectory based on the first scene information to obtain the predicted trajectory.

[0133] Step S308: Train the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

[0134] It should be noted that the explanations of steps S306 to S308 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0135] In any embodiment of this disclosure, the training method of the negative sample generation model is as follows: acquiring second scene information of a second driving scenario and a second labeled trajectory associated with the second scene information; encoding the second scene information using a scene encoding network to obtain second scene features; determining the target velocity field to be fitted by the decoding network based on the difference between the second labeled trajectory and the second noise trajectory; using the decoding network to determine the fourth velocity field at each time step based on the second intermediate state and the second scene features at each time step; and training the negative sample generation model based on the difference between the fourth velocity field and the target velocity field.

[0136] The explanation of the second scene information can be found in the description of the first scene information in the previous embodiments, and the explanation of the second labeled trajectory can be found in the description of the first labeled trajectory in the previous embodiments. The implementation principle is similar and will not be repeated here.

[0137] The second noise trajectory is obtained by sampling noise based on noise distribution parameters. The second noise trajectory may be different from the first noise trajectory.

[0138] The second intermediate state is generated based on the second noise trajectory and the second labeled trajectory. For example, the second noise trajectory is labeled as... The second intermediate state at time step t It can be represented as: ,in, This refers to the vectorized representation of the second labeled trajectory.

[0139] The target velocity field is a physical vector field generated based on the difference between the second labeled trajectory and the second noise trajectory. For example, the labeled target velocity field is... Then we have: .

[0140] For example, the training objective of a negative sample generation model can be: (3) Among them, L FM It refers to the loss function of the negative sample generation model. The velocity field predicted by the negative sample generation model at time step t. It refers to the fourth velocity field.

[0141] In summary, by using the second labeled trajectory of the second driving scenario as the supervision signal during the training of the negative sample generation model, a target velocity field evolving from the second noisy trajectory to the second labeled trajectory is constructed. This drives the decoding network to learn and predict a fourth velocity field aligned with the intermediate state and second scene features. This enables the negative sample generation model to accurately model the dynamic evolution of conditional trajectories. It ensures that even when noise perturbations or weakened scene conditions are introduced during the inference phase (e.g., discarding scene features or using a CFG scale parameter <1), the decoding network can still generate candidate trajectories that closely approximate real driving behavior in terms of geometric structure and semantic plausibility. This provides a high-fidelity, highly diverse pool of candidate trajectories for subsequent negative sample trajectory selection. Consequently, the generated negative sample trajectories possess both spatial proximity to the first labeled trajectory and effectively embed unsafe factors, significantly improving the accuracy of safety boundary characterization during comparative training, thereby enhancing the risk discrimination capability and decision robustness of the driving model.

[0142] The training method of the driving model in this embodiment introduces a CFG scale parameter of less than 1 during the generation of negative sample trajectories, and combines it with the amplification of noise distribution parameters. The decoding network can intentionally weaken its strong dependence on the features of the first scene during conditional generation, thereby significantly improving the randomness and behavioral diversity of the generated trajectory while maintaining semantic consistency with the first driving scene. As a result, the multiple candidate trajectories generated by the negative sample generation model can closely surround the first labeled trajectory (expert demonstration trajectory) in space. At the same time, due to noise disturbance and low CFG guidance, unsafe factors such as deviation from the legal drivable area and intrusion into the conflict area are naturally introduced. Furthermore, by combining the safety scoring strategy to select negative sample trajectories that are "geometrically close but significantly less safe than the first labeled trajectory", a high-resolution supervision signal can be provided for comparative training. This enables the driving model to accurately capture subtle differences near the safety boundary, effectively strengthening its ability to perceive and avoid critical risks, thereby achieving a higher level of safety and decision robustness in complex and dynamic real driving scenarios.

[0143] As one possible implementation method, Figure 4 A flowchart illustrating another method for training a driving model provided as an exemplary embodiment of this disclosure.

[0144] It should be noted that the training method of the driving model can be executed alone, or it can be executed together with any embodiment of this disclosure or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this disclosure do not limit this.

[0145] like Figure 4 As shown, the training method for this driving model may include the following steps S401 to S406: Step S401: Obtain the first scene information of the first driving scenario, as well as the first labeled trajectory and negative sample trajectory associated with the first scene information; wherein, the spatial distance between the negative sample trajectory and the first labeled trajectory is the shortest, and the safety level of the first labeled trajectory is higher than that of the negative sample trajectory.

[0146] It should be noted that the explanation of step S401 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0147] In any embodiment of this disclosure, the negative sample trajectory may be generated using a negative sample generation model based on the first scene information.

[0148] Step S402: Using the second visual encoding network in the driving model, the visual data of at least one perspective in the first scene information is encoded to obtain the second visual features of the bird's-eye view.

[0149] The network structure of the second visual encoding network is not restricted. The second visual encoding network can share model parameters with the first visual encoding network in the negative sample generation model, ensuring consistent understanding of the same driving scenario between the negative sample generation model and the driving model, thus improving trajectory planning accuracy. Alternatively, the second visual encoding network can also not share model parameters with the first visual encoding network, meaning the first and second visual encoding networks are completely independent encoding networks, thereby improving the flexibility and applicability of the method.

[0150] In this embodiment of the disclosure, a second visual encoding network in the driving model can be used to visually encode the visual data of at least one perspective in the first scene information to obtain the second visual features of the bird's-eye view.

[0151] Step S403: The second text encoding network in the driving model is used to encode the text data in the first scene information to obtain the second text features.

[0152] The network structure of the second text encoding network is not restricted. The second text encoding network can share model parameters with the first text encoding network in the negative sample generation model, ensuring consistent understanding of the same driving scenario between the negative sample generation model and the driving model, thus improving trajectory planning accuracy. Alternatively, the second text encoding network can also not share model parameters with the first text encoding network, meaning the first and second text encoding networks are completely independent encoding networks, thereby improving the flexibility and applicability of the method.

[0153] In this embodiment of the disclosure, a second text encoding network in the driving model can be used to encode the text data in the first scene information to obtain the second text features.

[0154] Step S404: Fuse the second visual features and the second text features to obtain scene fusion features.

[0155] In this embodiment of the disclosure, feature fusion technology can be used to fuse the second visual features and the second text features to obtain the scene fusion features of the first driving scene.

[0156] Step S405: The trajectory prediction network in the driving model is used to perform trajectory planning based on scene fusion features to obtain the predicted trajectory.

[0157] It should be noted that the explanation of the predicted trajectory in the foregoing embodiments also applies to this embodiment, and will not be repeated here.

[0158] In this embodiment of the disclosure, a trajectory prediction network in the driving model can be used to perform trajectory planning based on scene fusion features in order to obtain a predicted trajectory.

[0159] Step S406: Train the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

[0160] It should be noted that the explanation of step S406 can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0161] The driving model training method of this disclosure uses a second visual encoding network and a second text encoding network to independently encode the visual data and text data in the first scene information, and then fuses them to generate a unified scene fusion feature. This dual-modal fusion mechanism significantly enhances the driving model's comprehensive understanding of complex driving scenarios, avoids the limitations of single-modal perception, and generates predicted trajectories that are more consistent with real traffic logic, safer, and semantically consistent. This effectively improves the planning reliability and generalization performance of the intelligent driving system in open and dynamic environments.

[0162] As one possible implementation method, Figure 5 A flowchart illustrating another method for training a driving model, provided as an exemplary embodiment of this disclosure.

[0163] It should be noted that the training method of the driving model can be executed alone, or it can be executed together with any embodiment of this disclosure or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this disclosure do not limit this.

[0164] like Figure 5 As shown, the training method for this driving model may include the following steps S501 to S505: Step S501: Obtain the first scene information of the first driving scenario, as well as the first labeled trajectory and negative sample trajectory associated with the first scene information; wherein, the spatial distance between the negative sample trajectory and the first labeled trajectory is the shortest, and the safety level of the first labeled trajectory is higher than that of the negative sample trajectory.

[0165] Step S502: The driving model is used to plan the trajectory based on the first scene information to obtain the predicted trajectory.

[0166] It should be noted that the explanations of steps S501 to S502 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0167] Step S503: Determine the imitation learning loss based on the first difference between the predicted trajectory and the first labeled trajectory.

[0168] In this embodiment of the disclosure, the imitation learning loss can be determined based on the first difference between the predicted trajectory and the first labeled trajectory; wherein the imitation learning loss is positively correlated with the first difference.

[0169] For example, taking the predicted trajectory as the difference trajectory, the calculation method of the learning loss can be simulated as follows: (4) in, This refers to the learning loss through imitation. It refers to predicting the trajectory. It refers to the difference trajectory corresponding to the first labeled trajectory.

[0170] Step S504: Determine the negative distance loss based on the second difference between the predicted trajectory and the negative sample trajectory.

[0171] In this embodiment of the disclosure, the negative distance loss can be determined based on the second difference between the predicted trajectory and the negative sample trajectory; wherein the negative distance loss is negatively correlated with the second difference.

[0172] For example, taking the predicted trajectory as the difference trajectory, the negative distance loss can be calculated as follows: (5) in, This refers to negative distance loss. It refers to the difference trajectory corresponding to the negative sample trajectory.

[0173] Step S505: Train the driving model based on the imitation learning loss and the negative distance loss.

[0174] In this embodiment of the disclosure, the target loss can be determined based on the imitation learning loss and the negative distance loss, and the model parameters in the driving model can be adjusted based on the target loss to minimize the target loss.

[0175] Among them, the target loss is positively correlated with the imitation learning loss, and the target loss is also positively correlated with the negative distance loss.

[0176] It should be noted that the above example only uses the training termination condition of the driving model as the objective of minimizing the loss, but this disclosure is not limited to this. For example, the training termination condition can also be: the training time reaches a set time, the training rounds reach a set number of rounds, etc.

[0177] In any embodiment of this disclosure, the driving model may further include an auxiliary task network, wherein the auxiliary task network is used to perform map reconstruction based on scene fusion features to obtain map reconstruction results, and / or the auxiliary task network is used to perform target detection based on scene fusion features to obtain target detection results.

[0178] Among them, the map reconstruction results are used to indicate the semantic and / or vector information of map elements in the first driving scenario.

[0179] Map elements refer to static or semi-static environmental entities and their semantic attributes in a digital map. For example, map elements include, but are not limited to, lane lines, pillars, cones, parking spaces, speed bumps, traffic signs, etc.

[0180] Semantic information refers to the category, functional attributes, and contextual meaning of map elements (e.g., "solid lane lines cannot be crossed", "cones indicate construction areas", "whether parking spaces are occupied"); vector information refers to the geometric shape, spatial location, orientation, and topological relationship of map elements (e.g., the centerline coordinate sequence of lane lines, the quadrilateral boundary of parking spaces, the length and orientation of speed bumps).

[0181] Among them, the target detection results are used to indicate the location information and / or category information of obstacles in the first driving scenario.

[0182] Obstacles include, but are not limited to: static obstacles (buildings, traffic facilities, etc.) and dynamic obstacles (such as surrounding vehicles, pedestrians, etc.).

[0183] Understandably, by introducing an auxiliary task network based on scene fusion features into the driving model and simultaneously performing perception tasks such as map reconstruction and object detection, the backbone network can obtain richer supervision signals during training, promoting the deep fusion of visual and textual features and improving representation capabilities. At the same time, the semantic and vector information provided by map reconstruction results can enhance the driving model's prior knowledge of static scenes, while object detection results strengthen the real-time perception of dynamic risks. Together, they provide multi-granular and multi-dimensional contextual support for trajectory planning, thereby significantly improving the environmental consistency, safety, and robustness of predicted trajectories, especially demonstrating stronger generalization ability and reliability in highly complex and partially observable driving scenarios.

[0184] In any embodiment of this disclosure, the training method of the driving model is, for example, as follows: acquiring map annotation information and obstacle annotation information associated with the first scene information; determining the auxiliary task loss based on the third difference between the map annotation information and the map reconstruction result, and / or the fourth difference between the obstacle annotation information and the target detection result; determining the target loss based on the imitation learning loss, the negative distance loss and the auxiliary task loss; and adjusting the model parameters in the driving model based on the target loss.

[0185] Among them, map annotation information is used to indicate the semantic and / or vector information of actual map elements in the first driving scenario.

[0186] Among them, obstacle labeling information is used to indicate the location and / or category information of actual obstacles in the first driving scenario.

[0187] Among them, the auxiliary task loss is positively correlated with the third difference, and the auxiliary task loss is also positively correlated with the fourth difference.

[0188] For example, an auxiliary task network is used to reconstruct a map based on scene fusion features. The resulting map reconstruction is illustrated by the auxiliary task loss, which can also be called the map reconstruction loss. The target loss L can be calculated using the following formula: (6) in, This refers to the weighting coefficient corresponding to the imitation learning loss; This refers to the weighting coefficient corresponding to the negative distance loss; This refers to map reconstruction loss. This refers to the weighting coefficient corresponding to the map reconstruction loss.

[0189] Taking the predicted trajectory as an example of the difference trajectory, the target loss L can be calculated as follows: (7) In summary, an auxiliary task loss based on map reconstruction and object detection is introduced—that is, utilizing the geometric-semantic differences between map annotation information and map reconstruction results, and the localization-classification differences between obstacle annotation information and object detection results—to provide a structured environmental perception supervision signal for the driving model. This auxiliary task loss not only promotes the visual-text encoding network to learn more discriminative and generalizable scene fusion features, but also enables the driving model to implicitly align high-order semantic maps with dynamic obstacle understanding during end-to-end training, thus providing more accurate and complete contextual support for trajectory planning. Within the framework of the aforementioned multi-task joint optimization, the main task (trajectory planning task) and the auxiliary task mutually enhance each other, improving the driving model's prediction accuracy and adaptability to complex, partially observable scenarios.

[0190] The driving model training method of this disclosure simultaneously constructs an imitation learning loss (measuring the deviation between the predicted trajectory and the first labeled trajectory) and a negative distance loss (encouraging the predicted trajectory to stay away from unsafe negative sample trajectories). This enables the driving model to not only learn "how to act" but also to explicitly learn "which trajectory areas should not be approached". The imitation learning loss ensures that the driving model reproduces high-quality driving behavior, while the negative distance loss maintains maximum separability with spatially adjacent but unsafe negative sample trajectories, explicitly shaping the safety decision boundary around the first labeled trajectory. The synergistic optimization of the two effectively solves the problem that pure imitation learning mechanisms in related technologies are prone to getting trapped in local suboptimal conditions or are insensitive to critical risks. This significantly improves the driving model's discrimination ability and risk avoidance initiative in safety asymmetry scenarios, thereby enhancing the safety and robustness of overall trajectory planning while maintaining the naturalness of the trajectory.

[0191] The above are various embodiments corresponding to the training method of the driving model. This disclosure also proposes an application method of the driving model (i.e., a vehicle control method).

[0192] As one possible implementation method, Figure 6 A schematic flowchart of a vehicle control method provided for an exemplary embodiment of this disclosure.

[0193] It should be noted that the vehicle control method can be executed alone, or it can be executed together with any embodiment of this disclosure or any possible implementation in the embodiment, or it can be executed together with any technical solution in the related technology. The embodiments of this disclosure do not limit this.

[0194] like Figure 6 As shown, the vehicle control method may include the following steps S601 to S602: Step S601: Obtain the actual scene information and vehicle information of the driving scenario in which the vehicle is located.

[0195] The actual scene information of the driving scenario in which the vehicle is located includes, but is not limited to, at least one of the following: visual data, text data, and point cloud data of the driving scenario. It should be noted that explanations of the visual data, text data, and point cloud data can be found in the relevant descriptions of any of the above embodiments, and will not be repeated here.

[0196] The vehicle information includes, but is not limited to, at least one of the following: vehicle status information (such as vehicle speed, acceleration, engine / motor status, remaining fuel / battery charge, tire pressure, braking system status, battery pack temperature data, etc.), environmental perception information (such as Global Positioning System (GPS) data, outside temperature, humidity, air quality, light intensity, etc.), driving behavior information (such as historical rapid acceleration / sudden braking records, steering angle, lane change frequency, etc.), maintenance and diagnostic information (such as Diagnostic Trouble Code (DTC), component life prediction, maintenance records, etc.), and network and communication status information (such as network signal strength, Bluetooth connection status, etc.).

[0197] In step S602, the driving model controls the vehicle based on the actual scene information and vehicle information.

[0198] The driving model can be the one described above. Figures 1 to 5 The AI ​​model trained by the training method proposed in any embodiment.

[0199] In this embodiment of the disclosure, a driving model can be used to output vehicle control information based on actual scene information and vehicle information, and control the vehicle accordingly.

[0200] The control information includes, but is not limited to: planned trajectory, target speed, steering wheel angle, accelerator / brake pedal opening, gear command, lane change decision signal, emergency braking trigger signal, and other low-level or mid-level control commands, thereby achieving complete closed-loop control from multimodal perception input to vehicle actuators.

[0201] It should be noted that the vehicle control method provided in this disclosure can be applied not only to the field of vehicles, but also to the field of robotics, to perform trajectory planning and motion control on robots, thereby improving the robot's autonomous navigation capabilities and task execution efficiency in complex environments.

[0202] The vehicle control method of this disclosure controls the vehicle based on multi-source scene information and fine-grained vehicle information through a trained driving model. This avoids the error accumulation and information loss in the perception-planning-control link in the traditional modular architecture, and significantly improves the real-time response, behavior rationality and operational safety of the intelligent driving system in open road environments.

[0203] In any embodiment of this disclosure, the technical solution provided can be applied to the research and development stage of a vehicle's intelligent driving system to train a driving model. This trained model, based on semantic understanding of the driving scenario, generates safe, comfortable driving actions that align with higher-level driving intentions in complex and dynamic traffic environments. For example, the application process of the technical solution provided in this disclosure may include, as follows: Figure 7 The two stages shown: The first stage, high-quality negative sample trajectory generation: Before or during driving model training, a negative sample generation model is used to actively construct "high-quality negative sample trajectories"—trajectories that spatially approximate the labeled trajectory (hereinafter referred to as expert demonstration trajectories) but contain unsafe factors (such as deviation from legal drivable areas, collision risks, etc.). This is achieved by guiding the CFG scale parameters and noise standard deviation (without a classifier)... The negative sample generation model can construct an ODE system based on the predicted velocity field and use the ODE solver to perform numerical integration on the ODE system to generate a diverse pool of candidate trajectories. The pool of candidate trajectories includes multiple candidate trajectories. Subsequently, each candidate trajectory in the pool of candidate trajectories is subjected to safety screening and distance screening to extract the candidate trajectory that is closest to the expert demonstration trajectory and is unsafe, which is then used as a high-quality negative sample trajectory.

[0204] Among them, the negative sample generation model includes Figure 7 The velocity field predictor based on the flow matching mechanism + ODE solver in the above embodiment may include the scene encoding network and decoding network described in the above embodiment.

[0205] The second stage, the general safety enhancement stage, introduces both imitation learning loss (bringing the predicted trajectory closer to the expert demonstration trajectory) and negative distance loss (pushing the predicted trajectory away from high-quality negative sample trajectories) during driving model training. Through this comparative optimization, the driving model can establish a clear safety boundary near the expert demonstration trajectory, thus enabling it to correctly distinguish between safe and unsafe decision directions when facing scenarios with safety asymmetry (such as the same left and right deviation error but different safety levels), significantly improving safety indicators.

[0206] As an example, the technical solution provided in this disclosure involves the training and use of two models: one is a negative sample generation model based on a flow matching mechanism, and the other is a driving model. The negative sample generation model is only used to construct hard negative sample trajectories during the training phase and is no longer used after training is complete. The driving model is jointly optimized during the training phase using imitation learning loss and negative distance loss, and runs independently during the usage phase (inference phase) without relying on the negative sample generation model.

[0207] The training objective is to simultaneously learn two abilities: "imitating expert demonstration trajectories" and "avoiding unsafe behaviors." The entire training process is divided into two phases: the first phase actively constructs hard negative sample trajectories using a negative sample generation model; the second phase introduces contrastive loss into the driving model, enabling it to approach the expert demonstration trajectories while avoiding unsafe negative sample trajectories. The training process mainly includes the following steps: 1. Training the negative sample generation model: Training a conditional flow matching negative sample generation model. The model learns a velocity field using expert demonstration trajectories as the target distribution. During training, conditions are randomly dropped with a set probability (e.g., p=0.1) to support subsequent classifier-free guidance (CFG). For example, the L1 loss function can be used to improve the quality of negative sample trajectories generated by the negative sample generation model.

[0208] The input to the negative sample generation model is a multi-view image. Kinematic state of the vehicle Navigation commands Noise trajectory Time step Intermediate state at time step t Output of the negative sample generation model: predicted velocity field .

[0209] The negative sample generation model and the driving model can share the same encoder structure, but their training processes are independent. That is, the negative sample generation model can use the same scene encoding network as the driving model. Alternatively, the negative sample generation model can also use a different scene encoding network than the driving model. However, the embodiments disclosed herein do not impose any limitations on this.

[0210] In this embodiment, the scene encoding network shares model parameters (or weights) between the negative sample generation model and the driving model to ensure that the two have a consistent understanding of the driving scene, or they may not share model parameters (or weights), and this disclosure does not limit this.

[0211] Among them, scene coding network From multiple perspective images Kinematic state of the vehicle Navigation commands Scene features (or scene fusion features, context features) C are extracted from the network. This scene encoding network is jointly optimized during the training phase of the negative sample generation model.

[0212] The decoding network in the negative sample generation model. This is also known as a velocity field prediction network. For example, the network structure of the decoding network is Transformer Decoder + LayerNorm + Feed-Forward Network, or the internal structure of the decoding network may include a multi-head self-attention layer, a cross-attention layer (fusion conditional coding) and a feed-forward network, and employ layer normalization.

[0213] The decoding network takes time step t, intermediate states at time step t, and conditional codes C as inputs, and outputs the predicted velocity field. .

[0214] The training method for the negative sample generation model is as follows: Conditional flow matching is used to match the target, and the target velocity x1-x0 is regressed using the L1 loss function. During training, the conditional encoding C is set to empty with a predetermined probability, thereby simultaneously learning conditional and unconditional generation, providing a foundation for subsequent CFG (Concurrent Flow Generation).

[0215] For example, the training objective of a negative sample generation model can be: ; Where t is the time step, which follows a uniform distribution U[0,1]; Noise trajectories, following a mean of 0 and a variance of . Gaussian distribution; Samples on the linear interpolation path (referred to as intermediate states at time step t in this disclosure). ; Experts demonstrate the vectorized representation of trajectories; C: Predicted velocity field; C: Conditional encoding (such as the first scene feature, second scene feature, or scene fusion feature in this disclosure, set to empty during training with a set probability (e.g., 0.1)). ).

[0216] 2. Construction of hard negative sample trajectories: Using a trained negative sample generation model, combined with CFG scale parameters (w<1) and amplified sampling noise. Multiple candidate trajectories are generated in parallel for each training scenario. Then, through a safety scoring strategy and a spatial distance screening mechanism, the unsafe trajectory that is spatially closest to the expert demonstration trajectory is selected from the multiple candidate trajectories and used as the hard negative sample trajectory.

[0217] For example, the third velocity field is sampled based on the classifier-free guided CFG scale parameters. According to this An ODE system is constructed, and the ODE solver is used to numerically integrate the ODE system to generate diverse and insecure candidate trajectories. The sampling method is as follows: ; Where w is the CFG scale parameter. When w < 1, the negative sample generation model tends to generate more diverse trajectories (i.e., weakens the dependence on the input scene conditions (C) and enhances randomness). When w > 1, it enhances the degree of adherence to the input scene conditions (C) and improves the consistency between the generated trajectory and the input context (i.e., improves conditional fidelity). Unconditional velocity estimation (obtained during training by conditional discarding).

[0218] For example, the method for filtering hard negative sample trajectories is as follows: ; in, This refers to the pool of unsafe candidate trajectories, containing N unsafe candidate trajectories. ; This is a safety scoring function; the lower the value, the stronger the candidate trajectory. The less safe; This refers to setting a scoring threshold to filter out unsafe trajectories; This refers to the trajectory demonstrated by the expert. This refers to the hard negative sample trajectory; It refers to Euclidean distance, which represents the sum of spatial distances over all points on the trajectory.

[0219] Taking N=64 as an example, firstly, for each training scenario, N=64 candidate trajectories are generated in parallel using a negative sample generation model. Then, a safety score (e.g., collision, leaving the drivable area) is calculated for each candidate trajectory based on a safety scoring strategy. An unsafe subset of trajectories with safety scores below a set threshold is selected. Finally, the candidate trajectory with the closest Euclidean distance to the expert demonstration trajectory is chosen from this subset as the hard negative sample trajectory. .

[0220] 3. Driving model training: Input multimodal sensor data and navigation commands, output predicted trajectory. (This involves) imitation learning loss. Based on this, add negative distance loss This causes the predicted trajectory to be pulled toward the expert demonstration trajectory while being pushed away from the hard negative sample trajectory. Optionally, the total loss (denoted as the target loss in this disclosure) may also include supervision from auxiliary tasks (such as map reconstruction, object detection). The parameters of the driving model are updated simultaneously through backpropagation via a multi-task loss function.

[0221] It should be noted that the negative sample generation model only serves as a data source during the training of the driving model and does not participate in gradient updates.

[0222] The input to the driving model includes real-time multi-view images. Kinematic state of the vehicle Navigation commands Expert demonstration trajectory (For example, including the position and heading angle at 8 time steps within the next 4 seconds); hard negative sample trajectory The driving model outputs include: predicted trajectory and outputs from the auxiliary task network (map reconstruction results and / or object detection results).

[0223] The driving model may also include auxiliary task networks (such as map reconstruction networks and object detection networks). The map reconstruction network is used to output semantic and / or vector information of map elements in the driving scene, and the object detection network is used to output the location and / or category information of obstacles in the driving scene to assist feature learning.

[0224] Among them, the trajectory prediction network and the auxiliary task network can share underlying features and improve feature quality through multi-task learning.

[0225] It should be noted that during the training phase of the driving model, its scene encoding network, trajectory prediction network, and auxiliary task network can be updated through backpropagation; the negative sample generation model is trained separately, and its model parameters are fixed after generating hard negative sample trajectories, and it does not participate in the gradient update of the driving model.

[0226] 4. Use or reasoning of driving models: Real-time trajectory planning is performed using a trained driving model, without relying on negative samples to generate the model.

[0227] In this disclosure, the scene encoding network of the driving model includes a visual encoding network and a text encoding network. These two networks can jointly extract scene features of the driving scene. The trajectory prediction network (such as a lightweight multilayer perceptron) within the driving model directly outputs a single-modal predicted trajectory via a single forward propagation, or combines a lightweight scoring method to output a multi-modal predicted trajectory. That is, the final output of the driving model can include the planned final trajectory (a sequence of relative coordinates and heading angles in the vehicle coordinate system).

[0228] The following will provide a detailed explanation of the model's usage process.

[0229] During the model inference phase, the trained driving model can be used to control the vehicle. The driving model can remove the auxiliary task network, retaining only the scene encoding network (visual encoding network + text encoding network) and the trajectory prediction network.

[0230] The input to the driving model is the same as in the training phase: real-time sensor data (including but not limited to: multi-view images, vehicle kinematics, navigation commands, etc.). The output of the driving model is a predicted trajectory. This predicted trajectory is then used as a differential trajectory. For example, you can... The final trajectory is obtained after inverse normalization and coordinate transformation. .

[0231] The decision-making process of the driving model's reasoning is as follows: The first step is for the scene coding network to extract scene fusion features from the input scene information.

[0232] The second step involves the trajectory prediction network outputting the predicted trajectory, such as a differential trajectory, via a single forward propagation. .

[0233] The third step is to divide the differential trajectory By restoring the absolute trajectory coordinates and heading angle, the final trajectory obtained from the planning is obtained.

[0234] The entire inference process requires only one network forward propagation, with computational overhead identical to the baseline model, and no additional sampling or filtering steps.

[0235] In summary, the technical solution provided in this disclosure differs from general end-to-end driving models in at least the following key ways: First, the mechanism for generating active hard negative sample trajectories eliminates the need for additional data acquisition. General end-to-end driving models passively acquire expert demonstration trajectories from datasets, while this disclosure actively constructs hard negative sample trajectories that are spatially close to the expert demonstration trajectory but unsafe through a negative sample generation model. This model incorporates a CFG scale parameter less than 1 and amplified sampling noise, specifically designed to explore unsafe regions near the expert manifold, rather than generating optimal trajectories. In other words, it generates negative sample trajectories that are spatially close to the expert demonstration trajectory but contain unsafe factors. These negative sample trajectories provide fine-grained contrast signals, helping the driving model establish clear safety boundaries.

[0236] Second, comparative safety loss. General end-to-end driving models only use imitation learning loss. To bring the predicted trajectory closer to the expert demonstration trajectory, this disclosure additionally introduces a negative distance loss. This mechanism subjects the driving model to forces in two directions during training: "closer to the expert demonstration trajectory" and "removal from the negative sample trajectory," thereby establishing clear safety boundaries. This comparative mechanism directly addresses the problem of safety asymmetry.

[0237] Third, the gradient of negative sample trajectories is involved. In related technologies, negative sample trajectories are passively filtered only through existing safety scoring functions, and during the optimization process, these negative sample trajectories only indirectly affect the driving model in the form of preference comparison. They are not explicitly constructed as high-quality negative sample trajectories that are "spatially similar but dangerous," nor do they directly affect the gradient update of the driving model. However, in this disclosure, hard negative sample trajectories are directly embedded into the loss function, and their gradients are backpropagated to the driving model, which can improve the driving model's safety decision-making ability.

[0238] Fourth, plug-and-play versatility. This disclosure does not change the original model architecture; it only adds negative distance loss and auxiliary task loss to the loss function, and can be applied to various driving models such as single-modal, multi-modal, and world models.

[0239] As an example of an application scenario, let's take a vehicle approaching a T-junction and needing to turn left. Assume there are no other vehicles in front of the vehicle, but there is a parking space on the right side of the T-junction, and the road boundaries are clear. The expert demonstration trajectory is to decelerate and make a smooth left turn, driving along the center line of the lane.

[0240] Input data includes: multi-view images (scene images taken by the front-view camera showing a T-shaped intersection with clear road boundaries and a parking space on the right that is not occupied by any vehicle); kinematic states (e.g., vehicle speed 8 m / s, acceleration 0 m / s², vehicle position 15 meters in front of the stop line at the intersection); navigation commands (e.g., "turn left").

[0241] The model processing flow mainly includes the following stages: Phase 1: Generating hard negative sample trajectories.

[0242] The negative sample generation model outputs N candidate trajectories, most of which exhibit safe left turns (similar to the expert demonstration trajectory), but some candidate trajectories also exhibit unsafe behavior: for example, candidate trajectory A makes a sharp turn in the center of the intersection, candidate trajectory B crosses the line into the intersection in advance, and candidate trajectory C over-turns and gets too close to the right curb.

[0243] A safety scoring strategy is used to score each candidate trajectory, and the 10 candidate trajectories with the lowest safety scores are selected as an unsafe subset. Then, from the unsafe subset, the candidate trajectory with the closest Euclidean distance to the expert demonstration trajectory is selected as the hard negative sample trajectory. Using the example above, although candidate trajectory A is dangerous, it is spatially closest to the expert demonstration trajectory (the two highly overlap at the beginning of the intersection, only deviating at the end), and therefore is selected as the hard negative sample trajectory.

[0244] Phase Two: Driving Model Training.

[0245] The driving model receives scene information of the current driving scenario and outputs a predicted trajectory, such as a differential trajectory. Calculate the imitation learning loss. At this point, the predicted trajectory may be close to the trajectory demonstrated by the expert, but with a slight deviation.

[0246] Calculate negative distance loss If the predicted trajectory matches the expert demonstration trajectory in the initial section before the intersection, but tends to make sharp turns in the latter part (similar to a negative sample trajectory), then... A large negative value will generate a strong push-away gradient. The total loss is backpropagated, prompting the driving model to adjust its parameters so that the predicted trajectory remains close to the expert demonstration trajectory while avoiding dangerous behaviors similar to the negative sample trajectory.

[0247] Phase 3: Reasoning and application of driving models.

[0248] After training, the driving model predicts a smooth left turn trajectory in the same T-junction driving scenario, which stays in the center of the lane and avoids sharp turns or crossing the line, which can significantly improve the driving model's compliance and collision-free index in the drivable area.

[0249] To implement the above embodiments, this disclosure also proposes a training device for a driving model.

[0250] Figure 8 This is a schematic diagram of the structure of a training device for a driving model provided as an exemplary embodiment of the present disclosure.

[0251] like Figure 8 As shown, the training device 800 for the driving model may include: a first acquisition module 810, a planning module 820, and a first training module 830.

[0252] The first acquisition module 810 is used to acquire first scene information of the first driving scenario, as well as a first labeled trajectory and a negative sample trajectory associated with the first scene information; wherein, the negative sample trajectory is closest to the first labeled trajectory, and the first labeled trajectory has a higher safety level than the negative sample trajectory. The planning module 820 is used to perform trajectory planning based on the first scenario information using a driving model to obtain a predicted trajectory; The first training module 830 is used to train the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

[0253] In one implementation of this disclosure, the negative sample trajectory is obtained using the following module: The first encoding module is used to encode the first scene information using the scene encoding network in the negative sample generation model to obtain the first scene features; The decoding module is used to decode the first scene features based on the first noise trajectory using the decoding network in the negative sample generation model to obtain multiple candidate trajectories; The first determining module is used to determine the negative sample trajectory that is spatially closest to the first labeled trajectory from multiple candidate trajectories.

[0254] In one implementation of this disclosure, the first determining module is configured to: perform a safety assessment on multiple candidate trajectories based on a set safety scoring strategy to obtain a safety score for the multiple candidate trajectories; wherein the safety score is used to indicate the safety level of the vehicle traveling along the candidate trajectories; filter the multiple candidate trajectories to obtain retained trajectories whose safety scores are lower than a set scoring threshold; and determine a negative sample trajectory from the retained trajectories based on the spatial distance between the retained trajectories and the first labeled trajectory.

[0255] In one implementation of this disclosure, the decoding module is used to: amplify a set noise distribution parameter; sample noise based on the amplified noise distribution parameter to obtain a first noise trajectory; and use a decoding network to decode the first scene features based on a set classifier-free guided CFG scale parameter and the first noise trajectory to obtain multiple candidate trajectories; wherein the CFG scale parameter is less than 1.

[0256] In one implementation of this disclosure, the negative sample generation model discards first scene features with a set probability. The decoding module is used to: use a decoding network to determine a first velocity field at each time step based on a first intermediate state and the first scene features at each time step; wherein the first intermediate state is generated based on a first noise trajectory and a first labeled trajectory; use a decoding network to determine a second velocity field at each time step based only on the first intermediate state at each time step; fuse the first velocity field and the second velocity field according to CFG scale parameters to obtain a third velocity field; and generate multiple candidate trajectories based on the third velocity field.

[0257] In one implementation of this disclosure, the negative sample generation model is trained using the following modules: The third acquisition module is used to acquire the second scene information of the second driving scenario, and the second labeled trajectory associated with the second scene information; The second encoding module is used to encode the second scene information using a scene encoding network to obtain the second scene features; The second determining module is used to determine the target velocity field to be fitted by the decoding network based on the difference between the second labeled trajectory and the second noise trajectory. The third determining module is used to determine the fourth velocity field at each time step using a decoding network, based on the second intermediate state and the second scene features at each time step; wherein the second intermediate state is generated based on the second noise trajectory and the second labeled trajectory. The second training module is used to train the negative sample generation model based on the difference between the fourth velocity field and the target velocity field.

[0258] In one implementation of this disclosure, the scene encoding network includes a first visual encoding network and a first text encoding network. A first encoding module is configured to: use the first visual encoding network to encode visual data from at least one perspective in the first scene information to obtain a first visual feature from a bird's-eye view; use the first text encoding network to encode text data in the first scene information to obtain a first text feature; and fuse the first visual feature and the first text feature to obtain a first scene feature.

[0259] In one implementation of this disclosure, the driving model includes a second visual encoding network, a second text encoding network, and a trajectory prediction network. The planning module 820 is used to: use the second visual encoding network to encode visual data from at least one perspective in the first scene information to obtain second visual features from a bird's-eye view; use the second text encoding network to encode text data in the first scene information to obtain second text features; fuse the second visual features and the second text features to obtain scene fusion features; and use the trajectory prediction network to plan the trajectory based on the scene fusion features to obtain a predicted trajectory.

[0260] In one implementation of this disclosure, the negative sample trajectory is generated using a negative sample generation model based on the first scene information; The second visual encoding network shares model parameters with the first visual encoding network in the negative sample generation model; And / or, The second text encoding network shares model parameters with the first text encoding network in the negative sample generation model.

[0261] In one implementation of this disclosure, the first training module 830 is configured to: determine an imitation learning loss based on a first difference between the predicted trajectory and the first labeled trajectory; wherein the imitation learning loss is positively correlated with the first difference; determine a negative distance loss based on a second difference between the predicted trajectory and the negative sample trajectory; wherein the negative distance loss is negatively correlated with the second difference; and train the driving model based on the imitation learning loss and the negative distance loss.

[0262] In one implementation of this disclosure, the driving model further includes an auxiliary task network; the auxiliary task network is used to perform map reconstruction based on scene fusion features to obtain map reconstruction results, and / or to perform target detection based on scene fusion features to obtain target detection results; Among them, the map reconstruction results are used to indicate the semantic and / or vector information of map elements in the first driving scenario; Among them, the target detection results are used to indicate the location information and / or category information of obstacles in the first driving scenario.

[0263] In one implementation of this disclosure, the first training module 830 is configured to: acquire map annotation information and obstacle annotation information associated with the first scene information; determine the auxiliary task loss based on a third difference between the map annotation information and the map reconstruction result, and / or a fourth difference between the obstacle annotation information and the target detection result; determine the target loss based on the imitation learning loss, the negative distance loss, and the auxiliary task loss; and adjust the model parameters in the driving model based on the target loss.

[0264] It should be noted that the foregoing explanation of the training method embodiment for any driving model also applies to the training device for the driving model of that embodiment, and will not be repeated here.

[0265] In the training device of the driving model in this embodiment, negative sample trajectories that are highly similar to the first labeled trajectory in spatial geometry but have lower safety are introduced (i.e., although the negative sample trajectory is highly similar to the first labeled trajectory in path shape, it has defects in safety dimension, such as intruding into the oncoming lane, deviating from the legal drivable area (such as crossing the solid lane line or driving off the curb), having potential collisions with static obstacles or dynamic traffic participants, or performing illegal operations in the prohibited lane-changing area, etc.). The driving model is compared and optimized by integrating the first labeled trajectory, the negative sample trajectory and the predicted trajectory output by the driving model. This allows the driving model to explicitly learn fine-grained safety decision boundaries around the first labeled trajectory. As a result, the driving model can not only reproduce the real driving behavior corresponding to the first labeled trajectory, but also effectively identify and actively avoid those nearby behaviors that seem reasonable and feasible in spatial geometry but actually have high risks. While maintaining the naturalness and smoothness of the planned trajectory, the safety index is significantly improved and the decision reliability in safety asymmetry scenarios is enhanced.

[0266] To implement the above embodiments, this disclosure also proposes a vehicle control device.

[0267] Figure 9 This is a schematic diagram of the structure of a vehicle control device provided for an exemplary embodiment of the present disclosure.

[0268] like Figure 9 As shown, the vehicle control device 900 may include a second acquisition module 910 and a control module 920.

[0269] The second acquisition module 910 is used to acquire the actual scene information and vehicle information of the driving scenario in which the vehicle is located. The control module 920 is used to control the vehicle based on actual scene information and vehicle information using a driving model; Among them, the driving model is adopted Figure 8 The driving model shown was trained using a training device.

[0270] It should be noted that the foregoing explanation of the vehicle control method embodiment also applies to the vehicle control device of this embodiment, and will not be repeated here.

[0271] In the vehicle control device of this disclosure, the vehicle is controlled by a trained driving model based on multi-source scene information and fine-grained vehicle information. This avoids the accumulation of errors and information loss in the perception-planning-control link in the traditional modular architecture, and significantly improves the real-time response, behavior rationality and operational safety of the intelligent driving system in open road environments.

[0272] To implement the above embodiments, this disclosure also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the driving model training method or vehicle control method as described in any of the foregoing embodiments.

[0273] It should be noted that the foregoing explanations of the training method or vehicle control method embodiment for any driving model also apply to the electronic device of this embodiment, and will not be repeated here.

[0274] Figure 10 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present disclosure. For example, the electronic device 1000 may be a server, a terminal, a vehicle, etc.

[0275] Reference Figure 10 The electronic device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0276] Processing component 1002 typically controls the overall operation of electronic device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.

[0277] The memory 1004 is configured to store various types of data to support the operation of the electronic device 1000. The memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof.

[0278] Power component 1006 provides power to various components of electronic device 1000. Power component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1000.

[0279] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the multimedia component 1008 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0280] Audio component 1010 is configured to output and / or input audio signals. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio signals when electronic device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.

[0281] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc.

[0282] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of electronic device 1000. For example, sensor assembly 1014 can detect the on / off state of electronic device 1000, the relative positioning of components, changes in the position of electronic device 1000 or a component of electronic device 1000, the presence or absence of user contact with electronic device 1000, the orientation or acceleration / deceleration of electronic device 1000, temperature changes of electronic device 1000, the presence of nearby objects, etc.

[0283] Communication component 1016 is configured to facilitate wired or wireless communication between electronic device 1000 and other devices. Electronic device 1000 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 further includes a Near Field Communication (NFC) module to facilitate short-range communication.

[0284] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more chips for performing the above-described method.

[0285] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, which can be executed by a processor 1020 of an electronic device 1000 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0286] To implement the above embodiments, this disclosure also proposes a vehicle, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the vehicle control method as described in any of the foregoing embodiments.

[0287] It should be noted that the foregoing explanation of the vehicle control method embodiment also applies to the vehicle in this embodiment, and will not be repeated here.

[0288] Figure 11 This is a block diagram illustrating a vehicle 1100 according to an exemplary embodiment. (Refer to...) Figure 11The vehicle 1100 may include various subsystems, such as an infotainment system 1110, a perception system 1120, a decision control system 1130, a drive system 1140, and a computing platform 1150. The vehicle 1100 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of the vehicle 1100 can be interconnected via wired or wireless means.

[0289] In some embodiments, the infotainment system 1110 may include a communication system, an entertainment system, and a navigation system, etc.

[0290] The perception system 1120 may include several types of sensors for sensing information about the environment surrounding the vehicle 1100. For example, the perception system 1120 may include a global positioning system (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU), a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device.

[0291] The decision control system 1130 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0292] The drive system 1140 may include components that provide powered motion to the vehicle 1100. In one embodiment, the drive system 1140 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0293] Some or all of the functions of vehicle 1100 are controlled by computing platform 1150. Computing platform 1150 may include at least one processor 1151 and memory 1152, and processor 1151 may execute instructions 1153 stored in memory 1152.

[0294] Processor 1151 can be any conventional processor. Processors may also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0295] The memory 1152 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0296] In addition to instruction 1153, memory 1152 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 1152 can be used by computing platform 1150.

[0297] In this embodiment of the disclosure, processor 1151 may execute instructions 1153 to complete all or part of the steps of any of the above method embodiments.

[0298] To implement the above embodiments, this disclosure also proposes a chip, wherein the chip includes an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is configured to execute the training method of the driving model or the vehicle control method provided in any of the foregoing embodiments.

[0299] It should be noted that the foregoing explanation of the training method or vehicle control method embodiment for any driving model also applies to the chip of this embodiment, and will not be repeated here.

[0300] Figure 12 This is a schematic diagram of the structure of a chip according to an exemplary embodiment of this disclosure. See also... Figure 12 The diagram shown is a schematic representation of the structure of chip 1200, but it is not limited to this.

[0301] Chip 1200 includes processing circuit 1201, which is configured to execute any of the above vehicle control methods or driving model training methods.

[0302] In some embodiments, chip 1200 further includes one or more interface circuits 1202. As one possible implementation, the interface circuit 1202 is connected to memory 1203. The interface circuit 1202 can be used to receive signals from memory 1203 or other devices, and can also be used to send signals to memory 1203 or other devices. For example, the interface circuit 1202 can read instructions stored in memory 1203 and send those instructions to processing circuit 1201.

[0303] In some embodiments, the interface circuit 1202 performs at least one of the communication steps such as sending and / or receiving in the above method, while the processing circuit 1201 performs other steps.

[0304] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc., can be used interchangeably.

[0305] In some embodiments, chip 1200 further includes one or more memories 1203 for storing instructions. Optionally, all or part of the memories 1203 may be located outside of chip 1200.

[0306] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the driving model training method or vehicle control method as described in any of the foregoing method embodiments.

[0307] It should be noted that the foregoing explanations of the training method or vehicle control method embodiments for any driving model also apply to the non-transitory computer-readable storage medium of the embodiments, and will not be repeated here.

[0308] To implement the above embodiments, this disclosure also proposes a computer program product having a computer program stored thereon, which, when executed by a processor, implements the driving model training method or vehicle control method as described in any of the foregoing method embodiments.

[0309] It should be noted that the foregoing explanations of the training method or vehicle control method embodiment for any driving model also apply to the computer program product of that embodiment, and will not be repeated here.

[0310] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0311] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0312] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0313] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and compact disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0314] It should be understood that various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0315] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0316] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0317] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for training a driving model, characterized in that, include: Obtain first scene information of a first driving scenario, as well as a first labeled trajectory and a negative sample trajectory associated with the first scene information; wherein, the negative sample trajectory is closest in space to the first labeled trajectory, and the first labeled trajectory has a higher safety level than the negative sample trajectory; A driving model is used to plan the trajectory based on the first scenario information to obtain the predicted trajectory; The driving model is trained based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

2. The method according to claim 1, characterized in that, The methods for obtaining the negative sample trajectory include: The first scene information is encoded using the scene encoding network in the negative sample generation model to obtain the first scene features; The decoding network in the negative sample generation model is used to decode the first scene features based on the first noise trajectory to obtain multiple candidate trajectories; From the multiple candidate trajectories, determine the negative sample trajectory that is spatially closest to the first labeled trajectory.

3. The method according to claim 2, characterized in that, The step of determining the negative sample trajectory that is spatially closest to the first labeled trajectory from the plurality of candidate trajectories includes: Based on the established safety scoring strategy, the safety of the multiple candidate trajectories is evaluated to obtain a safety score for the multiple candidate trajectories; wherein, the safety score is used to indicate the degree of safety of the vehicle traveling along the candidate trajectory; The multiple candidate trajectories are filtered to obtain the retained trajectories whose safety scores are lower than a set score threshold; The negative sample trajectory is determined from the retained trajectory based on the spatial distance between the retained trajectory and the first labeled trajectory.

4. The method according to claim 2, characterized in that, The decoding network in the negative sample generation model decodes the first scene features based on the first noise trajectory to obtain multiple candidate trajectories, including: The set noise distribution parameters are amplified. The first noise trajectory is obtained by sampling the noise based on the amplified noise distribution parameters. The decoding network is used to decode the first scene features based on the set classifier-free guided CFG scale parameter and the first noise trajectory to obtain the multiple candidate trajectories; wherein the CFG scale parameter is less than 1.

5. The method according to claim 4, characterized in that, The negative sample generation model discards the first scene features with a set probability. The decoding network decodes the first scene features based on a set classifier-free guided CFG scale parameter and the first noise trajectory to obtain multiple candidate trajectories, including: Using the decoding network, a first velocity field is determined at each time step based on a first intermediate state and the first scene features; wherein, the first intermediate state is generated based on the first noise trajectory and the first labeled trajectory; Using the aforementioned decoding network, the second velocity field at each time step is determined solely based on the first intermediate state at each time step; Based on the CFG scale parameters, the first velocity field and the second velocity field are fused to obtain the third velocity field; The multiple candidate trajectories are generated based on the third velocity field.

6. The method according to any one of claims 2-5, characterized in that, The training methods for the negative sample generation model include: Obtain second scene information of the second driving scenario, and second labeled trajectory associated with the second scene information; The second scene information is encoded using the scene encoding network to obtain the second scene features, and the target velocity field to be fitted by the decoding network is determined based on the difference between the second labeled trajectory and the second noise trajectory. Using the decoding network, a fourth velocity field is determined at each time step based on the second intermediate state and the second scene features; wherein the second intermediate state is generated based on the second noise trajectory and the second labeled trajectory; The negative sample generation model is trained based on the difference between the fourth velocity field and the target velocity field.

7. The method according to claim 2, characterized in that, The scene encoding network includes a first visual encoding network and a first text encoding network. The scene encoding network in the negative sample generation model encodes the first scene information to obtain first scene features, including: The first visual coding network is used to encode visual data from at least one perspective in the first scene information to obtain the first visual features of the bird's-eye view. The first text encoding network is used to encode the text data in the first scene information to obtain the first text feature; The first visual feature and the first text feature are fused to obtain the first scene feature.

8. The method according to claim 1, characterized in that, The driving model includes a second visual encoding network, a second text encoding network, and a trajectory prediction network. The step of using the driving model to perform trajectory planning based on the first scene information to obtain a predicted trajectory includes: The second visual coding network is used to encode the visual data of at least one perspective in the first scene information to obtain the second visual features of the bird's-eye view. The second text encoding network is used to encode the text data in the first scene information to obtain the second text features; The second visual feature and the second text feature are fused to obtain the scene fusion feature; The trajectory prediction network is used to perform trajectory planning based on the scene fusion features to obtain the predicted trajectory.

9. The method according to claim 8, characterized in that, The negative sample trajectory is generated using a negative sample generation model based on the first scene information; The second visual encoding network shares model parameters with the first visual encoding network in the negative sample generation model; And / or, The second text encoding network shares model parameters with the first text encoding network in the negative sample generation model.

10. The method according to claim 1, characterized in that, The step of training the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory includes: Based on the first difference between the predicted trajectory and the first labeled trajectory, an imitation learning loss is determined; wherein the imitation learning loss is positively correlated with the first difference; A negative distance loss is determined based on a second difference between the predicted trajectory and the negative sample trajectory; wherein the negative distance loss is negatively correlated with the second difference. The driving model is trained based on the imitation learning loss and the negative distance loss.

11. The method according to claim 10, characterized in that, The driving model also includes an auxiliary task network; the auxiliary task network is used to perform map reconstruction based on the scene fusion features to obtain map reconstruction results, and / or to perform target detection based on the scene fusion features to obtain target detection results; The map reconstruction result is used to indicate the semantic information and / or vector information of map elements in the first driving scenario; The target detection result is used to indicate the location and / or category information of obstacles in the first driving scenario.

12. The method according to claim 11, characterized in that, The step of training the driving model based on the imitation learning loss and the negative distance loss includes: Obtain map annotation information and obstacle annotation information associated with the first scene information; The auxiliary task loss is determined based on the third difference between the map annotation information and the map reconstruction result, and / or the fourth difference between the obstacle annotation information and the target detection result; The target loss is determined based on the imitation learning loss, the negative distance loss, and the auxiliary task loss; Based on the target loss, the model parameters in the driving model are adjusted.

13. A vehicle control method, characterized in that, include: Obtain actual scene information and vehicle information of the driving scenario in which the vehicle is located; The vehicle is controlled using a driving model based on the actual scene information and the vehicle information; The driving model is trained using the method described in any one of claims 1-12.

14. A training device for a driving model, characterized in that, include: The first acquisition module is used to acquire first scene information of a first driving scenario, as well as a first labeled trajectory and a negative sample trajectory associated with the first scene information; wherein, the negative sample trajectory is closest in space to the first labeled trajectory, and the first labeled trajectory has a higher safety level than the negative sample trajectory; The planning module is used to perform trajectory planning based on the first scenario information using a driving model to obtain a predicted trajectory; The first training module is used to train the driving model based on the predicted trajectory, the first labeled trajectory, and the negative sample trajectory.

15. The apparatus according to claim 14, characterized in that, The first training module is used for: Based on the first difference between the predicted trajectory and the first labeled trajectory, an imitation learning loss is determined; wherein the imitation learning loss is positively correlated with the first difference; A negative distance loss is determined based on a second difference between the predicted trajectory and the negative sample trajectory; wherein the negative distance loss is negatively correlated with the second difference. The driving model is trained based on the imitation learning loss and the negative distance loss.

16. The apparatus according to claim 15, characterized in that, The driving model also includes an auxiliary task network; the auxiliary task network is used to perform map reconstruction based on the scene fusion features to obtain map reconstruction results, and / or to perform target detection based on the scene fusion features to obtain target detection results; The map reconstruction result is used to indicate the semantic information and / or vector information of map elements in the first driving scenario; The target detection result is used to indicate the location and / or category information of obstacles in the first driving scenario.

17. The apparatus according to claim 16, characterized in that, The first training module is used for: Obtain map annotation information and obstacle annotation information associated with the first scene information; The auxiliary task loss is determined based on the third difference between the map annotation information and the map reconstruction result, and / or the fourth difference between the obstacle annotation information and the target detection result; The target loss is determined based on the imitation learning loss, the negative distance loss, and the auxiliary task loss; Based on the target loss, the model parameters in the driving model are adjusted.

18. A vehicle control device, characterized in that, include: The second acquisition module is used to acquire actual scene information and vehicle information of the driving scenario in which the vehicle is located. The control module is used to control the vehicle using a driving model based on the actual scene information and the vehicle information; The driving model is trained using the device described in any one of claims 14-17.

19. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method as claimed in any one of claims 1 to 12, and / or implements the steps of the method as claimed in claim 13.

20. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Implement the steps of the method as described in claim 13.

21. A non-transitory computer-readable storage medium storing computer program instructions thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method according to any one of claims 1 to 12, and / or implement the steps of the method according to claim 13.

22. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 12, and / or implements the steps of the method according to claim 13.

23. A chip, characterized in that, The chip includes an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is used to implement the method of any one of claims 1 to 12, and / or to implement the steps of the method of claim 13.

Images