Vehicle trajectory prediction and collision warning method based on large model counterfactual distillation

Abstract

This invention discloses a vehicle trajectory prediction and collision warning method based on counterfactual distillation of a large model, relating to the fields of computer vision, large language models, and intelligent transportation technologies. The technical solution includes the following steps: Step 1: Multi-source data acquisition and spatiotemporal sequence construction; Step 2: Offline multimodal semantic reasoning; Step 3: Online physical feature encoding and intent adaptation; Step 4: Online feature fusion and trajectory decoding; Step 5: Multi-task joint loss design and model distillation training; Step 6: Collision warning mechanism and output display. This invention utilizes the counterfactual reasoning capability of a large model to compensate for the shortcomings of high-risk data, and through intent adaptation and semantic consistency loss functions, efficiently transfers the cognitive logic of the large model to a lightweight edge network. Thus, in the online reasoning stage, high-accuracy, robust, and intent-interpretable collision warning and trajectory prediction can be achieved solely based on physical features.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, large language models and intelligent transportation technologies, and in particular to a method for vehicle trajectory prediction and collision warning based on counterfactual distillation of large models. Background Technology

[0002] With the rapid development of autonomous driving technology and intelligent transportation systems, the ability of vehicles to autonomously perceive and warn of risks in dynamic environments has become a key factor in ensuring road safety. Especially in the context of increasingly complex urban traffic, accurately identifying potential collision risks and providing early warnings has become a core technical challenge in the design of autonomous driving systems. Traditional collision prediction systems mostly rely on radar or a single vision sensor, typically using rule-based collision determination algorithms or simple physical trajectory extrapolation methods for risk assessment. While these methods are structurally simple, they generally suffer from poor adaptability to dynamic environments and weak target interaction recognition capabilities, making it difficult to meet the requirements for warning accuracy and lead time in complex traffic scenarios.

[0003] In recent years, with the widespread application of deep learning technology, data-driven trajectory prediction and collision risk modeling have made some progress. Most existing methods utilize temporal models (such as RNNs, LSTMs, or graph neural networks) to extract kinematic features from the vehicle's historical coordinate sequences for prediction. However, purely physical geometric data often fails to fully characterize the deep semantic information of traffic scenarios (such as right-of-way dynamics, driver attention states, and potential interaction intentions). When faced with highly unstructured and complex interaction scenarios such as "ghost pedestrians" and "forced lane changes," purely physical prediction models lacking semantic support are prone to prediction bias, leading to a lack of logical consistency in prediction results and a significant reduction in warning accuracy.

[0004] Meanwhile, Multimodal Large Language Models (MLLMs), with their vast prior knowledge and superior generalization reasoning capabilities, are beginning to demonstrate significant potential in semantic understanding and causal analysis of traffic scenarios. Some studies have attempted to introduce large models for post-event risk assessment and scenario analysis of traffic video streams. However, existing technologies face insurmountable gaps when it comes to real-time collision avoidance warnings: the massive number of parameters in large models results in extremely high inference latency (typically on the order of seconds) and consumes enormous computing resources, making it impossible to directly deploy them on resource-constrained vehicle-side edge computing devices to meet the millisecond-level response requirements of autonomous driving. Furthermore, real-world traffic accident data exhibits an extreme "long-tail distribution," and existing lightweight edge-side models, lacking sufficient high-risk collision samples for training, have extremely weak defensive generalization capabilities when facing rare and dangerous scenarios.

[0005] Existing technologies typically rely solely on the vehicle's historical physical trajectory (such as coordinates and speed) for extrapolation prediction, lacking semantic understanding of the deeper logic behind complex traffic scenarios (such as right-of-way games and target interaction intentions). When faced with complex long-tail interaction scenarios such as "ghost pedestrians" and "forced lane changes," the prediction accuracy and early warning lead time are significantly insufficient.

[0006] In recent years, although multimodal large language models (MLLMs) have demonstrated outstanding capabilities in semantic understanding and risk assessment in traffic scenarios, their inference latency is extremely high and their computational consumption is enormous, making it difficult to directly deploy them on vehicle-side edge computing devices to meet the millisecond-level real-time warning requirements. Furthermore, real-world traffic accident data exhibits an extreme "long-tail distribution," and existing models, lacking training with high-risk collision samples, struggle to cope with extreme road conditions. Moreover, traditional prediction networks are mostly "black box" models, only outputting coordinate sequences, lacking effective alignment between physical trajectories and high-level semantic intent, resulting in prediction results that lack logical consistency and interpretability.

[0007] Therefore, how to build a real-time vehicle collision prediction system that can have deep semantic cognition and counterfactual reasoning capabilities at the level of large models, break through the computing power bottleneck to meet the ultra-low latency requirements of the vehicle end, and effectively cope with long-tail high-risk scenarios remains an important technical problem that needs to be solved in this field. Summary of the Invention

[0008] The purpose of this invention is to provide a vehicle trajectory prediction and collision warning method based on counterfactual distillation of a large model. By constructing a knowledge distillation architecture that combines "offline multimodal semantic reasoning" and "online real-time prediction", the counterfactual reasoning capability of the large model is used to make up for the shortcomings of high-risk data. Through intent adaptation and semantic consistency loss function, the cognitive logic of the large model is efficiently transferred to a lightweight edge network. Thus, in the online reasoning stage, collision warning and trajectory prediction with high accuracy, strong robustness and intent interpretability can be achieved based solely on physical features.

[0009] The inventive concept of this invention is as follows: This method comprehensively utilizes various artificial intelligence technologies, including Visual Prompting, counterfactual causal reasoning of Multimodal Large Language Modeling (MLLM), spatiotemporal sequence modeling, and cross-modal feature mapping. By constructing a knowledge distillation dual-stream architecture of "offline multimodal large model - online lightweight edge network," the deep semantic understanding and intent recognition capabilities of the large model for long-tail high-risk scenarios are efficiently transferred to the edge-side lightweight prediction network. This allows for real-time and accurate prediction of potential collision events and vehicle interaction intentions in complex traffic scenarios without increasing vehicle-side inference latency. This technology can be widely applied in Advanced Driver Assistance Systems (ADAS), high-level autonomous driving systems, intelligent connected vehicles (ICV), and vehicle-road cooperative monitoring to achieve forward collision warning, dynamic risk perception, and highly reliable driving decision assistance in long-tail scenarios.

[0010] To achieve the aforementioned objectives, the present invention employs the following technical solution: a vehicle trajectory prediction and collision warning method based on large-model counterfactual distillation. The system architecture is divided into an "offline training phase" and an "online inference phase," and the specific execution steps are as follows: Step 1: Multi-source data acquisition and spatiotemporal sequence construction: The system connects to an onboard sensor platform (including cameras and vehicle chassis pose sensors / radar) to synchronously acquire continuous traffic video streams and historical trajectory coordinates of the vehicle and surrounding traffic participants.

[0011] Set the time window length to Extract the target vehicle at the current moment Historical two-dimensional coordinate sequence over a period of time Simultaneously, the system extracts key video frame sequences of the current scene from the video stream at a set frame rate. It serves as the input basis for offline multimodal semantic reasoning.

[0012] Step 2: Offline multimodal semantic reasoning: This step, performed offline in the cloud or on a high-performance training server, aims to extract deep causal cognitions from complex scenes using large models. It specifically includes the following sub-steps: 2.1) Visual Enhancement and Object Anchoring: Extracting Key Video Frames By utilizing open-vocabulary object detection models (such as Grounding DINO) and image segmentation models (such as SAM), the bounding boxes and instance contours (masks) of target vehicles and other traffic participants in the scene are accurately extracted. In the original image... Draw colored polygonal outlines on the surface and assign unique text identifiers (such as Veh-A, Ped-B) to each object, generating enhanced images with visual anchor points. This step addresses the "visual illusion" problem that large models are prone to in complex traffic scenarios.

[0013] 2.2) Counterfactual reasoning and risk intent generation: Enhance the image Input is taken into a multimodal large language model (MLLM, such as GPT-4V) and a carefully crafted "counterfactual prompt" is injected. The prompt template contains question constraints, such as: "Observe the current pose of Veh-A, assuming it maintains its current acceleration and ignoring right-of-way constraints, infer its future." "The trajectory of motion in seconds and the potential collision risk." MLLM, based on its vast common-sense priors, outputs structured natural language descriptive text. The content covers the discrete intentions of vehicles (such as "forced lane change" and "ghosting") and causal explanations.

[0014] 2.3) Semantic Feature Encoding: Using pre-trained text encoding networks (such as BERT), natural language descriptions are encoded. Mapped to a high-dimensional, continuous latent semantic space. The output of its [CLS] token is extracted as a global semantic representation, generating a dimension of... Teacher semantic vector This vector highly condenses the large model's ultimate understanding of the potential risks in the current scenario, and can be represented as:

[0015] Step 3: Online physical feature encoding and intent adaptation: This step is performed in real time on the vehicle-side edge computing device, completely eliminating the dependence on high-latency video streams and MLLM.

[0016] 3.1) Physical Feature Encoding: The historical trajectory coordinate sequence obtained in step 1 The input is fed into a temporal network based on LSTM or VectorNet to extract the vehicle's kinematic geometry features, with an output dimension of [missing information]. physical feature vector .

[0017] 3.2) Intent Feature Adaptation: A lightweight Intent Adapter module is constructed, consisting of a Multilayer Perceptron (MLP). Its core task is to rely on only a single physical feature. It uses nonlinear mapping to "guess" and reconstruct the semantic features generated by a large model. Its output is a predicted semantic vector. The dimension is also The calculation process is expressed as follows:

[0018] This module enables the edge-side lightweight network to achieve cross-modal intent inference capabilities.

[0019] Step 4: Online Feature Fusion and Trajectory Decoding After generating physical and semantic representations, the system integrates the physical feature vectors in the feature fusion unit. With predicted semantic vectors Concatenation is performed along the channel dimension to obtain a multimodal comprehensive feature representation. :

[0020] Then Input to the multi-head trajectory decoding module (Trajectory Decoder), output future Predicted trajectory coordinates of the frame And the overall collision risk probability of the current scenario.

[0021] Step 5: Multi-task joint loss design and model distillation training To ensure that student networks effectively absorb knowledge from teacher networks, this system incorporates a dual-monitoring mechanism during the training phase: 5.1) Semantic consistency loss (knowledge distillation loss) ): Cosine Embedding Loss is used to measure the predicted semantic vectors. With teacher semantic vector Similarity in the latent space. This loss term generates gradient backpropagation, specifically used to optimize the intent-fitting module, forcing it to learn the inference logic of MLLM:

[0022] 5.2) Trajectory Regression Loss ( ): Calculate the predicted trajectory points using the Smooth L1 loss function With the actual future trajectory The average displacement error (ADE) and final displacement error (FDE) between these factors ensure the accuracy of predictions at the spatial geometry level.

[0023] 5.3) Joint Optimization Strategy: Final overall loss function for:

[0024] in The distillation hyperparameters are dynamically adjustable. During training, the parameters of the large model and BERT are frozen, and only the network weights of the physical feature encoder, intent adaptation module, and trajectory decoder are updated.

[0025] Step 6: Collision Warning Mechanism and Output Display In actual inference deployment, the system disables the offline link in step 2 and the loss calculation module in step 5. Through extremely fast forward propagation in steps 3 and 4 (single frame time <30ms), the future trajectory distribution of the target vehicle is output in real time. When the trajectory decoding module outputs the collision risk probability... When the set safety threshold is exceeded, or when the predicted trajectory point interferes with the planned trajectory of the vehicle, the system immediately triggers a visual / auditory warning and sends the risk information to the autonomous driving decision-making and planning module to achieve millisecond-level active safety defense.

[0026] Meanwhile, the present invention proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed, it implements the steps of the method described in the present invention.

[0027] Furthermore, the present invention proposes a computer-readable storage medium having a computer program stored thereon, the computer program being configured to implement the steps of the method described in the present invention when invoked by a processor.

[0028] Finally, the present invention provides a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described in the present invention.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs a dual-stream architecture of "offline semantic reasoning - online real-time prediction". During the online reasoning phase, the system completely eliminates its reliance on high-performance multimodal models and massive video streams, requiring only a lightweight intent adaptation module to process the vehicle's physical trajectory features. This allows the system to possess "intelligence" at the level of large models while reducing single-frame reasoning time to milliseconds (<30ms), perfectly meeting the stringent real-time requirements of vehicle-side active safety systems (such as AEB).

[0030] 2. To address the extreme scarcity of real traffic accident data, this invention introduces a counterfactual reasoning mechanism based on MLLM. By using visual cues to guide the large model in extrapolating the potential risks of "failure to take evasive action," the system generates a large number of high-quality "virtual high-risk collision intentions" and risk labels. This generative cognitive enhancement mechanism significantly improves the model's defensive predictive ability and robustness in the face of rare and extreme scenarios (corner cases) such as "ghost pedestrians" and "forced lane changes."

[0031] 3. This invention introduces a "semantic consistency loss function" to force the edge intent adaptation module to align with the teacher semantic vectors of the large model in the latent space. This allows the system output not only the predicted trajectory but also an implicit understanding of the target vehicle's interaction intent (e.g., aggressive lane-changing, distracted departure). When the system triggers a collision warning, it can trace back its underlying semantic logic, greatly enhancing the trustworthiness of human-computer interaction and the interpretability of the decision-making and planning module.

[0032] 4. The modular decoupling design of this invention has extremely high engineering application value. The complex visual enhancement (contour extraction + ID annotation) and natural language feature encoding processes are all confined to the offline training stage (cloud); while in the deployment stage (vehicle-side), only the physical feature encoder, MLP intent adapter, and trajectory decoder need to be retained. This cross-modal knowledge distillation solution can achieve the decentralization of large model capabilities without the need to install expensive AI computing chips on the vehicle side, greatly reducing the hardware costs of pre-installed / aftermarket installations in smart cars.

[0033] 5. This invention integrates advanced traffic semantics (such as right-of-way games and driving styles) into physical features through dimensionality reduction. In complex structured or unstructured environments such as multi-vehicle dense games and occlusion crossings, the model can anticipate the other party's unreasonable intentions, effectively avoiding the defects of traditional models such as "predicted trajectory crossing" or "missed or false alarms", and significantly improving the advance warning and spatial positioning accuracy of collision warnings.

[0034] 6. This invention proposes a vehicle trajectory prediction and collision warning method based on counterfactual distillation of large models. In the scenario of autonomous driving and advanced driver assistance systems (ADAS), this method creatively combines the deep semantic cognition capability of multimodal large language models (MLLM) with the low latency characteristics of lightweight edge networks, and achieves efficient, accurate and interpretable prediction of complex traffic interaction behaviors. Attached Figure Description

[0035] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0036] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0038] Example 1: See Figure 1 This embodiment provides a technical solution for vehicle trajectory prediction and collision warning based on large model counterfactual distillation. The system architecture is divided into an "offline training phase" and an "online inference phase," and the specific execution steps are as follows: Step 1: Multi-source data acquisition and spatiotemporal sequence construction: The system connects to an onboard sensor platform (including cameras and vehicle chassis pose sensors / radar) to synchronously acquire continuous traffic video streams and historical trajectory coordinates of the vehicle and surrounding traffic participants.

[0039] Set the time window length to Extract the target vehicle at the current moment Historical two-dimensional coordinate sequence over a period of time Simultaneously, the system extracts key video frame sequences of the current scene from the video stream at a set frame rate. It serves as the input basis for offline multimodal semantic reasoning.

[0040] Step 2: Offline multimodal semantic reasoning: This step, performed offline in the cloud or on a high-performance training server, aims to extract deep causal cognitions from complex scenes using large models. It specifically includes the following sub-steps: 2.1) Visual Enhancement and Object Anchoring: Extracting Key Video Frames By utilizing open-vocabulary object detection models (such as Grounding DINO) and image segmentation models (such as SAM), the bounding boxes and instance contours (masks) of target vehicles and other traffic participants in the scene are accurately extracted. In the original image... Draw colored polygonal outlines on the surface and assign unique text identifiers (such as Veh-A, Ped-B) to each object, generating enhanced images with visual anchor points. This step addresses the "visual illusion" problem that large models are prone to in complex traffic scenarios.

[0041] 2.2) Counterfactual reasoning and risk intent generation: Enhance the image Input is taken into a multimodal large language model (MLLM, such as GPT-4V) and a carefully crafted "counterfactual prompt" is injected. The prompt template contains question constraints, such as: "Observe the current pose of Veh-A, assuming it maintains its current acceleration and ignoring right-of-way constraints, infer its future." "The trajectory of motion in seconds and the potential collision risk." MLLM, based on its vast common-sense priors, outputs structured natural language descriptive text. The content covers the discrete intentions of vehicles (such as "forced lane change" and "ghosting") and causal explanations.

[0042] 2.3) Semantic Feature Encoding: Using pre-trained text encoding networks (such as BERT), natural language descriptions are encoded. Mapped to a high-dimensional, continuous latent semantic space. The output of its [CLS] token is extracted as a global semantic representation, generating a dimension of... Teacher semantic vector This vector highly condenses the large model's ultimate understanding of the potential risks in the current scenario, and can be represented as:

[0043] Step 3: Online physical feature encoding and intent adaptation: This step is performed in real time on the vehicle-side edge computing device, completely eliminating the dependence on high-latency video streams and MLLM.

[0044] 3.1) Physical Feature Encoding: The historical trajectory coordinate sequence obtained in step 1 The input is fed into a temporal network based on LSTM or VectorNet to extract the vehicle's kinematic geometry features, with an output dimension of [missing information]. physical feature vector .

[0045] 3.2) Intent Feature Adaptation: A lightweight Intent Adapter module is constructed, consisting of a Multilayer Perceptron (MLP). Its core task is to rely on only a single physical feature. It uses nonlinear mapping to "guess" and reconstruct the semantic features generated by a large model. Its output is a predicted semantic vector. The dimension is also The calculation process is expressed as follows:

[0046] This module enables the edge-side lightweight network to achieve cross-modal intent inference capabilities.

[0047] Step 4: Online Feature Fusion and Trajectory Decoding After generating physical and semantic representations, the system integrates the physical feature vectors in the feature fusion unit. With predicted semantic vectors Concatenation is performed along the channel dimension to obtain a multimodal comprehensive feature representation. :

[0048] Then Input to the multi-head trajectory decoding module (Trajectory Decoder), output future Predicted trajectory coordinates of the frame And the overall collision risk probability of the current scenario.

[0049] Step 5: Multi-task joint loss design and model distillation training To ensure that student networks effectively absorb knowledge from teacher networks, this system incorporates a dual-monitoring mechanism during the training phase: 5.1) Semantic consistency loss (knowledge distillation loss) ): Cosine Embedding Loss is used to measure the predicted semantic vectors. With teacher semantic vector Similarity in the latent space. This loss term generates gradient backpropagation, specifically used to optimize the intent-fitting module, forcing it to learn the inference logic of MLLM:

[0050] 5.2) Trajectory Regression Loss ( ): Calculate the predicted trajectory points using the Smooth L1 loss function With the actual future trajectory The average displacement error (ADE) and final displacement error (FDE) between these factors ensure the accuracy of predictions at the spatial geometry level.

[0051] 5.3) Joint Optimization Strategy: Final overall loss function for:

[0052] in The distillation hyperparameters are dynamically adjustable. During training, the parameters of the large model and BERT are frozen, and only the network weights of the physical feature encoder, intent adaptation module, and trajectory decoder are updated.

[0053] Step 6: Collision Warning Mechanism and Output Display In actual inference deployment, the system disables the offline link in step 2 and the loss calculation module in step 5. Through extremely fast forward propagation in steps 3 and 4 (single frame time <30ms), the future trajectory distribution of the target vehicle is output in real time. When the trajectory decoding module outputs the collision risk probability... When the set safety threshold is exceeded, or when the predicted trajectory point interferes with the planned trajectory of the vehicle, the system immediately triggers a visual / auditory warning and sends the risk information to the autonomous driving decision-making and planning module to achieve millisecond-level active safety defense.

[0054] Example 2: This example proposes an electronic system, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method steps of the present invention.

[0055] Example 3: This example proposes a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the method described in this invention, which will not be repeated here.

[0056] Example 4: This example proposes a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, they implement the steps of the method described in this invention, which will not be repeated here.

[0057] It should be noted that the processing flow of embodiments 2-4 corresponds to the specific steps of the method provided in embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the method provided in embodiment 1 of the present invention.

[0058] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vehicle trajectory prediction and collision warning method based on large model counterfactual distillation, characterized in that, Includes the following steps: Step 1: Multi-source data acquisition and spatiotemporal sequence construction The system connects to an onboard sensor platform to synchronously acquire continuous traffic video streams and historical trajectory coordinates of the vehicle and surrounding traffic participants; Set the time window length to Extract the target vehicle at the current moment Historical two-dimensional coordinate sequence over a period of time The system extracts key video frame sequences of the current scene from the video stream at a set frame rate. , serving as the input basis for offline multimodal semantic reasoning; Step 2: Offline Multimodal Semantic Reasoning This step is performed offline in the cloud or on a high-performance training server, using large models to extract deep causal cognition of complex scenes; Step 3: Online physical feature encoding and intent adaptation: This step is performed in real time on the vehicle-side edge computing device; Step 4: Online Feature Fusion and Trajectory Decoding After generating physical and semantic representations, the system integrates the physical feature vectors in the feature fusion unit. With predicted semantic vectors By cascading and stitching along the channel dimension, a multimodal comprehensive feature representation is obtained. : ; Will Input to the multi-head trajectory decoding module, output future Predicted trajectory coordinates of the frame And the overall collision risk probability in the current scenario; Step 5: Design of multi-task joint loss and model distillation training; Step 6: Collision warning mechanism and output display.

2. The vehicle trajectory prediction and collision warning method based on large model counterfactual distillation according to claim 1, characterized in that, Step 2 includes: 2.1) Visual Enhancement and Object Anchoring Extracting key video frames Using an open-vocabulary object detection model and an image segmentation model, bounding boxes and instance contours of target vehicles and other traffic participants in a scene are extracted from the original image. Draw colored polygonal outlines on the surface and assign a unique text identifier to each object to generate an enhanced image with visual anchor points. ; 2.2) Counterfactual reasoning and risk intent generation Enhance the image Input at most a multimodal large language model (MLLM) and inject "counterfactual cue words". The cue word template contains question constraints, and the output is a structured natural language description text. The content covers the discrete intentions and causal explanations of vehicles; 2.3) Semantic Feature Encoding Using a pre-trained text encoding network, natural language descriptions are... Mapped to a high-dimensional continuous latent semantic space, the output of its token is extracted as a global semantic representation, generating a dimension of... Teacher semantic vector This vector encapsulates the large model's ultimate understanding of the potential risks in the current scenario, and is represented as: 。 3. The vehicle trajectory prediction and collision warning method based on large model counterfactual distillation according to claim 1, characterized in that, Step 3 includes: 3.1) Physical Feature Encoding: The historical trajectory coordinate sequence obtained in step 1 The input is fed into a temporal network based on Long Short-Term Memory (LSTM) or VectorNet to extract the vehicle's kinematic geometry features, with an output dimension of [missing information]. physical feature vector ; 3.2) Intent Feature Adaptation: A lightweight intent adaptation module is constructed, consisting of a multilayer perceptron (MLP), whose task is to adapt a single physical feature. It uses nonlinear mapping to "guess" and reconstruct the semantic features generated by a large model, and its output is a predicted semantic vector. The dimension is also The calculation process is expressed as follows: Through this module, the lightweight edge network gains cross-modal intent inference capabilities.

4. The vehicle trajectory prediction and collision warning method based on large model counterfactual distillation according to claim 1, characterized in that, Step 5 includes: 5.1) semantic consistency loss, knowledge distillation loss : Predicting semantic vectors using cosine embedding loss metric With teacher semantic vector The similarity in the latent space generates gradient backpropagation, which is used to optimize the intent-fitting module, which learns the inference logic of the multimodal large language model (MLLM). ； 5.2) Trajectory Regression Loss Calculate the predicted trajectory points using the Smooth L1 loss function. With the actual future trajectory The average displacement error ADE and the final displacement error FDE between them, and the prediction accuracy at the spatial geometry level: In the formula, and , They represent the first The predicted trajectory points and the actual trajectory points of the frame; 5.3) Joint Optimization Strategy: Final overall loss function For: ; wherein is a dynamically adjustable distillation hyperparameter, during training, the large model and BERT parameters are frozen, and the network weights of the physical feature encoder, intent adaptation module, and trajectory decoder are updated.

5. The vehicle trajectory prediction and collision warning method based on large model counterfactual distillation according to claim 1, characterized in that, In step 6, during actual inference deployment, the system shuts down the offline link in step 2 and the loss calculation module in step 5. Through the extremely fast forward propagation of steps 3 and 4, with a single frame latency of less than 30ms, the system outputs the future trajectory distribution of the target vehicle in real time. When the trajectory decoding module outputs the collision risk probability... When the set safety threshold is exceeded, or when the predicted trajectory point interferes with the planned trajectory of the vehicle, the system immediately triggers a visual / auditory warning and sends the risk information to the autonomous driving decision-making and planning module to achieve millisecond-level active safety defense.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1 to 5.

7. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is configured to implement the steps of the method according to any one of claims 1 to 5 when invoked by a processor.

8. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

Images