A vehicle collision mitigation method and system based on multimodal temporal feature fusion and future risk state prediction

HK30137555AActive Publication Date: 2026-09-11HONG KONG PRODUCTIVITY COUNCIL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
HK32026124474
Authority / Receiving Office
HK · HK
Patent Type
Applications
Current Assignee / Owner
Filing Date
2026-06-08
Publication Date
2026-09-11

Smart Images

  • Figure 00000025_0000
    Figure 00000025_0000
  • Figure 00000026_0000
    Figure 00000026_0000
  • Figure 00000027_0000
    Figure 00000027_0000
Patent Text Reader

Abstract

This Hong Kong short-term application relates to the field of intelligent driving technology, specifically disclosing a vehicle collision mitigation method and system based on multimodal temporal feature fusion and future risk state prediction. The method includes: acquiring multimodal perception data of the vehicle's forward environment and vehicle state data to establish temporal samples of the multimodal perception data; extracting single-modal features separately and mapping them to a unified feature space; performing feature-level multimodal temporal fusion using a cross-modal attention mechanism to obtain fused temporal features; inputting the fused temporal features into an encoder to model the historical temporal context and outputting a temporal feature sequence encoded from the historical temporal sequence; inputting the temporal feature sequence encoded from the historical temporal sequence into a decoder to output a future collision risk-related state sequence; generating graded braking control commands based on the collision risk-related state sequence and vehicle state data; and outputting the graded braking control commands to the vehicle control unit.
Need to check novelty before this filing date? Find Prior Art

Description

1. Description of a Vehicle Collision Mitigation Method and System Based on Multimodal Temporal Feature Fusion and Future Risk State Prediction Technical Field This application relates to the field of intelligent driving technology, and particularly to a vehicle collision mitigation method and system based on multimodal temporal feature fusion and future risk state prediction. Background Technology With the rapid development of intelligent driving and advanced driver assistance systems (ADAS), forward collision warning (FCW), automatic emergency braking (AEB), and collision mitigation technologies have become important technical directions for improving road traffic safety. Existing vehicle collision mitigation solutions typically acquire information on forward targets and the vehicle's operating status through cameras, 10 mm-wave radar, lidar, inertial measurement units (IMU), vehicle buses, etc., and make risk judgments based on indicators such as current distance, relative speed, and collision time. When a preset threshold is reached, a braking command is output to intervene in potential collision risks. However, existing technologies still have the following shortcomings in practical applications. 15 1. Existing AEB systems mainly rely on risk assessment at the current moment, lacking forward-looking modeling of future risk evolution, resulting in insufficient prediction lead and unsatisfactory intervention timing. Most existing AEB systems assess risk based on instantaneous indicators such as target distance, relative speed, and collision time at the current moment, and output braking commands when a threshold is reached. This approach is essentially a "current state triggered" control logic, lacking the ability to continuously model the risk evolution process over multiple future moments. In real-world road scenarios, collision risks often do not suddenly form at a single moment, but gradually evolve with factors such as the deceleration of the vehicle in front, the approach of the vehicle, and changes in the road environment. If the system only makes judgments based on the current moment, it is prone to problems such as delayed risk identification and insufficient prediction lead, resulting in late braking intervention, or intervention only when the risk is already quite imminent, making it difficult to obtain a more ideal intervention timing. 2. Multi-sensor systems typically employ target-level matching or shallow fusion, leading to insufficient utilization of cross-modal complementary information, decreased robustness in complex scenarios, and further impacting subsequent risk identification and control. While existing multi-sensor AEB or collision mitigation solutions incorporate various sensing methods such as cameras, millimeter-wave radar, and lidar, their fusion methods largely remain at the level of target-level matching, rule-level arbitration, or shallow fusion. For example, the results of target detection by different sensors are correlated, filtered, and merged before subsequent risk assessment. While this fusion method is relatively simple to implement, it fails to fully utilize the deep complementary relationships between different modalities at the semantic, geometric, dynamic, and temporal levels, and cannot fully explore the potential correlations between heterogeneous sensing data.In scenarios involving occlusion, changes in lighting, rain and fog, mixed traffic, and complex backgrounds, single-modal approaches are easily affected by interference, and shallow fusion cannot effectively compensate for this deficiency. Therefore, the overall robustness of the system often decreases, affecting subsequent risk identification and control effectiveness. 3. The weak connection between risk assessment and braking execution makes it difficult to effectively translate risk prediction results into graded braking control. In existing technologies, risk identification, risk assessment, control decision-making, and braking execution often belong to different modules. Although there is signal transmission between modules, in most cases, there is a lack of tight coupling based on a unified temporal representation. In other words, even if the system can output a certain risk judgment result, this result may not be able to naturally and directly map into a reasonable braking control output. Especially in scenarios requiring graded braking control, if there is a lack of clear correspondence between the risk assessment result and the control layer, it is easy to cause a disconnect between the risk assessment result and the control execution output, making it difficult to effectively translate the risk prediction result into control variables such as braking mode, target braking intensity, and deceleration trajectory, thus limiting the actual control effect of the collision mitigation system. 4. Difficulty in simultaneously achieving robust and smooth graded braking, leading to false triggering, missed triggering, or abrupt braking response. In vehicle collision mitigation scenarios, the system needs both strong risk identification robustness and the ability to output graded braking control with reasonable levels and smooth changes. However, existing solutions often fail to meet both requirements simultaneously. On the one hand, if the system adopts more sensitive triggering logic to improve safety redundancy, it may lead to more false triggering, resulting in unnecessary braking intervention in some scenarios. On the other hand, if the system raises the threshold to avoid false triggering, it may increase the risk of missed triggering and reduce the timeliness of collision mitigation. In addition, existing solutions often favor single-step triggering or simple threshold switching in control output, lacking consideration for the continuity of changes in risk level and vehicle state, which easily leads to abrupt braking response and insufficient smoothness, affecting the driving experience and system acceptability. Therefore, how to perform deeper unified feature representation and deeper cross-modal modeling of multimodal time-series sensing data, how to predict risk states at multiple future moments based on fused historical time-series features, and how to effectively map the predicted risk states to the output of graded braking control, thereby establishing a closed-loop collision mitigation technology chain between perception, prediction, decision-making and execution, have become urgent technical problems to be solved in this field.This application aims to address at least one of the aforementioned problems in the prior art by proposing a vehicle collision mitigation method and system based on multimodal temporal feature fusion and future risk state prediction. For forward collision mitigation scenarios, it fuses multimodal perception data and vehicle state data to predict collision risk-related states at multiple future moments, and further generates graded braking control commands based on the predicted risk states, thereby improving the forward-looking nature, hierarchical nature, and control smoothness of collision mitigation. In a first aspect, embodiments of this application provide a vehicle collision mitigation method based on multimodal temporal feature fusion and future risk state prediction, comprising: acquiring multimodal perception data of the vehicle's forward environment and vehicle state data; performing time synchronization and alignment on the multimodal perception data of the vehicle's forward environment and vehicle state data to establish a set of time-aligned multimodal perception data temporal samples; based on the multimodal perception data temporal samples, performing single-modal feature extraction on different modal data and mapping them to a unified feature space; in the unified feature space, employing a cross-modal attention mechanism to perform feature-level multimodal temporal fusion to obtain fused temporal features; inputting the fused temporal features into an encoder to model the historical temporal context and outputting a historical temporal encoded temporal feature sequence; inputting the historical temporal encoded temporal feature sequence into a decoder to output a collision risk-related state sequence for multiple future moments; and generating graded braking control commands based on the collision risk-related state sequence and vehicle state data. The graded braking control command is output to the vehicle control unit or braking actuator. In a preferred embodiment, the steps of acquiring multimodal perception data of the vehicle's forward environment and vehicle state data, synchronizing and aligning the multimodal perception data of the vehicle's forward environment and vehicle state data in time, and establishing a set of time-aligned multimodal perception data time series samples include: the multimodal perception data of the vehicle's forward environment includes one or more of the following: camera image data, lidar point cloud data and / or depth map data, millimeter-wave radar data; the vehicle state data includes one or more of the following: vehicle speed, longitudinal acceleration, lateral acceleration, yaw rate, braking state, steering state; establishing a set of time-aligned multimodal perception data time series samples includes: based on the time-synchronized and aligned multimodal perception data of the vehicle's forward environment and vehicle state data, setting the historical time window length to T, and extracting data at discrete times tT1, tT2, ..., tt to form a time-aligned multimodal perception data time series sample sequence: Xx, xx, ..., xx, where xx represents the multimodal perception data time series at time tt. Sample.In a preferred embodiment, the step of extracting single-modal features from different modal data based on time-series samples of multimodal perception data and mapping them to a unified feature space includes: extracting single-modal features from different modal data based on time-series samples of multimodal perception data, assuming that at time t, the original features extracted by each modality are: visual semantic features corresponding to camera images, spatial geometric features corresponding to LiDAR / depth data, dynamic features corresponding to millimeter-wave radar, and vehicle state features corresponding to vehicle state; mapping the different modal features to a unified feature space through their respective projection layers, resulting in a unified feature representation as: , , , , where , , , are the learnable projection matrices corresponding to different modalities; , , , are the corresponding bias terms; after mapping, the dimensions of each modal feature are unified to the same dimension d. 10 In a preferred embodiment, the step of extracting single-modal features from different modal data based on time-series samples of multimodal sensing data and mapping them to a unified feature space further includes: superimposing time position encoding and modality identifier encoding in the unified feature representation: , 15 where x represents the unified input representation of modality m at time t; e represents the time position encoding at time t; e represents the modality identifier encoding of modality m; m ∈ cam, geo, rad, ego. In a preferred embodiment, the step of performing feature-level multimodal temporal fusion using a cross-modal attention mechanism in a unified feature space to obtain fused temporal features includes: using spatial geometrically related modal features as query vectors; using one or more of visual semantic features, dynamic features, and vehicle state features as key vectors and value vectors; assuming that at time t, the unified feature representations are: where is the geometric feature representation, is the visual feature representation, is the dynamic feature representation, and is the vehicle state feature representation; then construct: HK 30137555 A 6 where represents the query vector; represents the key vector set; represents the value vector set; , , , , , , , are the learnable parameter matrix; represents the concatenation operation; calculating the cross-modal relevance weights using a scaled dot product attention mechanism: , 5 where represents the attention weight matrix at time t; represents the transpose of the key vector matrix; represents the key vector dimension; used to normalize the relevance; The value vector is weighted and summed based on the attention weights to obtain the fusion time-series feature at time t: ℎ , 10 where ℎ represents the fusion time-series feature obtained by cross-modal attention fusion at time t.In a preferred embodiment, the step of inputting the fused temporal features into an encoder, modeling the historical temporal context, and outputting a temporal feature sequence encoded by the historical temporal context includes: Assuming the fused temporal feature sequence obtained within a continuous time window at multiple consecutive moments is: H15hh, h1, …, h2, where H represents the fused temporal feature sequence within the historical time window; T represents the time window length; each ht represents the cross-modal fused feature at time t; Inputting the fused temporal feature sequence into an encoder to model the historical temporal context dependencies: S20EncoderH20, where S represents the temporal feature sequence encoded by the historical temporal context; Encoder represents the temporal context encoder; the encoder adopts an encoding structure based on a multi-head self-attention mechanism. In a preferred embodiment, in the step of inputting the historical time-series encoded temporal feature sequence into the decoder and outputting a collision risk-related state sequence for multiple future time periods; 25 Assume that the collision probability includes one or more of the following state quantities: distance to the preceding vehicle d, relative speed to the preceding vehicle v, collision time TTC, and risk level R; HK 30137555 A 7 The decoder output is represented as: Y y , y , … , y , where Y represents the collision risk-related state sequence for the next τ time periods; y represents the risk state vector at the i-th future time period; τ represents the prediction step size; yd , v , TTC , R , 5 where d represents the distance to the preceding vehicle at future time ti; v represents the relative speed to the preceding vehicle at future time ti; TTC represents the collision time at future time ti; and R represents the risk level at future time ti.In a preferred embodiment, the step of generating graded braking control commands based on the collision risk-related state sequence and vehicle state data includes: 10. Determining the collision risk level based on the collision risk-related state sequence and vehicle state data: The collision risk level includes at least: safe state, warning state, and immediate braking state; If the determination is made in conjunction with the prediction results of collision time TTC, relative speed v of the preceding vehicle, and distance d of the preceding vehicle, it includes: 15. Determining a safe state when one of the following conditions is met: TTC T or d D, where T is a safe time threshold and D is a safe distance threshold; in this state, active braking is not performed or the current state is maintained; Determining a warning state when one of the following conditions is met: 20. TTC or v 0 and |v| shows a significant increasing trend, where T is a higher risk time threshold; in this state, pre-braking or mild deceleration control is output; Determining an immediate braking state when one of the following conditions is met: TTC T and v 0, 25. In this state, higher-level braking control is output; Based on the risk level, a control output is generated, which includes at least one or more of the following: braking mode, target braking intensity, and longitudinal deceleration trajectory; HK 30137555 A 8 The control output is expressed as: uM, B, a∙, where u represents the control output at time t; M represents the braking mode; B represents the target braking intensity; and a∙ represents the longitudinal deceleration trajectory. In a preferred embodiment, the step of outputting the graded braking control command to the vehicle control unit or braking actuator further includes: collecting execution feedback information, which includes: actual deceleration, execution delay, and braking system status; Let the feedback quantity be: ra, Δt, s, 10 where represents the actual deceleration, represents the execution delay, and represents the braking system status; The execution feedback information is transmitted back to the upper-level control module for status correction, execution effect evaluation, and subsequent closed-loop control optimization.Secondly, embodiments of this application provide a vehicle collision mitigation system based on multimodal temporal feature fusion and future risk state prediction. The system is configured to implement any of the methods described in the first aspect. The system includes: a data acquisition and preprocessing module, used to acquire multimodal perception data of the vehicle's forward environment and vehicle state data, perform time synchronization and alignment on the multimodal perception data of the vehicle's forward environment and vehicle state data, and establish a set of time-aligned multimodal perception data temporal samples; a feature extraction and spatial mapping module, used to extract single-modal features from different modal data based on the multimodal perception data temporal samples, and map them to a unified feature space; a multimodal temporal fusion module, used to perform feature-level multimodal temporal fusion in the unified feature space using a cross-modal attention mechanism to obtain fused temporal features; and a historical context encoding module, used to input the fused temporal features into an encoder, model the historical temporal context, and output a historical temporal encoded temporal feature sequence. A collision risk state prediction module is used to input the time-series feature sequence of the historical time-series encoding (HK 30137555 A 9 columns) into a decoder and output a collision risk-related state sequence for multiple future time points; a graded braking control generation module is used to generate graded braking control commands based on the collision risk-related state sequence and vehicle state data; a control execution module is used to output the graded braking control commands to the vehicle control unit or braking actuator to execute collision mitigation actions. This application has the following beneficial effects: This application can improve the safety performance of existing ADAS or AEB products, enabling them to have stronger future risk prediction capabilities and more reasonable graded braking control capabilities, thereby forming a differentiated advantage in the market. Compared to simpler traffic scenarios, this application is more suitable for application environments with complex road environments, diverse types of traffic participants, and frequent scene changes, such as high-density urban traffic scenarios, helping to reduce the cost of false triggering, missed triggering, and scene adaptation for autonomous driving. This application is constructed in a modular manner and can be adapted and reused on different vehicle models, different sensor configurations, and different active safety platforms, possessing strong platform extensibility and market development potential. 15. Brief Description of the Drawings: Figure 1 is a general functional architecture diagram of the vehicle collision mitigation system provided in this application. Figure 2 is a mechanism diagram of feature-level fusion and future risk state prediction provided in this application. Figure 3 is a risk state-driven graded braking control diagram provided in this application. Figure 4 is a flowchart illustrating a vehicle collision mitigation method based on multimodal temporal feature fusion and future 20 risk state prediction provided in an embodiment of this application. Detailed Description of the Embodiments: The detailed description and technical content of this application are explained below in conjunction with the accompanying drawings. However, the drawings are for reference and illustration only and are not intended to limit this application.Figure 1 is the overall functional architecture diagram of the vehicle collision mitigation system provided in this application. Figure 2 is the feature-level fusion and future risk state prediction mechanism diagram provided in this application. Figure 3 is the risk state-driven graded braking control diagram provided in this application. As shown in Figures 1, 2, and 3, the specific implementation principle and method of this application are as follows. HK 30137555 A 10 First, the system acquires multimodal time-series data of the vehicle's forward environment perception and vehicle state data. The multimodal data includes at least two or more of the following: visual perception data (such as camera images), spatial geometric perception data (such as lidar point clouds or depth information), and dynamic perception data (such as millimeter-wave radar data); the vehicle state data may include vehicle speed, longitudinal / lateral acceleration, yaw rate, CAN bus signal, IMU data, and braking status, etc. 5 Since the sampling frequencies of different modes are inconsistent, the system needs to synchronize and align the above data in time, which can be achieved through hardware synchronization mechanisms or software interpolation, resampling, etc. Subsequently, the system uses a sliding window to construct continuous multi-frame temporal inputs to form temporal samples required for subsequent model processing. In feature processing, the system extracts single-modal features from each modality of data, such as extracting visual semantic features from image data, spatial geometric features from point cloud / depth data, motion dynamic features from radar data, and vehicle state features from vehicle state data. The extracted modal features are mapped to a unified feature dimension through a projection layer, forming a temporal feature representation in a unified feature space. To enhance the model's ability to distinguish temporal order and modal origin, temporal position encoding and modal identifier encoding can be introduced into the mapped feature representation. In the unified feature space, this application performs feature-level multimodal temporal fusion based on a cross-modal attention mechanism. The basic principle is as follows: Geometric spatial features are used as query vectors, and one or more of visual semantic features, motion dynamic features, and vehicle state features are used as key and value vectors. Attention-based relevance calculations facilitate information interaction between different modalities. Then, learnable modal weights are used for adaptive 20-weighted fusion to obtain fused temporal features. This step differs from existing solutions that often use target-level matching or rule-level arbitration; instead, it achieves feature-level temporal fusion of heterogeneous data in a unified feature space. The system then inputs the fused temporal features into an encoder to model historical temporal context dependencies and intermodal relationships, outputting a temporal feature sequence. This 25-times temporal feature sequence is then input into a decoder to output a collision risk-related state sequence for multiple future time points. This risk state sequence may include one or more of the following: distance to the preceding vehicle, relative speed to the preceding vehicle, collision time, and risk level.For example, the system can receive the fused timing features of the previous 15 time steps as input and output the collision risk-related state sequence of the next 5 time steps. After obtaining the future risk state sequence, the system determines the risk level based on the predicted risk state and the current vehicle state information, and generates a graded braking control output. In a preferred embodiment, the risk level may include a safe state (SAFE), a warning state (WARNING), and an immediate braking state (BRAKE). The system outputs the corresponding braking mode, target braking intensity, and longitudinal deceleration trajectory according to different risk levels to improve the hierarchy, smoothness, and controllability of collision mitigation control. Finally, the control command is output to the ECU or braking actuator to complete the collision mitigation action, and the actual deceleration, execution delay, and braking system state can be fed back to the upper-level module for subsequent control correction and closed-loop processing. This application can be applied to forward collision mitigation and active safety control in autonomous vehicles, smart shuttle buses, automated mobility platforms, and other mobile platforms. It can also serve as a core algorithm module for predictive collision mitigation in ADAS systems, supporting advanced automatic emergency braking (AEB), risk prediction, and graded braking control. This application has clear commercial application value in the fields of intelligent vehicle active safety and ADAS, particularly suitable for application scenarios such as smart shuttle buses and low-speed autonomous driving platforms, and for customers including control software suppliers, ADAS and autonomous driving suppliers. For the industry, this application helps drive the evolution of collision mitigation systems from "threshold-triggered AEB" to "predictive active safety control," which is of positive significance for improving driving safety, reducing accident risks, improving the driving experience, and promoting the autonomy of core algorithms for intelligent vehicles.Figure 4 is a flowchart illustrating a vehicle collision mitigation method based on multimodal temporal feature fusion and future risk state prediction provided in an embodiment of this application. As shown in Figure 4, the method includes: Step S1, data acquisition and preprocessing: acquiring multimodal perception data of the vehicle's forward environment and vehicle state data, performing time synchronization and alignment on the multimodal perception data of the vehicle's forward environment and vehicle state data, and establishing a set of time-aligned multimodal perception data temporal samples; the purpose of this step is to establish a set of time-aligned multimodal perception data temporal samples to provide a foundation for subsequent single-modal feature extraction, unified feature space mapping, and fusion calculation of the vehicle's forward environment. The time-aligned multimodal perception data includes the multimodal perception data of the vehicle's forward environment and vehicle state data; Step S2, single-modal feature extraction and unified feature space mapping: based on the multimodal perception data temporal samples, single-modal features are extracted from different modal data respectively and mapped to a unified feature space. Step S3, Feature-level multimodal temporal fusion based on cross-modal attention mechanism: In the unified feature space, feature-level multimodal temporal fusion is performed using a cross-modal attention mechanism to obtain fused temporal features; Step S4, Historical context encoding of fused temporal features: The fused temporal features are input into the encoder to model the historical temporal context and output the temporal feature sequence encoded by the historical temporal sequence; Step S5, Future multi-time collision risk state prediction: The temporal feature sequence encoded by the historical temporal sequence is input into the decoder to output the collision risk-related state sequence for multiple future times; Step S6, Generation of graded braking control based on risk state: Graded braking control commands are generated based on the collision risk-related state sequence and vehicle state data; Step S7, Control execution and state feedback: The graded braking control commands are output to the vehicle control unit (ECU) or braking actuator to execute collision mitigation actions. The multimodal perception data of the vehicle's forward environment includes at least two of the following: visual perception data (such as camera images), spatial geometric perception data (such as LiDAR point clouds or depth information), and dynamic perception data (such as millimeter-wave radar data); vehicle state data may include vehicle speed, longitudinal / lateral acceleration, yaw rate, CAN bus signals, IMU data, and braking status, etc. Since different sensors have different sampling frequencies, the system must first synchronize the multimodal data in time to ensure that the input data at the same moment correspond to each other in the time dimension. In practical implementation, time synchronization can be achieved through hardware timestamp alignment, resampling, interpolation, or inter-frame matching, thereby forming a multimodal input window under a unified time reference. As shown in the above steps, this invention proposes a feature-level multimodal temporal fusion technology based on a unified feature space.Unlike existing technologies that commonly employ target-level matching, rule-level arbitration, or shallow layer fusion methods, this invention does not merely merge the detection results from different sensors at the back end. Instead, it first extracts features from visual perception data, spatial geometric perception data, dynamic perception data, and vehicle state data, maps them to a unified feature space, and then performs temporal fusion and cross-modal information interaction within this unified feature space. As shown in the above steps, this invention proposes a future risk state prediction technology based on fused temporal features. Unlike existing AEB or collision mitigation systems that primarily rely on instantaneous indicators such as target distance, relative speed, and collision time at the current moment for judgment, this invention does not only determine the risk at the current moment but also models the risk evolution process over multiple future moments, thereby achieving an improvement from current-moment judgment to forward-looking risk prediction. As shown in the above steps, this invention proposes a risk state-driven graded braking control technology. Unlike existing control methods based on fixed thresholds, single-step triggering logic, or simple braking output, this invention directly maps the predicted future risk state to a graded braking control output, thus creating a closer technical connection between risk assessment results and control execution. In some embodiments, step S1, data acquisition and preprocessing: acquiring multimodal perception data of the vehicle's forward environment and vehicle state data, performing time synchronization and alignment of the multimodal perception data of the vehicle's forward environment and vehicle state data, and establishing a set of time-aligned multimodal perception data time series samples. In this step: the multimodal perception data of the vehicle's forward environment includes one or more of the following: camera image data, used to characterize the visual semantic information of the scene ahead; lidar point cloud data and / or depth map data, used to characterize the spatial geometric information of the target and environment ahead; millimeter-wave radar data, used to characterize dynamic information such as target relative speed and motion trend; the vehicle state data includes one or more of the following: vehicle speed v, longitudinal acceleration α, lateral acceleration α, yaw rate ω, braking state, steering state, and other state quantities collected by the CAN bus or IMU; establishing a set of time-aligned multimodal perception data time series samples includes: 25 Based on the time-synchronized and aligned multimodal perception data of the vehicle's forward environment and vehicle state data, setting the historical time window length to T, at discrete time tT... Data is extracted from 1, t T 2, … , t to form a time-aligned multimodal sensing data time series sample HK 30137555 A 14 sequence: X x , x , … , x , where x represents the multimodal sensing data time series sample at time t.In some embodiments, step S2, single-modal feature extraction and unified feature space mapping: Based on time-series samples of multimodal perception data, single-modal feature extraction is performed on different modal data respectively, and mapped to a unified feature space. This step includes: 5. Based on time-series samples of multimodal perception data, single-modal feature extraction is performed on different modal data respectively. Let the original features extracted by each modality at time be: visual semantic features corresponding to camera images, spatial geometric features corresponding to LiDAR / depth data: , dynamic features corresponding to millimeter-wave radar: , and vehicle state features corresponding to vehicle state: , where: cam represents camera mode; geo represents geometry mode; rad represents radar mode; ego represents ego-state, i.e., vehicle self-state mode; Through their respective corresponding projection layers, the different modal features are mapped to a unified feature space, resulting in a unified feature representation as: , , 15 , , where , , , are the learnable projection matrices corresponding to different modalities; , , , For the corresponding bias term; after mapping, the dimensions of each modal feature are unified to the same dimension d. 20 In some embodiments, step S2, single-modal feature extraction and unified feature space mapping: based on the time-series samples of multimodal perception data, single-modal feature extraction is performed on different modal data respectively, and mapped to a unified feature space. The step further includes: superimposing time position encoding and modality identification encoding in the unified feature representation to enhance the model's ability to identify time-series position and modality source: 25, where x represents the unified input representation of modality m at time t; e represents the time position encoding at time t; e represents the modality identification encoding of modality m; m ∈ cam, geo, rad, ego. HK 30137555 A 15 The purpose of this step is to unify heterogeneous, multi-dimensional, and different source modal features into the same representation space, thereby creating conditions for subsequent cross-modal feature-level fusion.In some embodiments, step S3, feature-level multimodal temporal fusion based on cross-modal attention mechanism: In a unified feature space, feature-level multimodal temporal fusion is performed using a cross-modal attention mechanism to obtain fused temporal features. The steps include: 5. Using spatial geometrically related modal features as query vectors; 6. Using one or more of visual semantic features, dynamic features, and vehicle state features as key vectors and value vectors; 7. Assuming that at time t, the unified feature representations are: is the geometric feature representation; is the visual feature representation; is the dynamic feature representation; is the vehicle state feature representation; then construct: , , , , where represents the query vector; represents the key vector set; represents the value vector set; , , , , , , , , are the learnable parameter matrix; represents the concatenation operation; 8. Calculating cross-modal relevance weights through a scaled dot product attention mechanism: 15. Where represents the attention weight matrix at time t; represents the transpose of the key vector matrix; represents the key vector dimension; is used to normalize the relevance. The value vector is weighted and summed based on the attention weights to obtain the fusion temporal feature at time t: ℎ where ℎ represents the fusion temporal feature obtained by cross-modal attention fusion at time t. 20 In a further embodiment, learnable modal weights or a gating mechanism are introduced to adaptively weight different modal information to enhance robustness in complex scenarios. For example, HK 30137555 A 16 Let the weights of different modalities be α, α, α, then: α α α 1 The final fusion temporal feature is further expressed as: h α h α h α h where: h, h, h represent different modal components; α, α, α are learnable or modal weights generated by a gating network. 5. The purpose of this step is to: 1) establish deep associations between geometric, visual, dynamic, and vehicle state information at the feature layer; 2) selectively retrieve cross-modal information related to the current spatial relationship through an attention mechanism; and 3) form a unified fusion feature representation that is more suitable for collision mitigation tasks.In some embodiments, step S4, encoding the historical context of the fused temporal features; 10. Inputting the fused temporal features into an encoder to model the historical temporal context and outputting the temporal feature sequence of the historical temporal encoding includes: Let the fused temporal feature sequence obtained at multiple consecutive moments within a continuous time window be: Hh, ht, …, ht where: H represents the fused temporal feature sequence within the historical time window; T represents the time window length; each ht represents the cross-modal fused feature at time t; 15. Inputting the fused temporal feature sequence into an encoder to model the historical temporal context dependency: SEncoder Ht where: S represents the temporal feature sequence of the historical temporal encoding (after encoding); Encoder represents the temporal context encoder; the encoder adopts an encoding structure based on a multi-head self-attention mechanism. 20. In a preferred embodiment, the encoder receives the fused features of the first 15 moments as input to form a temporal representation of the historical risk evolution process. The purpose of this step is to further learn the continuous evolution law of the fused features of different modalities within the historical time window, providing a temporal context basis for subsequent prediction of future risk states. HK 30137555 A 17 In some embodiments, step S5, future multi-time collision risk state prediction: inputting the historical time-series encoded temporal feature sequence into the decoder and outputting the collision risk-related state sequence for multiple future times; assuming the collision probability includes one or more of the following state quantities: distance to the preceding vehicle d, relative speed to the preceding vehicle v, collision time TTC, risk level R; the decoder output is represented as: Y y , y , … , y where: Y represents the collision risk-related state sequence for the next τ times; y represents the risk state vector for the i-th future time; τ represents the prediction step size; yd , v , TTC , R where: d represents the distance to the preceding vehicle at future time ti; v represents the relative speed to the preceding vehicle at future time ti; TTC represents the collision time at future time ti; R represents the risk level at future time ti. In an example implementation, the historical fused temporal features of the first 15 times can be input, and the collision risk-related states of the last 5 times can be predicted. The purpose of this step is to upgrade the system from "risk assessment at the current moment" to "risk state prediction at multiple future moments", making collision mitigation control more forward-looking.15 In some embodiments, step S6, generating graded braking control based on risk state: the step of generating graded braking control commands according to the collision risk-related state sequence and vehicle state data includes: determining the collision risk level according to the collision risk-related state sequence and vehicle state data: 20 The collision risk level includes at least: safe state (SAFE), warning state (WARNING), and immediate braking state (BRAKE); If the judgment is made in conjunction with the prediction results of collision time TTC, relative speed v of the preceding vehicle, and distance d of the preceding vehicle, it includes: When one of the following conditions is met, it is determined to be a safe state: 25 TTC T or d D, HK 30137555 A 18 Where: T is the safe time threshold, for example, 3 seconds; D is the safe distance threshold; in this state, the system maintains monitoring and does not perform active braking or only maintains the current state; When one of the following conditions is met, it is determined to be a warning state: TT or v 0 and |v| shows a significant increasing trend, where: T is the higher risk time threshold, for example, 1.5 seconds; in this state, the output 5 Pre-braking or mild deceleration control; When one of the following conditions is met, it is determined to be an immediate braking state: TTC T and v 0, indicating a high risk of rear-end collision; In this state, a higher level of braking control is output to avoid a collision or mitigate the consequences of a collision; 10. Based on the risk level, a control output is generated, which includes at least one or more of the following: braking mode M, target braking intensity B, longitudinal deceleration trajectory at; The control output is expressed as: uM, B, a∙ Where: u represents the control output at time t; M represents the braking mode; B represents the target braking intensity; a∙ represents the longitudinal deceleration trajectory. 15. The purpose of this step is to effectively transform the future risk state prediction results into hierarchical, controllable, and smooth braking outputs, realizing a direct mapping from risk identification to control decision. In some embodiments, step S7, control execution and state feedback: outputting the graded braking control command to the vehicle control unit (ECU) or braking actuator to execute the collision mitigation action, further includes: 20 collecting execution feedback information, the execution feedback information including: actual deceleration, execution delay, braking system state; let the feedback quantity be: ra, Δt, s where: , represents actual deceleration; represents execution delay; represents braking system state; 25 the execution feedback information is transmitted back to the upper-level control module for state correction, execution effect evaluation and subsequent closed-loop control optimization.Based on the same inventive concept, this application also provides a vehicle collision mitigation system based on multimodal temporal feature fusion and future risk state prediction. The system is configured to implement any of the methods described in the above embodiments. The system includes: a data acquisition and preprocessing module, used to acquire multimodal perception data of the vehicle's forward environment and vehicle state data, perform time synchronization and alignment on the multimodal perception data of the vehicle's forward environment and vehicle state data, and establish a set of time-aligned multimodal perception data temporal samples; a feature extraction and spatial mapping module, used to extract single-modal features from different modal data based on the multimodal perception data temporal samples, and map them to a unified feature space; a multimodal temporal fusion module, used to perform feature-level multimodal temporal fusion in the unified feature space using a cross-modal attention mechanism to obtain fused temporal features; a historical context encoding module, used to input the fused temporal features into an encoder, model the historical temporal context, and output a sequence of historical temporal encoded temporal features; and a collision risk state prediction module, used to sequence the historical temporal encoded temporal features. A column input decoder outputs a sequence of collision risk-related states for multiple future time points; a graded braking control generation module generates graded braking control commands based on the collision risk-related state sequence and vehicle state data; a control execution module outputs the graded braking control commands to the vehicle control unit (ECU) or braking actuator to execute collision mitigation actions. The above embodiments are merely preferred embodiments for fully illustrating this application, and the scope of protection of this application is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on this application are all within the scope of protection of this application.HK 30137555 A 1 Claim 1. A vehicle collision mitigation method based on multimodal temporal feature fusion and future risk state prediction, characterized in that it includes: acquiring multimodal perception data of the vehicle's forward environment and vehicle state data; synchronizing and aligning the multimodal perception data of the vehicle's forward environment and vehicle state data in time to establish a set of time-aligned multimodal perception data temporal samples; extracting single-modal features from different modal data based on the multimodal perception data temporal samples and mapping them to a unified feature space; performing feature-level multimodal temporal fusion using a cross-modal attention mechanism in the unified feature space to obtain fused temporal features; inputting the fused temporal features into an encoder to model the historical temporal context and outputting a historical temporal encoded temporal feature sequence; inputting the historical temporal encoded temporal feature sequence into a decoder to output a collision risk-related state sequence for multiple future moments; generating a graded braking control command based on the collision risk-related state sequence and vehicle state data. The graded braking control command is output to the vehicle control unit or braking actuator. 2. The method according to claim 1, characterized in that, in the step of acquiring multimodal perception data of the vehicle's forward environment and vehicle state data, synchronizing and aligning the multimodal perception data of the vehicle's forward environment and vehicle state data in time, and establishing a set of time-aligned multimodal perception data time series samples: the multimodal perception data of the vehicle's forward environment includes one or more of the following: camera image data, lidar point cloud data and / or depth map data, millimeter-wave radar data; the vehicle state data includes one or more of the following: vehicle speed, longitudinal acceleration, lateral acceleration, yaw rate, braking state, steering state; the step of establishing a set of time-aligned multimodal perception data time series samples includes: based on the vehicle's forward environment multimodal perception data and HK 30137555 A 2 vehicle state data that have been synchronized and aligned in time, setting the historical time window length to T, and extracting data at discrete times t T 1, t T 2, … , t to form a time-aligned multimodal perception data time series sample sequence: X x , x , … , x, where x represents a time-series sample of multimodal sensing data at time t.3. The method according to claim 1, characterized in that the step of extracting single-modal features from different modal data based on multimodal sensing data time series samples and mapping them to a unified feature space includes: extracting single-modal features from different modal data based on multimodal sensing data time series samples, assuming that at time 1, the original features extracted by each modality are: visual semantic features corresponding to camera images, spatial geometric features corresponding to LiDAR / depth data, dynamic features corresponding to millimeter-wave radar, and vehicle state features corresponding to vehicle state; mapping different modal features to a unified feature space through their respective corresponding projection layers, obtaining a unified feature representation as: , , , , where , , , are the learnable projection matrices corresponding to different modalities; , , , , are the corresponding bias terms; after mapping, the dimensions of each modal feature are unified to the same dimension d. 4. The method according to claim 3, characterized in that the step of extracting single-modal features from different modal data based on time-series samples of multimodal sensing data and mapping them to a unified feature space further includes: superimposing time position encoding and modality identifier encoding in the unified feature representation: 25, where x represents the unified input representation of modality m at time t; e represents the time position encoding at time t; e represents the modality identifier encoding of modality m; m ∈ cam, geo, rad, ego. HK 30137555 A 3 5. The method according to claim 1, characterized in that the step of using a cross-modal attention mechanism to perform feature-level multimodal temporal fusion in a unified feature space to obtain fused temporal features includes: using spatial geometrically related modal features as query vectors; using one or more of visual semantic features, dynamic features, and vehicle state features as key vectors and value vectors; assuming that at time , the unified feature representations are respectively: is the geometric feature representation, is the visual feature representation, is the dynamic feature representation, and is the vehicle state feature representation; then construct: , 10 , , , , , , where represents the query vector; represents the key vector set; represents the value vector set; , , , , , , , , are the learnable parameter matrix; represents the concatenation operation; 15 calculates the cross-modal relevance weights through a scaling dot product attention mechanism: , where represents the attention weight matrix at time ; represents the transpose of the key vector matrix; represents the key vector dimension; used to normalize the relevance; Based on the attention weights, the value vector is weighted and summed to obtain the fused temporal feature at time t: ℎ, where ℎ represents the fused temporal feature obtained by cross-modal attention fusion at time t.6. The method according to claim 1, characterized in that the step of inputting the fused temporal features into the encoder, modeling the historical temporal context, and outputting the temporal feature sequence of the historical temporal code 25 includes: Let the fused temporal feature sequence obtained in a continuous time window at multiple consecutive moments be: HK 30137555 A 4 H h , h , … , h , where H represents the fused temporal feature sequence in the historical time window; T represents the time window length; each h represents the cross-modal fused feature at time t; Input the fused temporal feature sequence into the encoder to model the historical temporal context dependency: 5 S Encoder H , where S represents the temporal feature sequence of the historical temporal code; Encoder represents the temporal context encoder; the encoder adopts an encoding structure based on a multi-head self-attention mechanism. 7. The method according to claim 1, characterized in that, in the step of inputting the time-series feature sequence encoded by the historical time series into the decoder and outputting the collision risk-related state sequence for multiple future time periods; let the collision probability include one or more of the following state quantities: distance to the preceding vehicle d, relative speed to the preceding vehicle v, collision time TTC, and risk level R; the decoder output is expressed as: Y y , y , … , y , where Y represents the collision risk-related state sequence for the next τ time periods; y represents the risk state vector at the i-th future time period; τ represents the prediction step size; yd , v , TTC , R , where d represents the distance to the preceding vehicle at the future time ti; v represents the relative speed to the preceding vehicle at the future time ti; TTC represents the collision time at the future time ti; and R represents the risk level at the future time ti.8. The method according to claim 1, wherein the step of generating graded braking control commands based on the collision risk-related state sequence and vehicle state data includes: determining the collision risk level based on the collision risk-related state sequence and vehicle state data: the collision risk level includes at least: safe state, warning state, and immediate braking state; if the judgment is made in conjunction with the predicted results of collision time TTC, relative speed v of the preceding vehicle, and distance d of the preceding vehicle, the following is included: when one of the following conditions is met, it is determined to be a safe state: TTC = T or d = D, where T is a safe time threshold and D is a safe distance threshold; in this state, active braking is not performed or only the current state is maintained; when one of the following conditions is met, it is determined to be a warning state: TT or v ≥ 0 and |v| shows a significant increasing trend, where T is a higher risk time threshold; in this state, pre-braking or mild deceleration control is output; when one of the following conditions is met, it is determined to be an immediate braking state: TTC = T and v ≥ 0; in this state, higher level braking control is output; Based on the risk level, a control output is generated, wherein the control output includes at least one or more of the following: braking mode, target braking intensity, and longitudinal deceleration trajectory; the control output is expressed as: 15 u M , B , a ∙ , where u represents the control output at time t; M represents the braking mode; B represents the target braking intensity; and a ∙ represents the longitudinal deceleration trajectory. 9. The method according to claim 1, wherein the step of outputting the graded braking control command to the vehicle control unit or braking actuator further includes: 20 collecting execution feedback information, wherein the execution feedback information includes: actual deceleration, execution delay, and braking system status; let the feedback quantity be: ra , Δt , s , where represents the actual deceleration, represents the execution delay, and represents the braking system status; the execution feedback information is transmitted back to the upper-level control module for status correction, execution effect evaluation, and subsequent closed-loop control optimization.HK 30137555 A 6 10. A vehicle collision mitigation system based on multimodal temporal feature fusion and future risk state prediction, characterized in that it is configured to implement the method described in any one of claims 1 to 9, the system comprising: a data acquisition and preprocessing module, used to acquire multimodal perception data of the vehicle's forward environment and vehicle state data, perform time synchronization and alignment on the multimodal perception data of the vehicle's forward environment and vehicle state data, and establish a set of time-aligned multimodal perception data temporal samples; a feature extraction and spatial mapping module, used to extract single-modal features from different modal data based on the multimodal perception data temporal samples, and map them to a unified feature space; a multimodal temporal fusion module, used to perform feature-level multimodal temporal fusion in the unified feature space using a cross-modal attention mechanism to obtain fused temporal features; and a historical context encoding module, used to input the fused temporal features into an encoder, model the historical temporal context, and output a historical temporal encoded temporal feature sequence; A collision risk state prediction module is used to input the historical time-series encoded time-series feature sequence into a decoder and output a collision risk-related state sequence for multiple future time periods; a graded braking control generation module is used to generate graded braking control commands based on the collision risk-related state sequence and vehicle state data; a control execution module is used to output the graded braking control commands to the vehicle control unit or braking actuator to execute collision mitigation actions. HK 30137555 A 1 Instruction Manual Drawings Figure 1 HK 30137555 A 2 Figure 2 HK 30137555 A 3 Figure 3 HK 30137555 A 4 Figure 4 HK 30137555 A.