A passenger protection method and system based on multi-modal perception fusion

Abstract

The application discloses a kind of passenger protection method and system based on multi-modal perception fusion.The method comprises: collecting passenger state data in cabin;Obtain vehicle driving state data and collision risk data;Passenger state data, vehicle driving state data and collision risk data are fused to generate passenger fusion state information;According to passenger fusion state information and collision risk data, determine whether the vehicle is in the pre-collision stage, collision stage or post-collision stage;In the pre-collision stage, execute posture reminder, active pre-tightening of safety belt or seat reset;In the collision stage, adjust the protection parameters of airbag and / or safety belt according to the passenger fusion state information obtained in the pre-collision stage and real-time passenger state;In the post-collision stage, generate accident rescue information and send it to the vehicle communication terminal and / or event data recorder.The application can improve the continuity, adaptability and accuracy of passenger protection in complex cabin scenarios.

Description

Technical Field

[0001] This invention relates to the field of vehicle safety control technology, and more specifically, to an occupant protection method and system based on multimodal perception fusion. Background Technology

[0002] With the development of intelligent cockpits, active safety systems, and passive safety systems, vehicle occupant protection is no longer limited to the deployment of airbags or seat belt restraint at the moment of a collision, but is gradually developing towards pre-collision prediction, adaptive restraint during a collision, and post-collision rescue coordination.

[0003] Existing vehicles are typically equipped with active and passive safety systems. Active safety systems primarily use sensors such as cameras, millimeter-wave radar, and lidar to perceive the vehicle's external environment and issue warnings, brakes, or steering assistance when a collision risk is detected. Passive safety systems, on the other hand, mainly activate restraint devices such as airbags, seatbelt pretensioners, and seatbelt force limiters after a collision, based on collision sensors and vehicle acceleration signals, to reduce the risk of injury to occupants.

[0004] However, existing occupant protection methods still have the following shortcomings:

[0005] First, existing passive safety systems typically determine their deployment strategy based on limited information such as collision intensity, vehicle speed, and seat occupancy status. This limited perception makes it difficult to accurately identify the occupant's true posture before and at the moment of impact. For example, an occupant might be in a forward-leaning, side-leaning, head-turned, reclining, dislodged, or zero-gravity seat position. In these situations, using fixed airbag and seatbelt strategies could result in excessive, insufficient, or mismatched restraint forces.

[0006] Second, active safety systems and passive safety systems are often relatively disconnected. After an active safety system triggers emergency braking, occupants may lean forward or deviate from their standard sitting posture due to inertia. However, traditional passive safety systems often fail to fully utilize information about changes in occupant posture before a collision, resulting in protection strategies during a collision that cannot match the real-time state of the occupants.

[0007] Third, existing post-collision rescue information typically includes basic information such as vehicle location, collision time, and vehicle status. It does not make sufficient use of information such as occupant vital signs, restraint system activation status, occupant posture changes, potential injury locations, and injury levels. This makes it difficult to provide rescue centers with more accurate accident rescue information and is also not conducive to subsequent accident reconstruction, property compensation, and safety strategy iteration.

[0008] Existing technologies include object control schemes that integrate active and passive safety features. These schemes acquire external vehicle image datasets and point cloud datasets to determine the object to be collided with and its feature set. They then obtain the collision condition feature set based on a collision condition prediction model and combine this with passenger characteristics to determine the airbag deployment time. While these schemes combine external collision conditions with passenger characteristics, their focus remains primarily on controlling the airbag deployment time. They do not fully address the issue of identifying the full range of occupant states based on multimodal perception fusion within the cabin and providing adaptive protection throughout the pre-collision, during-collision, and post-collision phases.

[0009] Therefore, it is necessary to provide an occupant protection system based on multimodal perception fusion to improve the continuity, adaptability and accuracy of occupant protection in complex cockpit scenarios. Summary of the Invention

[0010] To address the shortcomings of existing technologies, the purpose of this invention is to provide an occupant protection method and system based on multimodal perception fusion, which can achieve continuous and adaptive occupant protection before, during, and after a collision. This aims to solve the technical problem that existing automotive occupant protection systems, due to their single perception dimension and the separation of active and passive safety protection stages, cannot provide adaptive, continuous, and accurate full-cycle safety protection in complex cabin scenarios, especially in non-standard seating conditions such as zero-gravity seats, relaxed sitting postures, and dismounted postures.

[0011] To achieve the above-mentioned technical effects, the present invention adopts the following technical solution:

[0012] According to a first aspect of the present invention, an occupant protection method based on multimodal perception fusion is provided, comprising the following steps:

[0013] Collect occupant status data inside the cabin;

[0014] Acquire vehicle driving status data and collision risk data;

[0015] The occupant status data, vehicle driving status data, and collision risk data are fused to generate occupant fusion status information. Specifically, this includes: synchronizing data from different sources in time; converting data from different sources to a unified vehicle coordinate system or cockpit coordinate system; determining the confidence level of each source data based on at least one of the following: data validity, data quality parameters, and the degree of fluctuation of continuously sampled data; and fusing the data from each source based on the confidence level to generate the occupant fusion status information.

[0016] The vehicle's protection phase is determined based on the occupant fusion status information and collision risk data. The protection phase includes the pre-collision phase, the collision phase, and the post-collision phase.

[0017] In the pre-collision phase, at least one of the following pre-collision protection strategies is executed based on the occupant fusion state information: attitude reminder, active seat belt pretensioning, or seat reset.

[0018] During the collision phase, the protection parameters of the airbags and / or seat belts are adjusted based on the occupant fusion state information obtained in the pre-collision phase and the real-time occupant state at the time of the collision.

[0019] In the post-collision phase, accident rescue information is generated based on occupant fusion status information, vehicle collision information, and constraint execution status, and the accident rescue information is sent to the vehicle communication terminal and / or event data recorder.

[0020] Optionally, the data collected on the occupant status in the cabin includes: obtaining seat occupancy status and occupant body type through an occupant classification system; obtaining posture information of the occupant's head, chest, torso, or limbs through an image acquisition unit; obtaining occupant distance information, motion information, breathing information, or heart rate information through a radar sensing unit; obtaining seat rail position, backrest angle, cushion angle, or zero-gravity seat status through a seat status sensing unit; and obtaining seat belt buckle status, pull-out length, or tension status through a seat belt status detection unit.

[0021] Optionally, the occupant fusion status information includes one or more of the following: occupant presence, occupant body type, occupant vital signs, occupant three-dimensional key point posture, distance between occupant and airbag, occupant disengagement status, occupant behavioral intention, and occupant movement trend.

[0022] Optionally, during the pre-collision phase, at least one of the following pre-collision protection strategies may be implemented: when the occupant is in an off-position state or deviates from the safe contact area, a posture adjustment reminder may be output; when the vehicle undergoes emergency braking and the occupant has a tendency to lean forward, the seat belt may be controlled to perform active pretensioning; when the seat is in a zero-gravity seat state, the backrest angle exceeds a preset angle, or the occupant is far from the optimal contact area of ​​the airbag, a seat reset reminder may be output or the seat may be controlled to reset.

[0023] Optionally, during the collision phase, based on the occupant fusion state information obtained in the pre-collision phase and the real-time occupant state at the time of the collision, at least one of the following protection parameters is determined: airbag deployment time, airbag deployment level, airbag inflation volume, seat belt pretension level, seat belt force limiting level, and seat adjustment state.

[0024] Optionally, during the collision phase: when the occupant is in a standard seating position and a frontal collision occurs, the seat belts and airbags are controlled to operate according to preset standard protection parameters;

[0025] When the occupant is in an off-position state or the distance between the occupant and the airbag is less than the first distance threshold, the seat belt is pre-tensioned first, and the airbag is deployed in a way that reduces the inflation volume, delays the deployment, or deploys in a single-stage deployment mode.

[0026] When the occupant is in a relaxed sitting position, a zero-gravity seat position, or the distance between the occupant and the airbag is greater than the second distance threshold, the airbag deployment is delayed, or the airbag is first deployed with a preset first-level inflation volume and then supplemented with a preset second-level inflation volume.

[0027] When the occupant's head turns or torso tilts to the side, adjust the airbag deployment strategy and / or seatbelt force limiting strategy.

[0028] Optionally, in the post-collision phase, information such as the detonation status of the restraint system, vehicle speed, collision location, collision direction, number of occupants, occupant vital signs, and occupant posture is collected, and potential injury sites and injury levels of the occupants are predicted; the accident rescue information includes one or more of the following: number of occupants, occupant vital signs, potential injury sites, injury levels, restraint system execution status, vehicle collision information, and vehicle location information.

[0029] According to a second aspect of the present invention, an occupant protection system based on multimodal perception fusion is provided, applied to a vehicle, comprising:

[0030] Multimodal sensing components are used to collect occupant status data within the cockpit;

[0031] The vehicle status acquisition component is used to acquire vehicle driving status data and collision risk data;

[0032] The perception fusion component is used to fuse the occupant status data, vehicle driving status data, and collision risk data to generate occupant fusion status information.

[0033] The protection strategy generation component is used to determine the protection stage of the vehicle based on the occupant fusion state information and collision risk data, and to generate a corresponding occupant protection strategy. The protection stage includes a pre-collision stage, a collision stage, and a post-collision stage.

[0034] A constraint execution component is used to control the action of at least one actuator among the seat belt, airbag, and seat according to the occupant protection strategy;

[0035] The post-collision information processing component is used to generate accident rescue information based on occupant fusion state information, vehicle collision information and constraint execution state in the post-collision stage, and send the accident rescue information to the vehicle communication terminal and / or event data recorder.

[0036] The protection strategy generation component is used to execute at least one of the following pre-collision protection strategies in the pre-collision stage: posture reminder, active seat belt pretensioning, or seat reset; in the collision stage: to adjust the protection parameters of the airbag and / or seat belt; and in the post-collision stage: to trigger the generation and transmission of accident rescue information.

[0037] Optionally, the multimodal perception component includes an occupant classification system, an image acquisition unit, a radar perception unit, a seat status perception unit, and a seat belt status detection unit;

[0038] The occupant classification system is used to identify seat occupancy status and occupant body type.

[0039] The image acquisition unit is used to acquire posture information of the occupant's head, chest, torso, or limbs;

[0040] The radar sensing unit is used to acquire occupant distance information, motion information, breathing information, or heart rate information; the seat status sensing unit is used to acquire seat rail position, backrest angle, seat cushion angle, or zero-gravity seat status.

[0041] The seat belt status detection unit is used to obtain the seat belt buckle status, pull-out length, or tension status.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] First, this invention integrates multimodal sensing data from the OCS occupant classification system, image acquisition unit, radar sensing unit, and seat status sensing unit to identify occupant posture, vital signs, displacement status, distance between occupant and airbag, and occupant movement trend in real time, thereby improving the accuracy and stability of occupant status recognition.

[0044] Second, the present invention divides occupant protection into three consecutive stages: before, during, and after the collision. This makes the protection strategy no longer limited to mechanical detonation at the moment of the collision, but can dynamically adjust the protection parameters of the airbags and seat belts by combining the occupant status information before the collision and the real-time status during the collision.

[0045] Third, this invention can select different airbag deployment times, inflation volumes, seat belt pretension levels, and force limiting levels for different seating positions such as standard sitting posture, alighting posture, reclining posture, zero-gravity seat posture, head tilt, and torso tilt, thereby reducing the problem of increased risk of occupant injury due to excessive or insufficient restraint.

[0046] Fourth, this invention can simultaneously output information such as occupant vital signs, restraint system detonation status, potential damage locations and damage levels after a collision, providing rescue centers with more accurate accident information, improving rescue efficiency, and providing data support for accident reconstruction, property compensation and technological iteration. Attached Figure Description

[0047] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0048] Figure 1 This is the overall strategy flow of the method described in Example 1. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0051] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, all directional indications (such as up, down, left, right, front, back, bottom, etc.) in this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indication will also change accordingly. Furthermore, descriptions involving "first," "second," etc., in this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated.

[0052] Example 1

[0053] This embodiment provides an occupant protection method based on multimodal perception fusion, applied to vehicles. Its overall strategy flow is as follows: Figure 1 As shown.

[0054] First, occupant status data is collected within the cabin. Specifically, the OCS occupant classification system can be used to collect seat occupancy status and occupant body type; infrared cameras can be used to collect images of the occupant's head, chest, torso, and limbs, and identify the occupant's three-dimensional key point posture; millimeter-wave radar can be used to collect the distance between the occupant and the airbag, occupant breathing, heart rate, micro-movements, and movement trends; seat status sensors can be used to collect seat rail position, backrest angle, seat cushion angle, leg rest deployment status, and zero-gravity seat status; and seat belt status detection units can be used to collect seat belt buckle status, seat belt extension length, and seat belt tension.

[0055] Secondly, acquire vehicle driving status data and collision risk data. Vehicle driving status data can include vehicle speed, braking status, longitudinal acceleration, lateral acceleration, steering status, and vehicle attitude. Collision risk data can include the probability of collision, estimated collision time, collision direction, collision location, collision type, and collision severity. Collision risk data can come from external vehicle cameras, millimeter-wave radar, lidar, active safety control systems, collision sensors, or vehicle controllers.

[0056] Then, occupant status data, vehicle driving status data, and collision risk data are fused to generate occupant fusion status information. Specifically, data from different sources can be timestamped and synchronized, data collected by different sensors can be converted to a unified vehicle coordinate system or cockpit coordinate system, and the confidence level of each sensor data can be determined based on at least one of the following: data validity, data quality parameters, and the degree of fluctuation in continuously sampled data. Then, the sensor data are weighted and fused. Through fusion processing, occupant fusion status information can be obtained, including occupant presence, occupant body type, occupant vital signs, occupant 3D keypoint pose, distance between occupant and airbag, occupant disengagement status, occupant behavioral intent, and occupant movement trend.

[0057] The "displacement state" refers to a situation where the occupant is not in a normal, standard sitting posture. Examples include the occupant being too close to the airbag, leaning forward to retrieve an object, talking sideways, having their head significantly turned, their torso tilted to the side, or their body deviating from the center of the seat. By integrating data from infrared cameras, millimeter-wave radar, the OCS occupant classification system, and seat status sensors, the accuracy of displacement state detection can be improved.

[0058] Next, the vehicle's protection phase is determined based on occupant fusion status information and collision risk data. The protection phase includes a pre-collision phase, a collision phase, and a post-collision phase. When the vehicle has not yet experienced an actual collision, but experiences emergency braking, a collision probability exceeding a preset value, or an estimated collision time less than a preset value, it can be determined that the vehicle is in the pre-collision phase. When the collision sensor detects a collision impact, or the vehicle's deceleration exceeds a preset threshold, it can be determined that the vehicle is in the collision phase. When the collision impact ends and the vehicle enters accident rescue, accident recording, or accident communication status, it can be determined that the vehicle is in the post-collision phase.

[0059] In the pre-collision phase, a pre-collision protection strategy is implemented based on at least one of the following: occupant posture, distance between the occupant and the airbag, seat status, and vehicle braking status. If the occupant is in an dislodged position, or is not in a normal, standard seating posture and is too close to the airbag, a posture adjustment reminder is output. This posture adjustment reminder can be output through the vehicle's infotainment system, human-machine interface, voice prompts, instrument panel prompts, or audible and visual alerts.

[0060] If the vehicle brakes suddenly and the occupants tend to lean forward, and the vehicle is equipped with active pretensioners, the active pretensioners will be activated to restrain the occupants and reduce the risk of them deviating further from their standard seating position before a collision. If the seat is in a zero-gravity position, or the backrest angle exceeds a preset angle, causing the occupants to move away from the optimal airbag contact area, a seat reset reminder will be issued, or, if the vehicle safety control conditions are met, the seat rails, backrest, or leg rest will be controlled to perform a reset action.

[0061] During the collision phase, the protection parameters of the airbags and / or seat belts are adjusted based on the occupant fusion state information obtained before the collision and the real-time occupant state at the time of the collision. These protection parameters may include at least one of the following: airbag deployment time, airbag deployment level, airbag inflation volume, seat belt pretension level, seat belt force limiter level, and seat adjustment status.

[0062] Specifically, if the occupant is in a standard sitting position and the vehicle is involved in a frontal collision, the seat belts and airbags will be controlled to perform according to preset standard protection parameters based on the collision intensity and the occupant's body type. For example, the seat belt pretensioners will be controlled to deploy, the seat belt force limiters will be controlled to enter the corresponding force limit level, and the airbags will be controlled to deploy according to the standard deployment time and inflation volume.

[0063] If an emergency braking before a collision causes the occupant to lean forward, or if the occupant is already in an off-position state, resulting in their head or chest being too close to the airbag, the occupant's posture information and distance to the airbag are sent to the RCM restraint control module for assessment. The RCM can be understood as the restraint control module. The RCM restraint control module can control the seatbelt's priority deployment or pretensioning, while simultaneously controlling the airbag's deployment using the primary gas generator, reducing the airbag inflation volume, or delaying airbag deployment to avoid exacerbating occupant injury due to full-power airbag deployment.

[0064] If the occupant is in a relaxed sitting posture or the seat is in a zero-gravity position, causing the occupant to be far from the airbag, the deployment of the airbag can be delayed to match the timing of the airbag deployment with the occupant's forward movement trajectory; or the airbag can be controlled to deploy at the first level of power first, and then the second level of power is deployed with a delay, thereby providing a more suitable cushioning effect.

[0065] If an occupant leans forward due to inertia but their head turns to one side, or their torso is in a lateral tilt, the airbag deployment strategy and seatbelt force limiter can be adjusted based on the occupant's head direction, chest position, and collision direction to reduce the risk of head, neck, chest, or abdominal injuries.

[0066] In the post-collision phase, accident rescue information is generated based on occupant fusion status information, vehicle collision information, and restraint execution status, and is sent to the vehicle communication terminal and / or event data recorder. Specifically, information such as restraint system detonation status, vehicle speed, collision location, collision direction, vehicle deceleration, number of occupants, occupant posture, occupant vital signs, and seat and seat belt status can be summarized.

[0067] The detonation status of the restraint system can include whether the airbags have deployed, the airbag deployment level, whether the seat belts are pretensioned, and the seat belt force limiter level. Vehicle speed can include the vehicle speed before the collision or the vehicle speed at the moment of the collision. Occupant vital signs can include breathing, heartbeat, or micro-motion status detected by millimeter-wave radar.

[0068] Based on the above information, potential occupant injuries can be predicted. For example, if a vehicle is involved in a high-speed frontal collision, the airbags have deployed, the occupant was in a forward-leaning posture before the collision, and their vital signs are abnormal after the collision, it can be determined that the occupant is at risk of head, neck, or chest injuries. If a vehicle is involved in a side collision, the occupant's torso is tilted to the side, and the side airbags or side curtain airbags have deployed, it can be determined that the occupant is at risk of chest, abdominal, or head injuries.

[0069] Accident rescue information can include the number of occupants, occupant vital signs, potential injury locations, damage levels, vehicle location, collision location, collision direction, vehicle speed before collision, restraint system activation status, and vehicle identification information. Accident rescue information can be transmitted via the TBOX (Telematics Box) in-vehicle communication terminal to the rescue center, vehicle owner's terminal, vehicle back-end platform, or insurance claims platform, and simultaneously written to the EDR (Event Data Recorder). Here, TBOX stands for Telematics Box, i.e., the in-vehicle communication terminal; EDR stands for Event Data Recorder.

[0070] The method in this embodiment can realize full-cycle occupant protection based on multimodal perception fusion from before to after a collision, transforming vehicle safety protection from single-point triggering to continuous and adaptive protection.

[0071] Example 2

[0072] This embodiment provides an occupant protection system based on multimodal perception fusion, used to implement the method described in the first embodiment.

[0073] The occupant protection system includes a multimodal perception component, a vehicle state acquisition component, a perception fusion component, a protection strategy generation component, a constraint execution component, and a post-collision information processing component.

[0074] The multimodal perception component is used to collect occupant status data within the cabin. Specifically, the multimodal perception component may include at least two of the following: an OCS occupant classification system, an image acquisition unit, a radar sensing unit, a seat status sensing unit, and a seatbelt status detection unit.

[0075] The OCS occupant classification system can be installed on the seat cushion, seat frame, or seat pressure detection area to determine whether the seat is occupied and to identify the occupant's body type based on seat pressure distribution, occupant weight, or stress state. The occupant body type can include adults, children, smaller occupants, or larger occupants.

[0076] The image acquisition unit can be an infrared camera and can be positioned on the dashboard, A-pillar, roof, near the rearview mirror, or other locations capable of capturing occupant images. The image acquisition unit is used to acquire image information of the occupant's head, shoulders, chest, torso, arms, and legs, and to identify the occupant's three-dimensional keypoint pose. The three-dimensional keypoint pose can include the occupant's head position, chest position, shoulder position, torso tilt angle, head turning direction, and limb positions.

[0077] The radar sensing unit can be a millimeter-wave radar and can be installed in the ceiling, dashboard, seat back, seat side, or other locations inside the cabin. The millimeter-wave radar is used to acquire information such as the distance between the occupant and the airbag, occupant micro-movements, breathing, heartbeat, and movement trends. With millimeter-wave radar, occupant distance and vital signs data can still be obtained even in low light conditions, when occupant partially obstructs the view, or when camera recognition is limited.

[0078] The seat state sensing unit is used to acquire the seat slide rail position, backrest angle, seat cushion angle, leg rest deployment status, and zero-gravity seat status. With the increasing application of zero-gravity seats in vehicles, occupants may be in a state where the backrest is reclined, the leg rest is raised, and the body is far from the optimal contact area of ​​the airbag during driving. Therefore, seat state data plays a crucial role in the generation of subsequent protection strategies. The seat belt state detection unit is used to acquire whether the seat belt is buckled, the seat belt extension length, the seat belt tension, and the seat belt wearing status.

[0079] The vehicle status acquisition component is used to acquire vehicle driving status data and collision risk data. The vehicle driving status data may include vehicle speed, longitudinal acceleration, lateral acceleration, braking status, steering status, vehicle attitude, etc. The collision risk data may come from external vehicle cameras, external millimeter-wave radar, lidar, active safety systems, collision sensors, or vehicle controllers, and includes collision probability, estimated collision time, collision direction, collision location, collision type, and collision severity, etc.

[0080] The perception fusion component determines the confidence level of each sensor's data based on the data validity, data quality parameters, and the fluctuation level of continuously sampled data. Data validity can include whether the sensor is online, whether fault codes exist, whether communication is normal, and whether valid data is output. Data quality parameters can include the occlusion status, illumination status, and key point recognition completeness of the image acquisition unit, as well as the echo intensity, signal-to-noise ratio, and distance measurement validity of the radar perception unit. The fluctuation level of continuously sampled data can include the variation in occupant head position, chest position, distance between the occupant and the airbag, or vital sign data over multiple consecutive sampling periods. When a sensor malfunctions, has missing data, severe occlusion, low signal strength, or its continuous sampling data fluctuation exceeds a preset threshold, the confidence level of that sensor's data is reduced; conversely, when a sensor's data is continuously stable and consistent with other sensor data, the confidence level of that sensor's data is increased.

[0081] For example, when the infrared camera cannot accurately identify the occupant's chest position due to occupant obstruction, the system can increase the weight of millimeter-wave radar distance data and seat pressure data; when the millimeter-wave radar signal is weak, the system can increase the weight of the infrared camera and seat status sensor. This avoids misjudging occupant status due to a single sensor malfunction, improving the stability of occupant status recognition.

[0082] The occupant fusion status information may include the presence of an occupant, occupant body type, occupant vital signs, occupant three-dimensional key point posture, distance between the occupant and the airbag, occupant disengagement status, occupant behavioral intention, and occupant movement trend.

[0083] The protection strategy generation component is used to determine the vehicle's current protection phase based on occupant fusion status information and collision risk data, and to generate a corresponding occupant protection strategy. The protection phases include the pre-collision phase, the during-collision phase, and the post-collision phase. The protection strategy generation component can be integrated into the RCU constraint control unit, the RCM constraint control module, the cockpit domain controller, the safety domain controller, or the vehicle controller, or it can be set up as an independent control module. Here, RCU can be understood as a constraint control unit, and RCM can be understood as a constraint control module.

[0084] The restraint enforcement component is used to control the action of at least one actuator among the seat belt, airbag, and seat according to an occupant protection strategy. The seat belt may include an active pretensioner and a force limiter. The airbag may include a driver's airbag, a front passenger airbag, side airbags, side curtain airbags, knee airbags, and multi-stage airbags with primary and secondary gas generators. The seat may include a slide rail adjustment mechanism, a backrest adjustment mechanism, a seat cushion angle adjustment mechanism, and a leg rest adjustment mechanism.

[0085] In the pre-collision phase, if the system detects that the occupant is out of position or has deviated from the safe contact area, the protection strategy generation component can remind the occupant to adjust their posture through the vehicle's infotainment system, human-machine interface, voice prompts, instrument panel alerts, or audible and visual warnings. If the vehicle undergoes emergency braking and the occupant has a forward leaning tendency, and the vehicle is equipped with active pretensioner seat belts, the protection strategy generation component can control the active pretensioner seat belts to pretension in advance to restrain the occupant's body. If the seat is in a zero-gravity seat state, or the backrest angle exceeds a preset angle, causing the occupant to move away from the optimal contact area of ​​the airbag, the protection strategy generation component can output a seat reset reminder, or control the seat rails, backrest, or leg rest to perform a reset action.

[0086] During the collision phase, the protection strategy generation component determines at least one of the following protection parameters based on the occupant fusion state information obtained in the pre-collision phase and the real-time occupant state at the time of the collision: airbag deployment time, airbag deployment level, airbag inflation volume, seat belt pretension level, seat belt force limiting level, and seat adjustment state.

[0087] If the occupant is in a standard sitting position and a frontal collision occurs, the protection strategy generation component can control the seat belts and airbags to operate according to standard protection parameters based on the collision intensity and the occupant's body type. If the occupant is thrown forward due to emergency braking before the collision, or if the occupant is already in an off-position state, causing their head or chest to be too close to the airbag, the protection strategy generation component can send the occupant's posture information and distance to the airbag to the RCM restraint control module for judgment, and control the seat belts to deploy or pretension first, while controlling the airbags to deploy with the primary gas generator, reduce the inflation volume, or delay deployment. If the occupant's sitting posture is too relaxed, or the seat is in a zero-gravity position, causing the occupant to be far from the airbag, the airbag deployment can be delayed, or the airbag can be controlled to deploy with primary power first, followed by a delayed secondary power deployment. If the occupant's head is turned or their torso is tilted to the side, the airbag deployment strategy and seat belt force limiter level can be adjusted.

[0088] The post-collision information processing component is used to summarize vehicle collision information, restraint system execution status, and occupant status information in the post-collision stage, and generate accident rescue information. Accident rescue information may include the number of occupants, occupant vital signs, potential injury locations, injury levels, vehicle location, collision location, collision direction, vehicle speed before collision, restraint system detonation status, and vehicle identification information. The system can send accident rescue information to the rescue center, vehicle owner terminal, vehicle back-end platform, or insurance claims platform via the TBOX vehicle communication terminal, and simultaneously write the accident rescue information to the EDR event data logger.

[0089] In optional implementations, the image acquisition unit can employ, in addition to infrared cameras, visible light cameras, depth cameras, TOF cameras, or combinations of multiple cameras. TOF stands for Time of Flight. The radar sensing unit can employ, in addition to millimeter-wave radar, 4D millimeter-wave radar, in-cabin vital signs radar, ultra-wideband radar, or lidar. Multimodal data fusion, besides confidence-weighted fusion, can also employ Kalman filtering, Bayesian fusion, neural network models, decision tree models, or other machine learning models to achieve occupant status identification and risk level assessment.

[0090] The system of this embodiment can execute the occupant protection method described in Embodiment 1 to achieve continuous and adaptive occupant protection before, during, and after a collision.

[0091] The specific embodiments of the present invention have been described above. Based on the above description, those skilled in the art can make various changes and modifications without departing from the technical concept of the present invention.

Claims

1. An occupant protection method based on multimodal perception fusion, characterized in that, Includes the following steps: Collect occupant status data inside the cabin; Acquire vehicle driving status data and collision risk data; The occupant status data, vehicle driving status data, and collision risk data are fused to generate occupant fusion status information. Specifically, this includes: synchronizing data from different sources in time; converting data from different sources to a unified vehicle coordinate system or cockpit coordinate system; determining the confidence level of each source data based on at least one of the following: data validity, data quality parameters, and the degree of fluctuation of continuously sampled data; and fusing the data from each source based on the confidence level to generate the occupant fusion status information. The vehicle's protection phase is determined based on the occupant fusion status information and collision risk data. The protection phase includes the pre-collision phase, the collision phase, and the post-collision phase. In the pre-collision phase, at least one of the following pre-collision protection strategies is executed based on the occupant fusion state information: posture reminder, active seat belt pretensioning, or seat reset. During the collision phase, the protection parameters of the airbags and / or seat belts are adjusted based on the occupant fusion state information obtained in the pre-collision phase and the real-time occupant state at the time of the collision. In the post-collision phase, accident rescue information is generated based on occupant fusion status information, vehicle collision information, and constraint execution status, and the accident rescue information is sent to the vehicle communication terminal and / or event data recorder.

2. The occupant protection method based on multimodal perception fusion according to claim 1, characterized in that, The data collected on the occupant status in the cabin includes: obtaining seat occupancy status and occupant body type through an occupant classification system; obtaining posture information of the occupant's head, chest, torso, or limbs through an image acquisition unit; obtaining occupant distance, motion, breathing, or heart rate information through a radar sensing unit; obtaining seat rail position, backrest angle, cushion angle, or zero-gravity seat status through a seat status sensing unit; and obtaining seat belt buckle status, pull-out length, or tension status through a seat belt status detection unit.

3. The occupant protection method based on multimodal perception fusion according to claim 1, characterized in that, The occupant fusion status information includes one or more of the following: occupant presence, occupant body type, occupant vital signs, occupant three-dimensional key point posture, distance between occupant and airbag, occupant disengagement status, occupant behavioral intention, and occupant movement trend.

4. The occupant protection method based on multimodal perception fusion according to claim 1, characterized in that, In the pre-collision phase, at least one of the following pre-collision protection strategies is implemented: when the occupant is in an off-position state or deviates from the safe contact area, a posture adjustment reminder is output; when the vehicle undergoes emergency braking and the occupant has a tendency to lean forward, the seat belt is controlled to perform active pretensioning; when the seat is in a zero-gravity seat state, the backrest angle exceeds the preset angle, or the occupant is far from the optimal contact area of ​​the airbag, a seat reset reminder is output or the seat is controlled to reset.

5. The occupant protection method based on multimodal perception fusion according to claim 1, characterized in that, During the collision phase, based on the occupant fusion state information obtained in the pre-collision phase and the real-time occupant state at the time of the collision, at least one of the following protection parameters is determined: airbag initiation time, airbag deployment level, airbag inflation volume, seat belt pretension level, seat belt force limiting level, and seat adjustment status.

6. The occupant protection method based on multimodal perception fusion according to claim 5, characterized in that, During the collision phase: When the occupant is in a standard sitting position and a frontal collision occurs, the seat belts and airbags are controlled to perform according to the preset standard protection parameters; When the occupant is in an off-position state or the distance between the occupant and the airbag is less than the first distance threshold, the seat belt is pre-tensioned first, and the airbag is deployed in a way that reduces the inflation volume, delays the deployment, or deploys in a single-stage deployment mode. When the occupant is in a relaxed sitting position, a zero-gravity seat position, or the distance between the occupant and the airbag is greater than the second distance threshold, the airbag deployment is delayed, or the airbag is first deployed with a preset first-level inflation volume and then supplemented with a preset second-level inflation volume. When the occupant's head turns or torso tilts to the side, adjust the airbag deployment strategy and / or seatbelt force limiting strategy.

7. The occupant protection method based on multimodal perception fusion according to claim 1, characterized in that, In the post-collision phase, information such as the restraint system detonation status, vehicle speed, collision location, collision direction, number of occupants, occupant vital signs, and occupant posture is collected, and potential injury sites and injury levels of the occupants are predicted. The accident rescue information includes one or more of the following: number of occupants, occupant vital signs, potential injury sites, injury levels, restraint system execution status, vehicle collision information, and vehicle location information.

8. An occupant protection system based on multimodal perception fusion, applied to a vehicle, characterized in that, include: Multimodal sensing components are used to collect occupant status data within the cockpit; The vehicle status acquisition component is used to acquire vehicle driving status data and collision risk data; The perception fusion component is used to fuse the occupant status data, vehicle driving status data, and collision risk data to generate occupant fusion status information. The protection strategy generation component is used to determine the protection stage of the vehicle based on the occupant fusion state information and collision risk data, and to generate a corresponding occupant protection strategy. The protection stage includes a pre-collision stage, a collision stage, and a post-collision stage. A constraint execution component is used to control the action of at least one actuator among the seat belt, airbag, and seat according to the occupant protection strategy; The post-collision information processing component is used to generate accident rescue information based on occupant fusion state information, vehicle collision information and constraint execution state in the post-collision stage, and send the accident rescue information to the vehicle communication terminal and / or event data recorder. The protection strategy generation component is used to execute at least one of the following pre-collision protection strategies in the pre-collision stage: posture reminder, active seat belt pretensioning, or seat reset; in the collision stage: to adjust the protection parameters of the airbag and / or seat belt; and in the post-collision stage: to trigger the generation and transmission of accident rescue information.

9. The occupant protection system based on multimodal perception fusion according to claim 8, characterized in that, The multimodal perception component includes an occupant classification system, an image acquisition unit, a radar perception unit, a seat status perception unit, and a seat belt status detection unit. The occupant classification system is used to identify seat occupancy status and occupant body type. The image acquisition unit is used to acquire posture information of the occupant's head, chest, torso, or limbs; The radar sensing unit is used to acquire occupant distance information, motion information, breathing information, or heart rate information; the seat status sensing unit is used to acquire seat rail position, backrest angle, seat cushion angle, or zero-gravity seat status. The seat belt status detection unit is used to obtain the seat belt buckle status, pull-out length, or tension status.

Images