Active and passive security fusion object control method and device and computer equipment

Abstract

The invention relates to an active and passive security fusion object control method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an image data set and a point cloud data set outside a vehicle, and determining a to-be-collided object and an object feature set of the to-be-collided object based on the image data set and the point cloud data set; performing prediction processing on the object feature set and the control signal set of the vehicle according to a collision working condition prediction model to obtain a collision working condition feature set; and obtaining a passenger feature set of each passenger in the vehicle, and determining the detonation time of each airbag based on the collision condition feature set and each passenger feature set. By adopting the method, the accuracy of the object control method can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of automobiles, and particularly to an object control method, device, computer device, computer-readable storage medium, and computer program product for the integration of active and passive safety. Background Art

[0002] In the passive safety system of a vehicle, the airbag is an important component. During the daily driving of the vehicle, an object control method is used to control the airbag (object) to pop up to protect the safety of the passengers in the vehicle and improve the survival rate of the passengers in a collision accident.

[0003] The current object control method obtains the acceleration of the vehicle collected by various sensors in the vehicle and determines whether each acceleration reaches the preset airbag pop-up condition. If each acceleration meets the airbag pop-up condition, an ignition signal is sent to the airbag device to enable the airbag device to quickly generate gas to fill the airbag (object), so that the airbag pops up.

[0004] However, the current object control method only relies on the acceleration after a collision to determine whether to detonate the airbag, the detonation determinant is single, and the detonation time is inaccurate, which leads to a low accuracy of the current object control method. Summary of the Invention

[0005] Based on this, it is necessary to provide an object control method, device, computer device, computer-readable storage medium, and computer program product for the integration of active and passive safety in view of the above technical problems.

[0006] In a first aspect, the present application provides an object control method for the integration of active and passive safety, including:

[0007] Obtain an image data set and a point cloud data set outside the vehicle, and determine a to-be-collided object and an object feature set of the to-be-collided object based on the image data set and the point cloud data set;

[0008] Perform prediction processing on the object feature set and the control signal set of the vehicle according to a collision condition prediction model to obtain a collision condition feature set;

[0009] Obtain a passenger feature set of each passenger in the vehicle, and determine the detonation time of each airbag based on the collision condition feature set and each passenger feature set.

[0010] In one of the embodiments, the determining the to-be-collided object and the object feature set of the to-be-collided object based on the image data set and the point cloud data set includes:

[0011] Performing image recognition on each image data in the image dataset based on an image recognition model to obtain a to-be-collided object and an initial object feature set of the to-be-collided object;

[0012] Determining the speed and volume of the to-be-collided object according to the point cloud dataset and the image dataset;

[0013] Updating the initial object feature set based on the speed and volume of the to-be-collided object to obtain an object feature set.

[0014] In one embodiment, the performing image recognition on each image data in the image dataset based on an image recognition model to obtain a to-be-collided object and an initial object feature set of the to-be-collided object includes:

[0015] Performing image recognition on each image data in the image dataset based on an image recognition model to obtain each initial to-be-collided object and the category of each initial to-be-collided object;

[0016] Screening the to-be-collided object from each initial to-be-collided object according to a preset collision screening condition;

[0017] Querying the mass of the to-be-collided object in a database based on the category of the to-be-collided object, and constructing an initial object feature set of the to-be-collided object according to the category and mass of the to-be-collided object.

[0018] In one embodiment, the obtaining a passenger feature set of each passenger in the vehicle includes:

[0019] Obtaining an attribute feature set of each passenger through each seat sensor in the vehicle;

[0020] Obtaining a posture feature set of each passenger based on an image acquisition device and / or an infrared sensor;

[0021] Constructing a passenger feature set of each passenger according to each attribute feature set and each posture feature set.

[0022] In one embodiment, the determining the initiation time of each airbag based on the collision condition feature set and each passenger feature set includes:

[0023] Determining a first collision time between the vehicle and the to-be-collided object based on the speed and acceleration of the vehicle and a first distance between the vehicle and the to-be-collided object;

[0024] Determining an initial collision acceleration according to a preset collision process time, a speed difference in the collision condition feature set, an object feature set of the to-be-collided object, and attribute information of the vehicle;

[0025] For each of the passengers, based on the passenger feature set of the passenger, the initial collision acceleration, and the collision pose in the collision condition feature set, determine the second collision time of the passenger; the second collision time is the time from the head and neck of the passenger to the vehicle interior device;

[0026] Based on the first collision time and the second collision time, determine the activation time of the airbag corresponding to the passenger.

[0027] In one embodiment, for each of the passengers, based on the passenger feature set of the passenger, the initial collision acceleration, and the collision pose in the collision condition feature set, determining the second collision time of the passenger includes:

[0028] For each of the passengers, based on the attitude parameters and position vectors in the passenger feature set of the passenger, determine the second distance between the head and neck of the passenger and the device in the vehicle;

[0029] According to the seat belt restraint state and basic attributes in the passenger feature set and the collision pose in the collision condition feature set, correct the initial collision acceleration to obtain the collision acceleration;

[0030] According to the collision time algorithm, perform data processing on the collision acceleration and the second distance to obtain the second time to be collided.

[0031] In one embodiment, after determining the activation time of each airbag based on the collision condition feature set and each passenger feature set, the method further includes:

[0032] According to the passenger feature set of each passenger and the collision condition feature set, determine whether the airbag corresponding to each passenger needs to be activated;

[0033] In the case where the target airbag of the target passenger needs to be activated, generate a target ignition signal based on the target activation mode and activation time corresponding to the target passenger;

[0034] Send the target ignition signal to the airbag device to indicate that the target airbag corresponding to the target passenger pops up.

[0035] In a second aspect, the present application also provides an object control device for integrated active and passive safety, including:

[0036] An acquisition module, configured to acquire an image data set and a point cloud data set outside the vehicle, and determine a to-be-collided object and the object feature set of the to-be-collided object based on the image data set and the point cloud data set;

[0037] A prediction module, configured to perform prediction processing on the object feature set and the control signal set of the vehicle according to a collision condition prediction model to obtain a collision condition feature set;

[0038] A determination module, configured to obtain a passenger feature set of each passenger in the vehicle, and determine the inflation time of each airbag based on the collision condition feature set and each passenger feature set.

[0039] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0040] Obtain an image data set and a point cloud data set outside the vehicle, and determine a to-be-collided object and an object feature set of the to-be-collided object based on the image data set and the point cloud data set;

[0041] Perform prediction processing on the object feature set and the control signal set of the vehicle according to a collision condition prediction model to obtain a collision condition feature set;

[0042] Obtain a passenger feature set of each passenger in the vehicle, and determine the inflation time of each airbag based on the collision condition feature set and each passenger feature set.

[0043] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Obtain an image data set and a point cloud data set outside the vehicle, and determine a to-be-collided object and an object feature set of the to-be-collided object based on the image data set and the point cloud data set;

[0045] Perform prediction processing on the object feature set and the control signal set of the vehicle according to a collision condition prediction model to obtain a collision condition feature set;

[0046] Obtain a passenger feature set of each passenger in the vehicle, and determine the inflation time of each airbag based on the collision condition feature set and each passenger feature set.

[0047] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0048] Obtain an image data set and a point cloud data set outside the vehicle, and determine a to-be-collided object and an object feature set of the to-be-collided object based on the image data set and the point cloud data set;

[0049] Perform predictive processing on the object feature set and the control signal set of the vehicle according to the collision condition prediction model to obtain a collision condition feature set;

[0050] Obtain the passenger feature sets of each passenger in the vehicle, and determine the activation time of each airbag based on the collision condition feature set and each passenger feature set.

[0051] The above object control method, device, computer device, computer-readable storage medium and computer program product for active and passive safety integration obtain an image data set and a point cloud data set outside the vehicle, and determine a to-be-collided object and the object feature set of the to-be-collided object based on the image data set and the point cloud data set; perform predictive processing on the object feature set and the control signal set of the vehicle according to the collision condition prediction model to obtain a collision condition feature set; obtain the passenger feature sets of each passenger in the vehicle, and determine the activation time of each airbag based on the collision condition feature set and each passenger feature set. By using this method, the collision condition feature set is determined through the object feature set of the to-be-collided object, and the activation time of the airbag corresponding to each passenger is determined according to the collision condition feature set and the passenger feature set. Compared with directly detonating the airbag through acceleration, considering the three influencing factors of the to-be-collided object, the collision state of the vehicle and the passenger state, an accurate activation time is obtained, and then the airbag is detonated based on the accurate activation time, improving the accuracy of the object control method. Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0053] Figure 1 It is a schematic diagram of the control process of a traditional object control method in an exemplary embodiment;

[0054] Figure 2 It is a schematic diagram of the airbag action time under a traditional object control method in an exemplary embodiment;

[0055] Figure 3 It is a schematic flowchart of an object control method for active and passive safety integration in an embodiment;

[0056] Figure 4 It is a structural diagram of a collision condition prediction model in an exemplary embodiment;

[0057] Figure 5 It is an operation schematic diagram of a collision condition prediction model in an exemplary embodiment;

[0058] Figure 6 Schematic diagram of input and output of a collision condition model in an exemplary embodiment;

[0059] Figure 7 Schematic diagram of the process for determining a target collision object and the object feature set of the target collision object in an embodiment;

[0060] Figure 8 Schematic diagram of input and output of a multi-modal recognition model in an exemplary embodiment;

[0061] Figure 9 Schematic diagram of the process for determining a target collision object in an embodiment;

[0062] Figure 10 Schematic diagram of the operation process of an image recognition model in an exemplary embodiment;

[0063] Figure 11 Schematic diagram of the process for obtaining a passenger feature set in an embodiment;

[0064] Figure 12 Schematic diagram of the process for determining the detonation time in an embodiment;

[0065] Figure 13 Schematic diagram of the process for determining the second collision time in an embodiment;

[0066] Figure 14 Schematic diagram of the process for generating a target ignition signal in an embodiment;

[0067] Figure 15 Schematic diagram of input and output for determining an airbag ignition model in an exemplary embodiment;

[0068] Figure 16 Schematic diagram of the architecture of a control system in an exemplary embodiment;

[0069] Figure 17 Block diagram of the structure of an object control device for the integration of active and passive safety in an embodiment;

[0070] Figure 18 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0071] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0072] A large number of road traffic accidents occur globally every year, resulting in a huge loss of human resources and property. Vehicle collision safety and participant protection are the pillar studies for improving road traffic safety. In the passive safety system of vehicles, airbags are an important component. Airbags can improve the survival rate of passengers in the vehicle during a collision. Therefore, during the daily driving process of the vehicle, it is necessary to control the deployment of the airbag (object) through an object control method to protect the safety of the passengers in the vehicle and improve the survival rate of the passengers in the vehicle during a collision.

[0073] The current object control method obtains the acceleration of the vehicle collected by various sensors in the vehicle and determines whether each acceleration reaches the preset airbag deployment condition. If each acceleration meets the airbag deployment condition, an ignition signal is sent to the airbag device to enable the airbag device to quickly generate gas to fill the airbag (object), so that the airbag is deployed.

[0074] Specifically, Figure 1 is a schematic diagram of the control process of the traditional object control method in an exemplary embodiment. As Figure 1 shown, multiple airbag sensors are installed in the vehicle to monitor the collision situation of the vehicle. Each airbag sensor is respectively the safety sensor of the airbag itself, the front airbag sensor, and the central airbag sensor. Figure 1 It also includes an Electronic Control Unit (ECU for short). When a vehicle collision occurs, the airbag sensors detect the acceleration change, and then judge the severity of the collision based on the acceleration. When the sensors detect a strong enough collision signal, that is, when the acceleration reaches the airbag deployment condition, the control unit (ACU, Airbag Control Unit, also known as the airbag control unit) activates the airbag device, ignites the gas generator and quickly inflates the airbag. The gas generator usually uses chemical reactions or electrical ignition to generate gas, and these gases quickly fill the airbag. The gas is usually nitrogen or other inert gases. After the airbag is inflated, the airbag quickly expands and fills the vehicle space, forming a buffer zone to slow down the forward speed of the occupants and reduce the force of direct impact. When the collision ends, the airbag quickly deflates through small holes or valves so that the passengers can safely disengage from the airbag and return to the normal position.

[0075] However, the current object control method only relies on the acceleration after the collision to determine whether to detonate the airbag. The detonation determinant is single, and the detonation time is inaccurate, which leads to a low accuracy of the current object control method.

[0076] Specifically, the current object control method relies on the instantaneous acceleration after a collision to determine the airbag deployment timing and force, which often has a lag. Especially at the moment before a collision occurs, since the entire collision process only lasts for 200 ms (milliseconds), it is impossible to adjust the airbag deployment strategy in real time after the collision occurs. The effective action time domain of the airbag is only a few tens of milliseconds, and the occupant cannot obtain sufficient protection whether contacting the airbag too early or too late. Figure 2 It is a schematic diagram of the airbag action time in an exemplary embodiment. As Figure 2 shown, the gas generator will trigger the gas generator at 0 - 15 milliseconds or 0 - 20 milliseconds. At about 30 milliseconds, the airbag will open. At 40 milliseconds to 60 milliseconds, the airbag will be fully inflated and come into contact with the passenger. At 60 milliseconds to 120 milliseconds, the gas in the airbag will gradually be discharged.

[0077] Moreover, the rapidly ejected airbag is likely to cause secondary injuries to the passengers. Specifically, at an airbag deployment speed of up to 300 km / h (kilometers per hour), the airbag is like a "small bomb" that rapidly expands inside the vehicle and will generate a great impact force. When the airbag has not been fully deployed, the occupant should not come into contact with the airbag. If the passenger has leaned forward excessively or is close to the airbag deployment path before the airbag is fully deployed, they are likely to suffer secondary injuries. This is because the collision only lasts for about 200 ms, and the airbag must be deployed in a very short time (about 20 ms) after the collision occurs to be effective. Therefore, usually, a powerful explosive is required to detonate it. If the airbag detonation time can be advanced before the collision, the airbag deployment speed and its resulting additional hazards can be effectively reduced. Also, for different occupants, the contact part between the airbag and the occupant will be different. The position of a smaller-sized passenger will be closer to the airbag. Although the airbag is originally designed to come into contact with an adult's chest, in the face of children or adults with a smaller build, the airbag may hit their faces. Before the airbag detonates, due to the collision that occurs, the position of the passenger changes on the airbag deployment path, which will also cause the airbag not to come into normal contact with the passenger. For example, braking before the collision or before the airbag detonates, the space between the passenger and the airbag is smaller, or the vehicle leaves the road surface, or a long-duration collision will cause the airbag to detonate later or the passenger fails to use the seat belt correctly.

[0078] Furthermore, the current object control method has the problem of insufficient accuracy in collision detection. Specifically, traditional object control methods usually rely on acceleration sensors and pressure sensors for collision detection and airbag detonation decision-making, which makes the system have a low tolerance for sensor failures or sensor data errors. When complex traffic environments or irregular collisions occur, a single sensor may not be able to provide accurate collision information, resulting in inaccurate airbag detonation.

[0079] In addition, the current object control method also has the problem of lack of flexibility in the airbag deployment mode. Specifically, most of the existing airbags can only select a fixed detonation time and a single airbag deployment mode (single-stage detonation), and cannot be dynamically adjusted according to different types and severities of collisions. For example, the airbag deployment requirements for minor collisions and severe collisions are completely different. For example, in the case of a minor collision, the airbag detonation time is late and the inflation volume is small, while in the case of a severe collision, the airbag detonation time is early and the inflation volume is large to better inhibit the forward movement of the passenger and prevent collision with the interior trim (vehicle components).

[0080] The current object control method also ignores the state of the passengers. Specifically, the existing object control methods do not consider key factors such as the body characteristics, postures, and whether the seat belts are fastened of the occupants. The passenger state has an important impact on the airbag protection effect. For example, unbelted or out-of-position (such as lying posture) occupants will contact the airbag too early or too late during a collision, resulting in the inability to achieve the airbag protection effect and even suffering airbag injuries. People with significantly different body types cannot be individually protected under the same detonation criterion.

[0081] In response to the current object control method, the present application provides an object control method for the integration of active and passive safety. The collision condition feature set is determined through the object feature set of the object to be collided, and the detonation time of the airbag corresponding to each passenger is determined according to the collision condition feature set and the passenger feature set. Compared with directly detonating the airbag through acceleration, this method considers three influencing factors: the object to be collided, the collision state of the vehicle, and the passenger state, obtains an accurate detonation time, and then detonates the airbag based on the accurate detonation time, improving the accuracy of the object control method.

[0082] In one embodiment, as Figure 3 shown, an object control method for the integration of active and passive safety is provided. In the embodiments of the present application, taking the application of this method to the control system in a vehicle as an example for illustration, the embodiments of the present application do not limit the execution device of the object control method for the integration of active and passive safety, including the following steps 302 to step 306:

[0083] Step 302, obtain an image data set and a point cloud data set outside the vehicle, and determine the object to be collided and the object feature set of the object to be collided based on the image data set and the point cloud data set.

[0084] Among them, the object feature set includes the attribute features and motion features of the object to be collided. The point cloud data set is the data collected by the radar device.

[0085] In implementation, an image acquisition device and a radar device are installed on the vehicle. The image acquisition device collects various image data outside the vehicle in real time, obtains an image data set, and transmits the image data set to the control system. At the same time, the radar device collects various point cloud data outside the vehicle in real time, obtains a point cloud data set, and transmits the point cloud data set to the control system. The control system receives the image data set and the point cloud data set. Then, the control system performs data processing on the image data set based on an image recognition model to obtain a to-be-collided object and an initial object feature set of the to-be-collided object. The control system updates the initial object feature set based on the point cloud data set and the image data set to obtain an object feature set.

[0086] Specifically, the radar device is a lidar. The image acquisition device collects various image data outside the vehicle in real time, obtaining an image data set. At the same time, the lidar also collects various point cloud data outside the vehicle in real time, obtaining a point cloud data set. Then, the image acquisition device transmits the image data set to the control system through a transmission channel inside the vehicle. At the same time, the radar device transmits the image data set to the control system through a transmission channel inside the vehicle. The control system performs image recognition on the image data set based on an image recognition model to obtain a to-be-collided object and an initial object feature set of the to-be-collided object. The control system performs data processing on the point cloud data set and the image data set to obtain the speed and volume of the to-be-collided object. Then, the control system updates the initial object feature set based on the speed and volume of the to-be-collided object to obtain an object feature set.

[0087] In an alternative embodiment, the radar device can be a lidar or a millimeter-wave radar or a combination of a lidar and a millimeter-wave radar. The image acquisition device includes, but is not limited to, a vision camera. The vision camera collects an image data set and uses deep learning methods for the detection and prediction of the to-be-collided object. And, high-precision three-dimensional modeling is provided by the lidar to obtain a point cloud data set. Additionally, the image data set and the point cloud data set can be obtained by different devices.

[0088] In an alternative embodiment, the image data set and the point cloud data set are obtained based on different sensor combinations. For example, only relying on the radar and ultrasonic sensors to determine the to-be-collided object and using an inertial measurement unit (IMU) to assist in collision prediction. The single-sensor solution has a lower cost and relatively simple technology, and is suitable for use in low-cost vehicles or simple environments. However, in complex environments (such as low visibility, bad weather, etc.), the accuracy and robustness of a single sensor are poor. The sensor combination solution has enhanced reliability, but will increase the complexity and cost of the system.

[0089] Step 304, perform prediction processing on the object feature set and the control signal set of the vehicle according to a collision condition prediction model to obtain a collision condition feature set.

[0090] Among them, the control signal set of the vehicle characterizes the operating state of the vehicle. The collision condition feature set characterizes the state when the vehicle collides with the object to be collided.

[0091] In implementation, a collision condition prediction model is set in the control system. The control system obtains the speed, acceleration, and intervention signal set of the vehicle, and constructs the control signal set of the vehicle based on the speed, acceleration, target position, and intervention signal set. The control system inputs the object feature set and the control signal set into the collision condition prediction model, and performs prediction processing on the object feature set and the control signal set through the collision condition prediction model to obtain the collision condition feature set. The collision condition feature set characterizes the severity of the collision between the vehicle and the target vehicle. The collision condition feature set at least includes the speed difference and the collision pose. The speed difference is the change in speed before and after the collision, and the collision pose includes the collision direction and the position of the vehicle after the collision. For example, the collision direction includes frontal collision and side collision. The position of the vehicle after the collision characterizes whether the vehicle rolls over, flips, etc.

[0092] Specifically, an initial collision condition prediction model is preset in the control system. The control system obtains the training data set from each database and trains the initial collision condition prediction model based on the training data set to obtain the collision condition prediction model. The control system obtains the static prior information of the vehicle from the database, and obtains the speed, acceleration, target position, and intervention signal set of the vehicle. The intervention signal set is the action signal for the active safety control module of the vehicle to actively slow down or avoid before or at the initial stage of the collision. For example, braking signals, steering signals, and other intervention action signals, etc. The control system constructs a signal set according to the speed, acceleration, target position, and intervention signal set. Then, the control system inputs the static prior information of the vehicle, the object feature set, and the control signal set into the collision condition prediction model, and performs prediction processing on the object feature set and the control signal set through the collision condition prediction model to obtain the collision condition feature set.

[0093] In an exemplary embodiment, the control system obtains the vehicle structure model and collision response characteristic information of the vehicle from a database, combines the vehicle structure model and the collision response characteristic information to obtain the static prior information of the vehicle. The control system obtains the static prior information of the vehicle from the database, and obtains the speed, acceleration, target position, and intervention signal set of the vehicle. The control system constructs a signal set based on the speed, acceleration, target position, and intervention signal set. The collision condition prediction model includes a Transformer Encoder (the core part of the Transformer model, and the Transformer model is a deep learning model), a Vision Transformer (including a visual Transformer and a sliding window Transformer, the visual Transformer is abbreviated as ViT, and the sliding window Transformer is abbreviated as Swin), an enhanced MLP (Multi-Layer Perceptron, multi-layer perceptron, residual & self-attention), a multi-head cross-attention gating fusion module, and a shared fully connected layer. The control system inputs the object feature set and the static prior information into the enhanced MLP in the collision condition prediction model, and inputs the image data set into the Vision Transformer in the collision condition prediction model. At the same time, the control system inputs the control signal set into the Transformer Enooder in the collision condition prediction model. The input static prior information, image data set, object feature set, and control signal set are feature-encoded by the Transformer Encoder, Vision Transformer, and enhanced MLP, and the initial features obtained by the feature encoding are feature-fused based on the multi-head cross-attention gating fusion module to obtain an initial feature set. Then, the initial feature set is subjected to prediction processing through the shared fully connected layer in the collision condition prediction model to obtain a collision condition feature set.

[0094] Specifically, Figure 4 is a structural diagram of the collision condition prediction model in an exemplary embodiment. As Figure 4As shown in the figure, the collision condition prediction model includes a time series data branch, an image or point cloud branch, a structural prior branch, an enhanced MLP, a multi-head cross-attention gating fusion module, and a shared fully connected layer. Among them, the shared fully connected layer includes a collision type classification module (Softmax + Focal Loss, where Softmax is normalization and Focal Loss is focal loss), a collision severity regression module, and a collision attitude regression module. The time series data branch uses a time series encoding module based on Transformer Encoder to process time series signals such as vehicle speed, acceleration, and steering wheel angle. Compared with traditional RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), and GRU (Gated Recurrent Unit) models, Transformer Encoder can more efficiently capture long-range dependencies and complex non-linear features using the self-attention mechanism, thus significantly improving the expression ability of time series information. The image / point cloud branch uses a visual information encoding module based on Vision Transformer (including implementation schemes such as ViT or Swin Transformer) to extract global semantic information and local detail features from camera images or point cloud data. This module overcomes the limitations of traditional CNN (Convolutional Neural Network) or 3D-CNN (3D Convolutional Neural Network) in capturing global relationships and enhances the model's perception ability of complex environmental information. The structural prior branch is designed for the structural parameters of the vehicle (such as stiffness, mass, dimensions, etc.) and uses an enhanced multi-layer perceptron (MLP) module. This MLP module introduces residual connections and self-attention mechanisms on the basis of traditional fully connected layers to fully explore the internal correlations between various structural parameters and form prior features with higher expression ability.

[0095] Each branch encodes the input data to obtain initial features. Then, through the multi-head cross-attention and gated fusion module, the intermediate layer outputs of each branch are deeply fused (feature fusion) to obtain an initial feature set, realizing the adaptive weighting and dynamic interaction of different modality information. The fused comfort feature set is then passed to the shared fully connected layer, which adopts a residual connection and layer normalization strategy to ensure the stability and generalization performance of feature transmission. Finally, the comprehensive features processed by the shared fully connected layer enter three task-specific output heads respectively. The shared fully connected layer includes: a collision type classification head, a collision severity regression head, and a collision attitude regression module. The collision severity regression module is used to predict collision severity indicators such as Δv and acceleration waveforms; the collision attitude regression module is used to predict collision attitude parameters such as roll angle and yaw angle.

[0096] Before inputting the object feature set, control signal set, and static prior information into the collision condition prediction model, it is necessary to perform prediction preprocessing on the object feature set, control signal set, and static prior information. Specifically, the control system fuses the control signal set, object feature set, and static prior information to obtain a target feature parameter set (target mass m, dimensions (l, w, h), speed v, acceleration a, comprehensive stiffness coefficient k) for use by the backend collision prediction model. The object feature set is the collision target feature obtained by multi-sensor fusion (vision, lidar, millimeter-wave radar). The object feature set is used as the primary input (primary input 2) to enter the collision condition prediction model. Optionally, the control system can also construct a high-dimensional feature vector based on the control signal set, object feature set, and static prior information. Specifically, the high-dimensional feature vector includes dynamic parameters (relative speed Δv, relative position, predicted collision angle), static parameters (structural stiffness of the target vehicle body, characteristics of the front crush zone), and environmental parameters (road surface conditions, weather, presence of other obstacles). The control system quickly retrieves the most similar historical scenarios or simulation cases according to the "structural model" and "collision response characteristics" of the target vehicle model or similar vehicle models to obtain prior deformation coefficients, acceleration waveform references, etc. If a highly similar case cannot be found, the general stiffness and crush zone reference data of the same type of vehicle are selected as a fallback. The control system splices the kinematic data and sensor data of the vehicle for several frames (such as 15 frames before collision, 30 fps (frames per second)) into a temporal tensor, and adds the structural stiffness coefficient, mass, and external dimensions of the target vehicle body, etc., to merge into the same feature vector. Then, the control system normalizes variables such as speed and acceleration using standardization or Min-Max (minimum-maximum) normalization to maintain the numerical stability of the input features.

[0097] The collision prediction model predicts collision accident characteristics that are strongly correlated with passenger injury severity, generating a collision condition feature set. This collision condition feature set includes: 1. Δv (velocity difference): the change in velocity before and after the collision. 2. Collision posture: The spatial position and attitude (roll, yaw) characteristics of the vehicle at the moment of collision. 3. Acceleration waveform: The acceleration time history at the time of collision, used for optimizing airbag control strategies. 4. Structural deformation: The deformation state of the vehicle body after the collision, used to assess the vehicle's residual safety margin.

[0098] Figure 5 FIG. 1 is a schematic diagram of the operation of a collision condition prediction model in an exemplary embodiment. Figure 5 As shown in the figure, after the vehicle is powered on, the control system loads the inference engine to execute the object control method for active and passive safety fusion until the vehicle completes its journey. The vehicle's sensors and image acquisition devices collect image data and point cloud datasets. These datasets represent environmental perception data. Simultaneously, the sensors also collect vehicle status data, including velocity and acceleration. Based on the image data and point cloud datasets, the control system determines the potential collision object and its object dataset. The control system cleans and assembles features from the object dataset, vehicle status data, and static prior information to obtain an initial target feature parameter set. The control system then obtains an intervention signal set. This includes active safety signals such as braking signals, steering signals, and ESC (Electronic Stability Control) status. The control system adds the intervention signal set to the initial target feature parameter set to obtain a target feature parameter set. The control system then inputs the target feature parameter set into the collision scenario prediction model, which then predicts and processes the target feature parameter set to obtain a collision scenario feature set. The collision scenario feature set includes speed differences. The control system calibrates the collision condition feature set based on the confidence level. If the collision condition confidence level falls below the confidence threshold, the control system performs a secondary search or multi-model run-in. Based on the collision condition feature set and the passenger feature set, the control system determines the control strategy for each airbag, including but not limited to adjusting the detonation timing and the airbag deployment method. If the control system experiences an anomaly or failure, such as a sensor failure, the control system controls the airbag using traditional object control methods.

[0099] On this basis, a deep learning algorithm (collision condition prediction model) is used to perform multi-task learning on tags such as collision types (frontal, side, rear-end, rollover, etc.) and collision severity (Δv (speed difference), peak acceleration, deformation level). The vehicle active safety module (braking, steering intervention) will be activated before or at the initial stage of a collision, which may change the collision trajectory and relative speed. During each cycle of inference, the latest braking or steering information (i.e., the intervention signal set) is incorporated into the model input, or used as a correction term to adjust the output Δv and collision attitude. If the active safety action significantly decelerates or changes the direction, the collision severity prediction is updated in real time.

[0100] Due to the extremely short prediction window before a collision and the high real-time requirements for vehicle safety control, prediction errors have a greater impact on passenger injuries. If the deep learning model has time-consuming predictions or limited resources, nearest neighbor searches (such as KD-Tree (K-Dimensional Tree, a binary tree data structure), LSH hashing (Locality-Sensitive Hashing, a technique for approximate nearest neighbor search in high-dimensional data)) can also be directly performed on the object feature set and the control signal set without the intervention signal set (relative speed, angle, structure type, etc.) and the collision case library to find several historical scenarios or simulation results with the highest similarity, so as to approximately infer the characteristics of the collision accident. This search result can complement or verify the output of the neural network to improve the prediction robustness.

[0101] In an optional embodiment, an initial collision condition prediction model is pre-set in the control system. The control system collects a training data set from each data source. Specifically, each data source includes a real accident database, a virtual risk generation library, a target vehicle model collision simulation database, and a typical experimental calibration airbag scenario matrix. The control system collects, collates, and annotates collision scenario data from each data source, and combines sensor simulation data (the "simulated output" of lidar, vision, and millimeter-wave radar in a virtual scenario) with real road test data to form a training sample set. Data resources from the "typical experimental calibration airbag scenario matrix", "target vehicle model collision simulation database", "virtual risk generation library", and "real accident database" provide prior information on different target types, collision scenarios, vehicle body structure models, and historical real accidents, supporting the modeling and simulation of collision prediction. The typical experimental calibration airbag scenario matrix calibrates the triggering and deployment characteristics of the airbag system under different collision types, vehicle speeds, and collision angles in the laboratory through a large number of actual collision tests and airbag deployment tests. This matrix contains standardized collision scenario parameters and airbag response mechanisms, providing reference data for the collision prediction model. The real accident database collects a large amount of real traffic accident data, including vehicle type, collision angle, speed difference, and collision consequences (degree of vehicle damage, casualty situation). These data provide real-world priors for the calibration and verification of the collision prediction model. The virtual risk generation library uses simulation tools to generate a large amount of virtual pre-collision scenario data. By varying vehicle relative position, speed difference, road conditions, and weather conditions through a parameterization method, a comprehensive virtual risk scenario database is established to provide more scenario references beyond the scope of real tests for collision prediction. The target vehicle model collision simulation database is for common vehicle target models (such as sedans, SUVs (Sports Utility Vehicles), trucks, etc.), and uses computer simulation to establish a collision model library. Using multi-body dynamics models and finite element analysis or hybrid methods, the collision process is predicted and simulated based on the input parameters. A large number of multi-body dynamics are used to quickly predict the pose change and acceleration characteristics of the vehicle at the moment of collision. A small batch of finite element analysis can accurately model the deformation, stress distribution, and energy absorption characteristics of the vehicle structure under the condition of higher computational cost. This database contains the impact simulation results in the real accident database and virtual risk generation library under different initial conditions (speed, angle, position), including the degree of deformation, collision energy absorption characteristics, and failure modes of the vehicle body load-bearing area. The collection frequency of the above various types of data is generally 100 - 1000 Hz (Hz is hertz according to sensor / test requirements); the key outputs of annotation include collision type labels (Front (front collision) / Side (side collision) / Rear (rear collision) / Multiple (multiple collisions) / rollover, etc.), collision severity labels (Δv range, peak acceleration, vehicle body deformation level), and vehicle attitude and position information (roll angle, yaw angle, etc.).

[0102] The training dataset contains various training data subsets, each of which represents a sample collision event. Each training data subset contains a training input data subset and a validation input data subset. The training input data subset at least contains the control signal set of the sample vehicle, the static prior information of the sample vehicle, and the object feature set of the sample target object in the sample collision event. The validation input data subset at least contains the sample collision condition feature set of the sample collision event. The control system inputs each training input data subset into the initial collision condition prediction model, performs prediction processing on each training input data subset through the initial collision condition prediction model to obtain each sample prediction result, and performs data processing on each sample prediction result and each validation input data subset to obtain the loss value of the initial collision condition prediction model. The control system determines whether the loss value exceeds a preset loss threshold. If the loss value does not exceed the loss threshold, the control system determines the initial collision condition prediction model as the collision condition prediction model. If the loss value exceeds the loss threshold, the control system continues to execute the step of inputting each training input data subset into the initial collision condition prediction model until the loss value does not exceed the loss threshold.

[0103] The collision condition prediction model obtained through training can, 500 milliseconds before a collision, combine real-time data and historical data to calculate the severity of the collision between the vehicle and the object to be collided, and predict the collision condition feature set. The collision condition feature set represents the type, location, and direction of the collision. The collision condition feature set includes the collision pose (frontal collision, side collision), and the collision severity (change in collision speed, acceleration waveform, deformation of the vehicle's own structure). When it is detected that a collision is about to occur or there is a high risk, the control system calls a deep learning model trained offline for rapid inference (typical cycle <10 - 20 ms (milliseconds)) to obtain the collision accident feature set (Δv, acceleration waveform, collision angle, vehicle deformation level, etc.).

[0104] In an exemplary embodiment, Figure 6 is a schematic diagram of the input and output of the collision condition model in an exemplary embodiment. As Figure 6As shown, the inputs of the collision condition model include Primary Input 1, Primary Input 2, and Primary Input 3. Primary Input 1 is the input when training the initial collision condition model to obtain the collision condition model. Primary Input 1 includes a training data set constructed from a typical experimental calibration airbag deployment scenario matrix, a collision simulation database of the target vehicle model, a virtual dangerous state generation library, and a real accident database. Primary Input 2 is the object feature set of the object to be collided with, which is also the collision target feature. Primary Input 3 is the control signal set of the vehicle. The collision condition prediction model performs collision prediction processing on the data of the primary inputs to obtain a collision condition feature set. The collision condition feature set (collision accident feature) includes at least the collision pose and the speed difference. The collision condition feature set may also include the acceleration waveform and the structural deformation.

[0105] Optionally, the training input data subset may also include environmental parameters (weather data), etc. The training verification input data subset includes the casualty situation, etc. The embodiments of the present application do not limit the content of the training input data subset and the training verification data subset. The more data types included in the training input data subset, the more accurate the collision condition prediction model obtained by training will be.

[0106] Optionally, the collision condition feature set may also include the acceleration waveform and the structural deformation. The acceleration waveform is the acceleration time history when the vehicle collides. This acceleration waveform is used to optimize the airbag control strategy. The structural deformation is the deformed state of the vehicle body after the collision, which is used to evaluate the remaining safety margin of the vehicle. The embodiments of the present application do not limit the collision condition features included in the collision feature set.

[0107] Step 306: Obtain the passenger feature set of each passenger in the vehicle, and determine the deployment time of each airbag based on the collision condition feature set and each passenger feature set.

[0108] Among them, the passenger feature set includes the attribute feature set and the posture feature set of the passenger.

[0109] In implementation, the control system obtains the passenger feature set of each passenger in the vehicle through each sensor and each image acquisition device. Then, the control system determines the initial collision acceleration based on the collision condition feature set. This initial collision acceleration is the initial collision acceleration of each passenger. The control system determines the first collision time between the vehicle and the object to be collided with. Then, the control system determines the second time for each passenger to the vehicle components according to the initial collision acceleration, the passenger feature set, and the collision condition feature set. Then, the control system determines the deployment time of the airbag corresponding to each passenger based on the first collision time and each second collision time.

[0110] In an optional embodiment, the control system calculates the airbag deployment speed and pressure control curve based on the collision condition feature set (acceleration waveform steepness, Δv magnitude) and the occupant position in the occupant feature set (whether close to the airbag module), and determines the ignition time of each airbag, ensuring that it can protect the passengers in time without causing excessive secondary harm to the passengers.

[0111] In the above object control method for the integration of active and passive safety, the collision condition feature set is determined through the object feature set of the object to be collided, and the ignition time of the airbag corresponding to each passenger is determined according to the collision condition feature set and the passenger feature set. Compared with directly detonating the airbag through acceleration, three influencing factors, namely the object to be collided, the collision state of the vehicle, and the passenger state, are considered, and an accurate ignition time is obtained. Then, the airbag is detonated based on the accurate ignition time, improving the accuracy of the object control method.

[0112] In an exemplary embodiment, as Figure 7 shown, the specific processing process of determining the object to be collided and the object feature set of the object to be collided in step 302 includes steps 702 to 706. Among them:

[0113] Step 702, perform image recognition on each image data in the image data set based on the image recognition model to obtain the object to be collided and the initial object feature set of the object to be collided.

[0114] In implementation, the control system inputs the image data set into the image recognition model, performs image recognition on each image data set through the image recognition model to obtain each initial object to be collided. The control system screens the objects to be collided that meet the preset collision screening conditions among the initial objects to be collided, and determines the initial object feature set of the object to be collided. The initial object feature set includes the category and mass of the object to be collided.

[0115] Specifically, the collision screening conditions are pre-set in the control system. The control system performs image recognition on each image data of the image data set based on the image recognition model to obtain each initial object to be collided and the category of each initial object to be collided. Then, the control system screens the objects to be collided among the initial objects to be collided according to the collision screening conditions. The control system determines the mass of the object to be collided in the database, and constructs the initial object feature set of the object to be collided with the mass and category of the object to be collided.

[0116] Step 704, determine the speed and volume of the object to be collided according to the point cloud data set and the image data set.

[0117] In implementation, the radar data includes a LiDAR (Light Detection and Ranging) three-dimensional point cloud bounding box. The control system obtains the length, width, and height of the object to be collided with from the LiDAR three-dimensional point cloud bounding box, and calculates the volume of the object to be collided with based on the length, height, and width of the object to be collided with. Then, the control system calculates the target longitudinal velocity, target lateral velocity, and acceleration of the object to be collided with based on the radial velocity data, LiDAR three-dimensional data, and visual change coordinates in the radar data.

[0118] In an exemplary embodiment, the radar data includes a LiDAR (Light Detection and Ranging) three-dimensional point cloud bounding box, LiDAR three-dimensional data, and mirror image velocity data. The control system obtains the length, width, and height of the object to be collided with from the LiDAR three-dimensional point cloud bounding box, and calculates the volume of the object to be collided with based on the length, height, and width. The control system determines the visual change coordinates of the object to be collided with according to each image data, and determines the target longitudinal velocity, target lateral velocity, and acceleration of the object to be collided with based on the visual change coordinates, radial velocity data, and LiDAR three-dimensional data.

[0119] Step 706, update the initial object feature set based on the velocity and volume of the object to be collided with to obtain an object feature set.

[0120] In implementation, the control system adds the velocity and volume of the object to be collided with to the initial object feature set to obtain an object feature set. Since the initial object feature set includes the category and mass of the object to be collided with, the updated object feature set includes the category, mass, volume, target longitudinal velocity, target lateral velocity, and acceleration of the object to be collided with.

[0121] In an alternative embodiment, the control system queries the stiffness of the object to be collided with in the database according to the category of the target object, and adds the stiffness of the object to be collided with to the object feature set to obtain an updated object feature set.

[0122] In an exemplary embodiment, Figure 8 is a schematic diagram of the input and output of a multi-modal recognition model in an exemplary embodiment. As Figure 8 shown, the vehicle collects data on the road and the surrounding environment in real time through multiple sensors (visual cameras, lidar, millimeter wave radar) to obtain an image dataset and a point cloud dataset. Determine the object to be collided with through the image dataset and the point cloud dataset, and determine the object feature set of the object to be collided with. The object feature set includes four physical attributes strongly related to collisions: the volume, mass, velocity (vector), and stiffness of the object to be collided with.

[0123] Specifically, the control system performs image recognition on each image data based on an image recognition model, thereby obtaining a 2D (two-dimensional) bounding box of the initial object to be collided detected in the RGB (Red, Green, Blue, a color standard) image data, and through instance segmentation to the corresponding 3D (three-dimensional) information for preliminary estimation, obtaining the object to be collided. Then, the control system distinguishes whether the object to be collided is a soft target (such as pedestrians, animals) or a hard target (such as vehicles, roadblocks) according to the recognized target type. The control system determines the mass of the estimated object to be collided in the database based on the target category of the object to be collided (such as cars, trucks, motorcycles, pedestrians).

[0124] The control system obtains the length, width, and height of the target from the LiDAR three-dimensional point cloud bounding box to estimate the volume of the object to be collided. Then, the control system calculates the target longitudinal velocity, target lateral velocity, and acceleration of the object to be collided according to the radial velocity data in the radar data, the LiDAR three-dimensional data, and the visual change coordinates.

[0125] For the acquisition of raw data (image data sets and point cloud data sets) from vision, LiDAR, millimeter-wave radar, etc., the control system performs time and space synchronization and registration on the raw data in a unified coordinate system, and constructs a rule-based classifier by integrating the registered raw information. Then, the control system estimates the mass and comprehensive stiffness characteristics of the object to be collided. The deep learning inference module of the control system is deployed on the automotive electronic control unit (ECU), GPU (Graphics Processing Unit), and dedicated AI (Artificial Intelligence) chips to achieve object detection, classification, speed and volume estimation, etc.

[0126] The database includes the typical mass ranges of various vehicles (for example, a small car is 1 ton, an SUV is 2.5 tons, a truck can reach more than 3 tons, etc.), as well as stiffness information (classified according to vehicle types and front-end structures). Soft targets (pedestrians, animals) and hard targets (road piles, guardrails, etc.) are classified by stiffness; the stiffness coefficients of pedestrians / animals are much smaller than those of vehicles or roadblocks.

[0127] As the vehicle and the object to be collided approach each other, the control system will continuously update the speed, position, and direction. Once a potential collision risk is detected (such as TTC < 0.5s, TTC = Time to Collision), the latest output of the multi-modal recognition module is called: parameters such as volume, mass, speed (including longitudinal and lateral components), and stiffness will be sent to the vehicle's collision prediction model.

[0128] In this embodiment, by integrating the image datasets and point cloud datasets collected by various sensors such as image acquisition devices and radar devices, the surrounding environment of the vehicle can be perceived more accurately. Based on the image datasets and point cloud datasets, the objects to be collided with and the object feature sets of the objects to be collided with are determined, realizing the real-time evaluation of potential collision risks. Compared with traditional single-sensor systems, the present application significantly reduces sensor errors and incomplete information through the fusion of multi-source information (such as images, distances, speeds, etc.), providing more comprehensive and accurate collision prediction capabilities.

[0129] In an exemplary embodiment, as Figure 9 shown, the specific processing procedure of step 702 includes steps 902 to 906. Among them:

[0130] Step 902: Perform image recognition on each image data in the image dataset based on an image recognition model to obtain each initial object to be collided with and the category of each initial object to be collided with.

[0131] In implementation, the control system inputs each image data in the image dataset into the image recognition model, and performs image recognition on each image data through the image recognition model to obtain each initial object to be collided with and the category of each initial object to be collided with.

[0132] Step 904: Screen the objects to be collided with from each initial object to be collided with according to preset collision screening conditions.

[0133] In implementation, collision screening conditions are preset in the control system. For each initial object to be collided with, the control system determines whether the initial object to be collided with meets the collision screening conditions according to the distance and angle between the initial object to be collided with and the vehicle. If the initial object to be collided with meets the collision screening conditions, the initial object to be collided with is determined as the object to be collided with.

[0134] In an exemplary embodiment, the collision screening condition is an initial object to be collided with whose angle with the vehicle is forward ±30 degrees (plus or minus 30 degrees) and the distance from the vehicle does not exceed 50 meters. The control system screens, according to the collision screening conditions, the initial objects to be collided with whose angle with the vehicle is forward ±30 degrees and the distance from the vehicle does not exceed 50 meters as the objects to be collided with.

[0135] Optionally, the collision screening conditions can be adjusted according to the type and speed of the vehicle, and the present application embodiment does not limit the collision screening conditions.

[0136] Step 906: Query the mass of the object to be collided with in the database based on the category of the object to be collided with, and construct an initial object feature set of the object to be collided with according to the category and mass of the object to be collided with.

[0137] In implementation, the control system determines the mass of the object to be collided with in the data according to the category of the object to be collided with. Then, the control system combines the category and mass of the object to be collided with to obtain an initial feature set of the object to be collided with.

[0138] In an exemplary embodiment, the control system looks up a table to obtain the average mass or mass range according to the category of the object to be collided with (such as a sedan, a truck, a motorcycle, a pedestrian), and then corrects the mass according to the volume estimated by the 3D bounding box.

[0139] In an alternative embodiment, Figure 10 FIG. is a schematic diagram of the operation process of an image recognition model in an exemplary embodiment. When the vehicle starts, the control system continuously obtains an image data set and a point cloud data set based on the image recognition model, and determines the object to be collided with until the vehicle stops running or the power is cut off or the risk of the vehicle is lifted. The control system obtains the image data set and the point cloud data set, and performs time synchronization processing on the image data set and the point cloud data set. Specifically, the control system synchronizes time by aligning the visual frames, LiDAR point cloud frames, and millimeter wave radar detection frames in the image data set and the point cloud data set through a unified time stamp (GPS / IMU / ROS, where GPS is the Global Positioning System, IMU is the Inertial Measurement Unit, and ROS is the Robot Operating System). Spatial registration is completed by calibrating the external parameters of the coordinate systems of the camera, lidar, and millimeter wave radar in the offline calibration stage; in the online stage, coordinate transformation is performed according to the external parameter matrix to ensure that the same initial object to be collided with corresponds to the same position in the data of different sensors.

[0140] Then, the control system performs image recognition on the image data set and the point cloud data set after time synchronization processing based on the image recognition model to obtain each initial object to be collided with and the type of each initial object to be collided with. The image recognition model uses a target detection and instance segmentation network (such as Faster R-CNN, YOLO, Mask R-CNN, where Faster R-CNN is the Faster Region Convolutional Neural Network, YOLO is an image recognition model, and Mask R-CNN is the Mask Region Convolutional Neural Network). The input format of the image recognition model is an input RGB image, and the output is a 2D bounding box / instance segmentation mask + the category probability of the initial object to be collided with.

[0141] When determining the object to be collided with based on an image recognition model, the volume and speed of the object to be collided with can be determined simultaneously through a neural network. Specifically, the input of the 3D object detection network of the LiDAR branch (such as PointNet++, VoxelNet, SECOND, PointNet++ is a deep learning model for processing point cloud data, VoxelNet is a voxel network, and SECOND is a single-shot multi-box detector) is point cloud data, and the output is a 3D bounding box (x, y, z, length, width, height, orientation). The volume of the object to be collided with can be determined through the 3D object detection network of the LiDAR branch. The radar branch inputs in the form of point traces or sparse signals, and a sequence network based on Transformer processes the relative distance and relative speed features to obtain the speed of the object to be collided with.

[0142] After determining the speed, volume, and category of the object to be collided with, it is necessary to match the object to be collided with through a multi-modal fusion layer, combined with the radar speed information, to output a unified object detection and speed estimation result. Then, the control system performs attribute estimation (attribute regression) on the object to be collided with. Based on the fused features, the attribute regression head trains an estimation head for the target volume, mass, and stiffness (classification or regression). The mass and stiffness are "calibrated and regressed" according to an external database. The loss function of the detection part uses the commonly used classification (focal loss) + bounding box regression (Smooth L1) loss in object detection. The loss function for the stiffness / mass classification method uses cross-entropy. When the preliminary mass estimation output by the network has a large gap with the prior database, online or offline calibration and correction will be performed. The same applies to stiffness estimation, and interval mapping can be performed according to the vehicle type or target material to correct the value output by the neural network.

[0143] As the vehicle and the object to be collided with approach each other, the control system continuously updates the target speed, position, and direction of the object to be collided with. Once a potential collision risk is detected (such as TTC < 0.5s, TTC = Time to Collision), the object features of the object to be collided with: parameters such as volume, mass, speed (including longitudinal and lateral components), and stiffness will be sent to the collision condition prediction model of the vehicle.

[0144] In this embodiment, the object to be collided with is determined in each image data through an image recognition model and collision screening conditions, and the object that will cause a collision is obtained. By determining the initial object feature set of the object to be collided with, the attribute features of the object to be collided with are obtained, which is convenient for predicting the situation when the vehicle collides with the object to be collided with subsequently.

[0145] In an exemplary embodiment, such as Figure 11As shown, the specific processing procedure for obtaining the passenger feature sets of each passenger in the vehicle in step 306 includes steps 1102 to 1106. Among them:

[0146] Step 1102, obtain the attribute feature sets of each passenger through each seat sensor in the vehicle.

[0147] In implementation, each seat in the vehicle is provided with a seat sensor. The control system obtains the attribute feature set of each passenger through each seat sensor in the vehicle.

[0148] Specifically, since different passengers will sit on different seats, and the attributes and sitting postures of each passenger are different. Therefore, different airbag control strategies need to be adopted for each passenger. For the airbag control strategy, it is necessary to first collect the attribute feature set of the passenger through the in-cabin sensors. The in-cabin sensors include seat pressure sensors, vision sensors for estimating the occupant's height, and seat belt sensors. The seat pressure sensor and the vision sensor are used to determine whether it is an adult, a child, or the body posture of the occupant. The control system obtains the data transmitted by the seat pressure sensor, the occupant's weight, and the vision sensor, and determines the basic attributes of the passenger through the data transmitted by the seat pressure sensor and the vision sensor. The seat belt sensor is used to transmit the seat belt usage situation and the tension and locking situation of the pretensioner. The control system determines the safety attribute of each passenger based on the seat belt sensor, and constructs the attribute data set of the passenger according to the safety attribute and the basic attribute.

[0149] Step 1104, obtain the posture feature sets of each passenger based on the image acquisition device and / or the infrared sensor.

[0150] In implementation, the control system obtains the initial posture feature set of each passenger through the image acquisition device and / or the infrared sensor. The initial posture feature set includes the head and neck position and body posture of the passenger. The control system performs a priori classification on the basic physical signs (body size) and habitual sitting postures of the passenger according to the initial posture feature set and the historical experiment calibration, and updates the classified sitting posture to the initial posture feature set to obtain the posture feature set.

[0151] Optionally, it can, but is not limited to, be the infrared sensor to identify the initial posture feature set of each passenger, or collect the passenger image data set through the image acquisition device and perform image recognition on the passenger image data set to obtain the initial posture feature set of each passenger.

[0152] Step 1106, construct the passenger feature sets of each passenger according to the attribute feature sets and the posture feature sets.

[0153] In implementation, the control system performs data filtering, data cleaning, and coordinate alignment on the attribute feature set and pose feature set of each passenger to obtain the passenger feature set of the passenger. The passenger feature set includes the passenger position vector P (relative to the vehicle coordinate system), the pose parameters A (head and neck inclination angle, upper body forward inclination angle), the seat belt restraint state S (fastened, unfastened, pre-tensioner state), and the basic attributes C of the passenger (such as classification of children, adults, small-sized females, medium-sized males, etc.).

[0154] In this embodiment, by collecting the passenger feature sets of each passenger, the sitting postures of the passengers and the basic attributes of the passengers are obtained, which is convenient for optimizing the airbag detonation time according to the characteristics of the passengers subsequently.

[0155] In an exemplary embodiment, as Figure 12 shown, the specific processing process of determining the detonation time of each airbag based on the collision condition feature set and each passenger feature set in step 306 includes steps 1202 to 1208. Among them:

[0156] Step 1202, based on the speed, acceleration of the vehicle, and the first distance between the vehicle and the object to be collided, determine the first collision time between the vehicle and the object to be collided.

[0157] In implementation, a first collision time algorithm is preset in the control system. The control system obtains the acceleration and speed of the vehicle, and determines the first distance between the vehicle and the object to be collided from the latest image data. Then, the control system performs data processing on the speed, acceleration, and first distance of the vehicle according to the first collision time formula to obtain the first collision time between the vehicle and the object to be collided.

[0158] Step 1204, according to the preset collision process time, the speed difference in the collision condition feature set, the object feature set of the object to be collided, and the attribute information of the vehicle, determine the initial collision acceleration.

[0159] Among them, the collision condition feature set includes the speed difference. The object feature set of the object to be collided includes the stiffness correction coefficient and mass of the object to be collided. The attribute information of the vehicle includes the mass and stiffness correction coefficient of the vehicle.

[0160] In implementation, a collision process time is preselected and set in the control system. The control system performs data processing on the collision process time, the speed difference, the object feature set of the object to be collided, and the attribute information of the vehicle according to the initial collision acceleration algorithm to obtain the initial collision acceleration.

[0161] Among them, the initial collision acceleration algorithm is shown as the following formula (1):

[0162] (1)

[0163] Among them, in the above formula (1), Δv is the velocity difference, t is the time of the collision process, and a is the initial collision acceleration. The stiffness correction coefficient of the vehicle, is the stiffness correction coefficient of the object to be collided, is the mass of the vehicle, is the mass of the object to be collided.

[0164] Step 1206: For each passenger, based on the passenger's passenger feature set, the initial collision acceleration, and the collision pose in the collision condition feature set, determine the second collision time of the passenger.

[0165] Among them, the second collision time is the time from the passenger's head and neck to the vehicle components.

[0166] In implementation, for each passenger, the control system determines the distance from the passenger to the vehicle components based on the passenger feature set. Then, the control system corrects the initial collision acceleration according to the passenger feature set and the collision condition feature set to obtain the collision acceleration. The control system determines the second collision force according to the collision acceleration and the second distance.

[0167] Step 1208: Based on the first collision time and the second collision time, determine the detonation time of the airbag corresponding to the passenger.

[0168] In implementation, for each passenger, the control system performs a subtraction operation on the first collision time and the second collision time corresponding to the passenger to obtain the detonation time of the airbag corresponding to the passenger.

[0169] In this embodiment, according to the real-time changes of the collision condition feature set, the passenger feature set, and the object feature set of the object to be collided, the detonation timing and force of the airbag for each passenger are dynamically optimized, improving the accuracy of the airbag detonation time, and further improving the accuracy of the object control method. Moreover, this adjustment not only considers the severity of the collision, but also combines factors such as the passenger's body type, sitting posture, seat belt usage, etc. for personalized configuration, thus ensuring that the airbag system can provide more precise and targeted protection.

[0170] In an exemplary embodiment, as Figure 13 shown, the specific processing process of determining the second collision time of each airbag based on the collision condition feature set and each passenger feature set in Step 1206 includes Step 1302 to Step 1306. Among them:

[0171] Step 1302: For each passenger, based on the attitude parameter and position vector in the passenger's passenger feature set, determine the second distance between the passenger's head and neck and the vehicle components.

[0172] Among them, each passenger's passenger feature set includes the attitude parameter and position vector of the passenger.

[0173] In implementation, for each passenger, the control system determines the head and neck coordinates of the passenger according to the attitude parameters and position vectors of the passenger. Then, the control system determines the device coordinates of the devices in the vehicle corresponding to each passenger, and determines the second distance between the head and neck of the passenger and the device according to the head and neck coordinates and the device coordinates.

[0174] In an exemplary embodiment, for each passenger, the control system determines the head and neck coordinates of the passenger according to the attitude parameters and position quality of the passenger , and determines the device coordinates of the devices in the vehicle corresponding to the passenger . The device is the device that the passenger is most likely to hit. The control system performs coordinate calculation on the head and neck coordinates and the device coordinates according to the second distance algorithm to obtain the second distance between the head and neck of the passenger and the device. Among them, the second distance algorithm is shown in the following formula (2):

[0175] (2)

[0176] Among them, in the above formula (2), is the second distance, is the head and neck coordinates, is the device coordinates.

[0177] Optionally, the devices in the vehicle may include, but are not limited to, a steering wheel, a front row seat, etc. Embodiments of the present application relate to the devices of the vehicle.

[0178] Step 1304, modify the initial collision acceleration according to the seat belt restraint state and basic attributes in the passenger feature set and the collision pose in the collision condition feature set to obtain the collision acceleration.

[0179] Among them, the passenger feature set includes the seat belt restraint state and basic attributes of the passenger. The collision condition feature set includes the collision pose of the vehicle.

[0180] In implementation, the control system modifies the initial collision acceleration according to the seat belt restraint state, basic data and collision pose to obtain the collision acceleration. For example, if the seat belt restraint state is the buckled state, the control system reduces the initial collision acceleration to obtain the collision acceleration. If the basic attribute of the passenger indicates that the passenger is a woman, the control system reduces the initial collision acceleration. If the collision pose indicates that the vehicle will roll over, the lateral acceleration of the vehicle is increased.

[0181] Step 1306, perform data processing on the collision acceleration and the second distance according to the collision time algorithm to obtain the second time to collision.

[0182] In implementation, a collision time algorithm is pre-set in the control system. The control system processes the collision acceleration and the second distance to obtain the second collision time. Among them, the second collision time algorithm is shown in the following formula (3):

[0183] (3)

[0184] Among them, in the above formula (3), is the second collision time, is the second distance, is the collision acceleration.

[0185] In this embodiment, the second collision time when the passenger impacts the vehicle interior device is determined through the collision condition feature set and the passenger feature set of each passenger, which is convenient for subsequently determining the initiation time based on the second collision time.

[0186] In an exemplary embodiment, after determining the initiation time, it is also necessary to inflate the airbag according to the initiation time point. Therefore, as Figure 14 shown, after step 1206 is executed, the specific processing procedure of the airbag control method includes steps 1402 to 1406. Among them:

[0187] Step 1402, determine whether the airbags corresponding to each passenger need to be inflated according to the passenger feature set of each passenger and the collision condition feature set.

[0188] In implementation, an initiation condition is pre-set in the control system. The control system judges whether the airbag corresponding to each passenger needs to be inflated according to the passenger feature set of this passenger and the collision condition feature set, and obtains a judgment result. If the judgment result indicates that the airbag of this passenger needs to be inflated, the control system determines this passenger as the target passenger and determines the airbag corresponding to the target passenger as the target airbag.

[0189] Specifically, the passenger feature set includes the seat belt restraint state of the passenger. The collision condition feature set includes the speed difference and the structural deformation. The control system determines the collision intensity of each passenger in the vehicle according to the speed difference, the structural deformation and the seat belt restraint condition of this passenger. The control system judges whether the collision intensity reaches the preset initiation condition. If the collision intensity reaches the initiation condition, the control system determines this passenger as the target passenger and determines the airbag corresponding to the target passenger as the target airbag.

[0190] Optionally, the initiation condition is determined according to multiple collision experiments, and the initiation condition is not limited in the embodiments of the present application.

[0191] Step 1404, in the case where the target airbag of the target passenger needs to be inflated, generate a target ignition signal based on the target initiation mode and the initiation time corresponding to the target passenger.

[0192] In implementation, when there is a need to inflate the target airbag corresponding to the target passenger, the control system determines the target inflation mode of the target airbag, which is also the target inflation mode corresponding to the target passenger, based on the passenger feature set and the collision condition feature set of the target passenger. Then, the control system generates a target ignition signal according to the target inflation mode and the inflation time corresponding to the target passenger.

[0193] Specifically, various inflation modes are preset in the control system, namely single-stage inflation or multi-stage inflation mode. The control system dynamically selects the target inflation mode according to the passenger feature set and the collision condition feature set of each passenger. For a minor collision, the control system determines a low-intensity single-stage inflation mode as the target inflation mode, and for a severe collision, the control system determines a high-intensity multi-stage inflation mode as the target inflation mode. Moreover, the control system can adjust the inflation time (delayed inflation or on-time inflation) of the target airbag in real time based on the collision condition features and the passenger feature set of the target passenger.

[0194] When there is a need to inflate the target airbag corresponding to the target passenger, the control system determines the target inflation mode of the target airbag, which is also the target inflation mode corresponding to the target passenger, based on the passenger feature set and the collision condition feature set of the target passenger. Each collision condition feature in the collision condition feature set will have different effects on the target inflation mode and inflation intensity of the target airbag, as shown in Table 1 below:

[0195] Table 1

[0196]

[0197] Table 1 is a table showing the influence relationship between the collision condition feature set and the inflation mode and inflation intensity of the target airbag. As shown in Table 1 above, the collision condition feature set includes collision pose, speed difference , acceleration waveform, and structural deformation. The collision pose includes collision direction and collision type. Optionally, the control system can also determine the inflation mode and inflation intensity of the target airbag according to the intervention signal set.

[0198] Meanwhile, each passenger feature in the passenger feature set of the target passenger will have different effects on the target inflation mode and inflation intensity of the target airbag, as shown in Table 2 below:

[0199] Table 2

[0200]

[0201] Table 2 is a table showing the influence of passenger features in the passenger feature set on the target inflation mode and inflation intensity of the target airbag. The passenger feature set includes the seat belt restraint condition of the passenger, the basic attributes of the passenger (body type, height, weight), the sitting posture state of the passenger, and the distance between the passenger and the vehicle interior components (steering wheel, instrument panel).

[0202] For example, when the collision intensity reaches the detonation condition, the occupant is in a normal sitting position, the seat belt is fastened, and the peak acceleration is moderate, the deployment of a standard single-stage airbag (single-stage detonation mode) can meet the safety requirements. When the collision is more severe (high Δv, severe structural deformation) or the occupant's posture is abnormal (seat belt not fastened, excessive forward tilt), a two-stage or multi-stage detonation strategy, i.e., a multi-stage detonation mode, is enabled.

[0203] It should be noted that the target detonation mode of the target airbag will be flexibly adjusted according to the collision situation characterized by the collision condition feature set and the passenger state characterized by the passenger feature set. The airbag detonation criterion for determining the target detonation mode in this application needs to comprehensively consider collision-related factors and occupant-related factors. Through multi-sensor data fusion, real-time signal processing, and adaptive algorithms, the optimal detonation timing and detonation intensity are calculated. Its essence is to achieve contact between the protected object and the fully deployed airbag at the appropriate moment according to the coupling effect of the collision dynamic characteristics and the occupant state, so as to minimize the risk of passenger injury in a collision event, while taking into account the response speed and protection effect of the system in different collision scenarios.

[0204] The control system refers to the detonation time corresponding to the target passenger and generates a target ignition signal corresponding to the target airbag according to the target detonation mode.

[0205] Step 1406: Send the target ignition signal to the airbag device to instruct the target airbag corresponding to the target passenger to pop out.

[0206] In implementation, the control system sends the target ignition signal to the airbag device to instruct the target airbag corresponding to the target passenger to pop out according to the target detonation mode to protect the target passenger.

[0207] In an alternative embodiment, an airbag detonation model is set in the control system to determine the airbag detonation strategy and detonation time. Figure 15 It is a schematic diagram of the input and output for determining the airbag detonation model in an exemplary embodiment. As Figure 15 shown, the airbag detonation model receives secondary input 1 (collision condition feature set) and secondary input 2 (passenger feature set). The passenger feature set is derived from the in-vehicle sensing system and the internal state monitoring module, including seat belt signal, occupant position, posture, body size basic physical signs, and responses (such as whether leaning forward / backward, whether lowering the head).

[0208] The airbag deployment model uses a set of collision condition characteristics (such as acceleration waveform, Δv) to quickly estimate the possible kinematic responses of passengers at the moment of collision in a very short time. That is, the airbag deployment model inputs the acceleration waveform into a simplified multi-body dynamics model of "human-seat-seat belt" to predict the degree of inertial forward movement of the occupant and the change in the relative distance between the head and neck and the steering wheel or dashboard at the moment of collision. If the occupant is not wearing a seat belt or is in a forward-leaning posture, the head and chest displacement at the moment of collision is greater and the time is shorter. If the occupant is in a backward-leaning posture, there will be a situation where the occupant dives relative to the seat at the moment of collision, resulting in poor seat belt protection effect and excessive spinal curvature, which needs to be specially marked for the airbag algorithm to make decisions.

[0209] After obtaining the set of collision condition characteristics (secondary input 1) and the set of characteristics of each passenger (secondary input 2), the airbag deployment model makes decisions based on a predefined deployment strategy to obtain the deployment time and target deployment mode of the target airbag. Specifically, it includes: 1. Determination of the deployment time window: The airbag deployment needs to make a decision within a short millisecond time. It is judged whether the airbag trigger condition is met within 200 ms before the collision starts. 2. Multi-stage deployment: When the collision intensity reaches a specific threshold, the occupant is in a normal sitting posture, the seat belt is fastened, and the acceleration peak value is medium, the standard single-stage airbag deployment can meet the safety requirements. Conditions for multi-stage deployment: When the collision is more severe (high Δv, severe structural deformation) or the occupant's posture is abnormal (not wearing a seat belt, excessive forward lean), the secondary or multi-stage deployment strategy is enabled. 3. Deployment speed and deployment form: According to the characteristics of the collision accident (steepness of the acceleration waveform, size of Δv) and the occupant's position (whether close to the airbag module), the airbag deployment speed and pressure control curve are calculated to ensure that it can protect in time and cause no excessive secondary injury to the occupant.

[0210] In actual applications, due to prediction errors, a reduced-order deployment strategy may be adopted, that is, only the dynamic deployment time is considered. Without changing the original airbag deployment principle and deployment method, the contact situation between different collision conditions and passengers is optimized by changing the deployment time.

[0211] The deployment time fully considers the actual posture of the passenger, whether the seat belt is fastened, the vehicle speed and other occupant, collision dynamics and vehicle states to make decisions on different deployment times, so that the airbag can play the best protection role in the collision scenario as much as possible, while avoiding causing secondary injury to the occupant.

[0212] No deployment: After system analysis, if it is determined that the collision intensity is relatively light and not sufficient to cause major injuries, or there is no substantial collision at all (such as a minor scrape), the airbag deployment will not be triggered.

[0213] Early activation: If the system detects that the occupant is not wearing a seatbelt and the posture has leaned forward, the threshold time for airbag activation is advanced, and the decision time delay is shortened. Early activation enables the airbag to complete preliminary deployment before the occupant fully leans forward into the dangerous area, thus providing protection in a shorter time.

[0214] On-time activation: Under the traditional airbag control logic, once the thresholds of acceleration, collision intensity, relative speed, etc. are met, the system quickly triggers airbag ignition at the standard design time point, avoiding any unnecessary delay.

[0215] Delayed activation: When the predicted collision intensity is small, or the occupant's posture is normal and the seatbelt restraint is good, and the front obstacle is judged not to be serious, etc., the ignition timing of the airbag can be delayed, that is, a controllable extremely short delay window is retained after reaching the traditional collision threshold.

[0216] In this embodiment, by screening the target airbags that need to be activated and determining the target activation mode corresponding to the target airbags, different airbag protections are provided for each passenger, avoiding the "one-size-fits-all" fixed protection level of the traditional airbag system and reducing the risk of injury caused by excessive or insufficient airbag deployment. When the collision intensity is relatively light, the risk of airbag-related injury to the occupant is reduced by delaying activation or reducing the airbag deployment force; in the case of a severe collision, sufficient protection is provided by quickly and highly deploying the airbag.

[0217] In an exemplary embodiment, Figure 16 is a schematic diagram of the architecture of the control system in an exemplary embodiment. As Figure 16 shown, the control system includes a multi-modal target recognition model, a collision condition prediction model, and an airbag ignition model. In the risk perception stage, the multi-modal target recognition model is used to determine the target object to be collided and the object feature set of the target object to be collided. In the pre-collision stage, the collision condition prediction model is used to determine the collision condition feature set. The airbag ignition model is used to determine the activation time and activation mode of the airbag. In the active airbag ignition stage, the airbag is ignited according to the activation time and activation mode. In the collision stage, the deployed airbag protects the passenger from injury. The airbag activation is dynamically adjusted through three models: multi-modal target recognition, collision condition prediction, and airbag ignition.

[0218] Through the use of an efficient collision prediction algorithm and a fast data processing mechanism, this application can predict a collision in real time before it occurs. The control system can complete the entire process from collision prediction to airbag control within an extremely short time, improving the vehicle's reaction speed at the moment of collision, thereby reducing the airbag deployment speed. This plays an important role in reducing the degree of injury to occupants during a collision and enhancing safety. It improves the vehicle's response time to emergencies, ensuring that the airbag system can be quickly activated at critical moments to provide timely protection for the occupants. In high-risk collision scenarios, through accurate judgment and rapid response, the probability of injury caused by the collision is effectively reduced.

[0219] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0220] Based on the same inventive concept, the embodiments of this application also provide a main-passive safety fusion object control device for implementing the main-passive safety fusion object control method involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the main-passive safety fusion object control device provided below can refer to the limitations on the main-passive safety fusion object control method in the above text, and will not be elaborated here.

[0221] In an exemplary embodiment, as Figure 17 shown, a main-passive safety fusion object control device 1700 is provided, including: an acquisition module 1701, a prediction module 1702, and a determination module 1703, where:

[0222] The acquisition module 1701 is configured to acquire an image data set and a point cloud data set outside the vehicle, and determine a to-be-collided object and an object feature set of the to-be-collided object based on the image data set and the point cloud data set.

[0223] The prediction module 1702 is configured to perform prediction processing on the object feature set and the vehicle's control signal set according to a collision condition prediction model to obtain a collision condition feature set.

[0224] A determination module 1703, configured to obtain a set of passenger characteristics of each passenger in the vehicle, and determine the detonation time of each airbag based on the collision condition characteristic set and the set of passenger characteristics of each passenger.

[0225] In an exemplary embodiment, the obtaining module 1701 includes a first obtaining sub-module and a first determining sub-module. Among them, the first determining sub-module includes:

[0226] A first recognition sub-module, configured to perform image recognition on each image data in the image data set based on an image recognition model to obtain a to-be-collided object and an initial object feature set of the to-be-collided object;

[0227] A second determining sub-module, configured to determine the speed and volume of the to-be-collided object according to the point cloud data set and the image data set;

[0228] A first updating sub-module, configured to update the initial object feature set based on the speed and volume of the to-be-collided object to obtain an object feature set.

[0229] In an exemplary embodiment, the first recognition sub-module includes:

[0230] A second recognition sub-module, configured to perform image recognition on each image data in the image data set based on an image recognition model to obtain each initial to-be-collided object and the category of each initial to-be-collided object;

[0231] A first screening sub-module, configured to screen the to-be-collided object from each initial to-be-collided object according to a preset collision screening condition.

[0232] A first query sub-module, configured to query the mass of the to-be-collided object in the database based on the category of the to-be-collided object, and construct an initial object feature set of the to-be-collided object according to the category and mass of the to-be-collided object.

[0233] In an exemplary embodiment, the determination module 1703 includes a second obtaining sub-module and a third determining sub-module. Among them, the second obtaining sub-module includes:

[0234] A fourth determining sub-module, configured to determine a first collision time between the vehicle and the to-be-collided object based on the speed, acceleration of the vehicle, and a first distance between the vehicle and the to-be-collided object;

[0235] A fifth determining sub-module, configured to determine an initial collision acceleration according to a preset collision process time, a speed difference in the collision condition characteristic set, an object feature set of the to-be-collided object, and attribute information of the vehicle;

[0236] A sixth determining sub-module, configured to, for each passenger, determine a second collision time of the passenger based on the passenger's set of passenger characteristics, the initial collision acceleration, and the collision pose in the collision condition characteristic set; the second collision time is the time for the passenger's head and neck to reach the vehicle interior device.

[0237] A seventh determination sub-module, configured to determine the initiation time of the airbag corresponding to the passenger based on the first collision time and the second collision time.

[0238] In an exemplary embodiment, the determination module 1703 includes a second acquisition sub-module and a third determination sub-module. Among them, the third determination sub-module includes:

[0239] An eighth determination sub-module, configured to, for each passenger, determine a second distance between the head and neck of the passenger and the device in the vehicle based on the attitude parameter and the position vector in the passenger feature set of the passenger.

[0240] A correction sub-module, configured to correct the initial collision acceleration according to the seat belt restraint state and the basic attributes in the passenger feature set and the collision pose in the collision condition feature set to obtain the collision acceleration.

[0241] A first processing sub-module, configured to perform data processing on the collision acceleration and the second distance according to the collision time algorithm to obtain a second time to collision.

[0242] In an exemplary embodiment, the active and passive safety integrated object control device 1700 further includes:

[0243] A ninth determination sub-module, configured to determine whether the airbags corresponding to the passengers need to be initiated according to the passenger feature sets and the collision condition feature sets of the passengers.

[0244] A generation sub-module, configured to generate a target ignition signal based on the target initiation mode and the initiation time corresponding to the target passenger when there is a target airbag of the target passenger that needs to be initiated.

[0245] A sending sub-module, configured to send the target ignition signal to the airbag device to instruct the target airbag corresponding to the target passenger to pop up.

[0246] Each module in the above-mentioned active and passive safety integrated control system device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor in the computer device in hardware form or independent of it, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0247] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 18As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for controlling an object that integrates active and passive security. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0248] Those skilled in the art can understand that Figure 18 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0249] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0250] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0251] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0252] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0253] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope recorded in the present application.

[0254] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An object control method integrating active and passive safety, characterized in that, The method includes: Obtaining an image data set and a point cloud data set outside the vehicle, and determining a to-be-collided object and an object feature set of the to-be-collided object based on the image data set and the point cloud data set; Performing prediction processing on the object feature set and the vehicle's control signal set according to a collision condition prediction model to obtain a collision condition feature set; Obtaining a passenger feature set of each passenger in the vehicle, and determining the detonation time of each airbag based on the collision condition feature set and each passenger feature set.

2. The method according to claim 1, characterized in that, The determining the to-be-collided object and the object feature set of the to-be-collided object based on the image data set and the point cloud data set includes: Performing image recognition on each image data in the image data set based on an image recognition model to obtain a to-be-collided object and an initial object feature set of the to-be-collided object; Determining the speed and volume of the to-be-collided object according to the point cloud data set and the image data set; Updating the initial object feature set based on the speed and volume of the to-be-collided object to obtain an object feature set.

3. The method according to claim 2, wherein The performing image recognition on each image data in the image data set based on the image recognition model to obtain a to-be-collided object and an initial object feature set of the to-be-collided object includes: Performing image recognition on each image data in the image data set based on the image recognition model to obtain each initial to-be-collided object and the category of each initial to-be-collided object; Screening the to-be-collided object from each of the initial to-be-collided objects according to a preset collision screening condition; Querying the mass of the to-be-collided object in a database based on the category of the to-be-collided object, and constructing an initial object feature set of the to-be-collided object according to the category and mass of the to-be-collided object.

4. The method according to claim 1, wherein The obtaining the passenger feature set of each passenger in the vehicle includes: Obtaining an attribute feature set of each passenger through each seat sensor in the vehicle; Obtaining a posture feature set of each passenger based on an image acquisition device and / or an infrared sensor; Constructing a passenger feature set of each passenger according to each attribute feature set and each posture feature set.

5. The method according to claim 1, characterized in that, The determining the detonation time of each airbag based on the collision condition feature set and each passenger feature set includes: Determining a first collision time between the vehicle and the to-be-collided object based on the speed and acceleration of the vehicle and a first distance between the vehicle and the to-be-collided object; Determining an initial collision acceleration according to a preset collision process time, a speed difference in the collision condition feature set, the object feature set of the to-be-collided object, and the attribute information of the vehicle; For each passenger, determining a second collision time of the passenger based on the passenger feature set of the passenger, the initial collision acceleration, and a collision pose in the collision condition feature set; the second collision time is the time for the head and neck of the passenger to reach a device in the vehicle; Determining the detonation time of the airbag corresponding to the passenger based on the first collision time and the second collision time.

6. The method according to claim 5, characterized in that, The for each passenger, determining the second collision time of the passenger based on the passenger feature set of the passenger, the initial collision acceleration, and a collision pose in the collision condition feature set includes: For each of the passengers, determine a second distance between the head and neck of the passenger and a device in the vehicle based on the posture parameters and position vectors in the passenger feature set of the passenger; Modify the initial collision acceleration according to the seat belt restraint state and basic attributes in the passenger feature set and the collision pose in the collision condition feature set to obtain a collision acceleration; According to the collision time algorithm, perform data processing on the collision acceleration and the second distance to obtain a second time to collision.

7. The method according to claim 1, characterized in that After determining the inflation times of the airbags based on the collision condition feature set and each passenger feature set, the method further includes: Determine whether the airbags corresponding to the passengers need to be inflated according to the passenger feature sets of the passengers and the collision condition feature set; In the case where the target airbag corresponding to the target passenger needs to be inflated, generate a target ignition signal based on the target inflation mode and inflation time corresponding to the target passenger; Send the target ignition signal to the airbag device to instruct the target airbag corresponding to the target passenger to pop out.

8. An object control device integrating active and passive safety, characterized in that, The device includes: An acquisition module, configured to acquire an image data set and a point cloud data set outside the vehicle, and determine a collision object to be collided and an object feature set of the collision object to be collided based on the image data set and the point cloud data set; A prediction module, configured to perform prediction processing on the object feature set and the vehicle control signal set according to a collision condition prediction model to obtain a collision condition feature set; A determination module, configured to acquire the passenger feature sets of the passengers in the vehicle, and determine the inflation times of the airbags based on the collision condition feature set and each passenger feature set.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images