Vehicle collision processing method and device, electronic device and storage medium

By comprehensively utilizing vehicle status, vital signs, and audio/video data, combined with deep learning models, accurate identification of vehicle collisions and assessment of occupant injuries have been achieved, solving the false alarm and missed alarm problems of existing systems and improving the accuracy of traffic accident handling and rescue efficiency.

CN120922118APending Publication Date: 2025-11-11TIANJIN FAW TOYOTA MOTOR CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202511053030.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-29
Publication Date
2025-11-11

AI Technical Summary

Technical Problem

Existing vehicle collision recognition systems suffer from false alarms and missed alarms in special scenarios, making it difficult to meet the high-precision requirements of modern traffic environments and failing to accurately identify collisions and assess the degree of injury to occupants.

Method used

By acquiring vehicle status data, vital sign data, and audio/video data, and combining short-time Fourier transform, optical flow method, and attitude classification model, a damage assessment model with CNN branch, LSTM branch, and fusion layer is adopted to achieve multi-dimensional judgment of collision and determine the degree of injury.

Benefits of technology

It improves the accuracy of collision recognition, reduces false alarms, and enables timely assessment of occupant injuries, thus aiding in rescue and accident analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120922118A_ABST
    Figure CN120922118A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle collision processing method and device, an electronic device and a storage medium, and relates to the technical field of vehicles. The method comprises the following steps: acquiring vehicle state data of a vehicle; under the condition that the vehicle state data meets the first-level collision condition, vital sign data and audio and video data of a user in the vehicle are collected; the first-level collision condition is used for indicating whether a collision possibility exists or not; judging whether a secondary collision condition is met or not based on the vehicle state data and the audio and video data; the secondary collision condition is used for indicating whether collision occurs; and determining the injury degree of the user based on the vehicle state data, the vital sign data and the audio and video data under the condition that the secondary collision condition is met. The method can be used for identifying whether the vehicle has the collision accident or not, and is used for solving the problem of inaccurate identification of the vehicle collision accident.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a vehicle collision handling method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous development of society and the economy, the number and density of vehicles on the road are increasing, leading to a rise in the frequency of traffic accidents. Among these, vehicle collisions pose a serious threat to the lives of occupants. In vehicle collisions, timely and accurate identification of the collision and effective assessment of the extent of occupant injuries are crucial for saving lives and reducing the consequences of injury.

[0003] Currently, although vehicle safety systems have a certain degree of collision detection and protection functions, there is still a high probability of false alarms and missed alarms in some special scenarios, making it difficult to meet the high-precision requirements for vehicle collision recognition in modern traffic environments. Summary of the Invention

[0004] The purpose of this application is to provide a vehicle collision processing method, apparatus, electronic device, and storage medium, aiming to solve the problem of how to improve the accuracy of vehicle collision recognition.

[0005] In a first aspect, a vehicle collision handling method is provided, the method comprising: acquiring vehicle status data; if the vehicle status data meets the first-level collision conditions, collecting vital sign data and audio / video data of the user in the vehicle; the first-level collision conditions are used to indicate whether a collision is possible; based on the vehicle status data and audio / video data, determining whether the second-level collision conditions are met; the second-level collision conditions are used to indicate whether a collision has occurred; if the second-level collision conditions are met, determining the degree of injury to the user based on the vehicle status data, vital sign data, and audio / video data.

[0006] The vehicle collision handling method provided in this application acquires vehicle status data, and then, if the vehicle status data meets the Level 1 collision conditions, collects the vital signs data and audio / video data of the user inside the vehicle. The Level 1 collision conditions indicate the likelihood of a collision. Then, based on the vehicle status data and audio / video data, it determines whether the Level 2 collision conditions are met, indicating whether a collision has occurred. Finally, if the Level 2 collision conditions are met, the degree of injury to the user is determined based on the vehicle status data, vital signs data, and audio / video data. This method acquires vehicle status data and collects the user's vital signs data and audio / video data when the Level 1 collision conditions are met. It further combines the vehicle status data and audio / video data to determine whether the Level 2 collision conditions are met to confirm a collision. Finally, it integrates multiple data points to determine the degree of injury to the user, thereby achieving multi-dimensional collision assessment, reducing false alarms, improving accuracy, and assisting in rescue and accident analysis based on the assessment of user injuries.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, based on vehicle state data and audio / video data, it is determined whether the conditions for a Level 2 collision are met, including: extracting vehicle acceleration from the vehicle state data; extracting impact feature frequency band energy from the audio data using short-time Fourier transform; obtaining user head displacement from the video data using optical flow and attitude classification models; and determining that the conditions for a Level 2 collision are met when the vehicle acceleration is greater than a speed threshold, the impact feature frequency band energy is greater than the environmental noise energy, and the user head displacement indicates that the displacement distance of the user's head relative to the steering wheel / seat is greater than a distance threshold.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, collecting vital sign data of a user in a vehicle includes: collecting the user's heart rate via a heart rate monitoring module installed in the vehicle's steering wheel; collecting the user's respiratory rate and body movement amplitude via a piezoresistive membrane installed on the vehicle's seat; and monitoring changes in the user's skin resistance via a conductivity sensor installed in the vehicle's steering wheel.

[0009] In conjunction with the first aspect mentioned above, in one possible implementation, the degree of injury to the user is determined based on vehicle status data, vital sign data, and audio / video data, including: inputting vehicle status data, vital sign data, and audio / video data into an injury assessment model and outputting an injury level; the injury level is used to reflect the degree of injury to the user; the injury level includes at least: minor injury level, serious injury level, and critical injury level.

[0010] In conjunction with the first aspect mentioned above, in one possible implementation, the injury assessment model includes a CNN branch, an LSTM branch, and a fusion layer; the CNN branch is used to extract a first feature from the audio and video data; the LSTM branch is used to extract a second feature from the vehicle state data and vital signs data; and the fusion layer is used to weight and fuse the first and second features through an attention mechanism to output the injury level.

[0011] In conjunction with the first aspect mentioned above, in one possible implementation, the method further includes: reporting the collision to the cloud and providing repair suggestions to the user when the injury level is minor; and displaying an accident report summary on the in-vehicle display screen.

[0012] In conjunction with the first aspect mentioned above, in one possible implementation, the method further includes: triggering automatic rescue function and vehicle emergency function in the case of serious or critical injury; wherein, the automatic rescue function includes: calling for rescue and reporting the location and injury level, and sending accident-related information to the traffic management system; the vehicle application function includes: unlocking the vehicle door, turning on the hazard warning lights and cabin lighting, and reminding the user to stay awake through voice interaction.

[0013] Secondly, a vehicle collision processing device is also provided, comprising: an acquisition module and a processing module, enabling the vehicle collision processing device to implement the method described in the first aspect. The acquisition module is used to acquire vehicle status data; the acquisition module is also used to, when the vehicle status data meets the first-level collision conditions, collect the vital signs data and audio / video data of the user in the vehicle, the first-level collision conditions indicating the possibility of a collision; the processing module is used to, based on the vehicle status data and audio / video data, determine whether the second-level collision conditions are met, the second-level collision conditions indicating whether a collision has occurred; the processing module is also used to, when the second-level collision conditions are met, determine the degree of injury to the user based on the vehicle status data, vital signs data, and audio / video data.

[0014] In conjunction with the second aspect above, in one possible implementation, the processing module is further configured to report the collision situation to the cloud and provide repair suggestions to the user when the damage level is minor; the processing module is also configured to display an accident report summary on the vehicle display screen.

[0015] In conjunction with the second aspect mentioned above, in one possible implementation, the processing module is further configured to trigger automatic rescue and vehicle emergency functions in cases of serious or critical injury. The automatic rescue function includes: calling for help and reporting the location and injury level, and sending accident-related information to the traffic management system; the vehicle emergency functions include: unlocking the doors, activating hazard warning lights and cabin lighting, and reminding the user to remain conscious via voice interaction.

[0016] Thirdly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the method of the first aspect described above.

[0017] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an electronic device, cause the electronic device to implement the method described in the first aspect.

[0018] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the steps of the relevant method described in the first aspect above, so as to implement the method of the first aspect above.

[0019] The beneficial effects of the second to fifth aspects mentioned above can be referred to the corresponding description of the first aspect, and will not be repeated here. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This application provides a flowchart of an existing accident identification system.

[0022] Figure 2 This application provides a schematic diagram of the composition of a vehicle collision handling system.

[0023] Figure 3 A schematic flowchart of a vehicle collision handling method provided in this application;

[0024] Figure 4 A flowchart illustrating a method for determining secondary collision conditions provided in this application;

[0025] Figure 5 A flowchart illustrating a method for taking corresponding measures based on the level of injury provided in this application;

[0026] Figure 6 A flowchart illustrating another method for taking corresponding measures based on the level of injury provided in this application;

[0027] Figure 7 This application provides a schematic diagram of the composition of a vehicle collision handling device.

[0028] Figure 8 This is a schematic diagram of the structure of a vehicle collision handling device provided in this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0031] In the embodiments of this application, the terms "first," "second," "third," "fourth," "fifth," and "sixth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," "third," "fourth," "fifth," and "sixth" may explicitly or implicitly include one or more of that feature.

[0032] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0033] "A and / or B" includes the following three combinations: A only, B only, and a combination of A and B.

[0034] As described in the background section, with the increase in the number of vehicles, the frequency of traffic accidents is rising, and vehicle collisions pose a serious threat to the lives of occupants.

[0035] Existing accident recognition systems typically consist of an in-vehicle infotainment system (IVS) algorithm module, inertial sensors (three-axis acceleration / angular velocity), a vehicle bus data interface (vehicle speed, gear position, seatbelt status, etc.), and a cloud server. The IVS integrates the collision recognition algorithm and acquires vehicle dynamics signals via the CAN bus, while the cloud server is responsible for data storage and interfacing with the OEM. Figure 1 This is a flowchart illustrating an existing accident identification system provided in this application. Figure 1 As shown, existing accident recognition systems collect data such as vehicle acceleration, angular velocity, vehicle speed, gear position, and seat belt status through vehicle sensors. The collected data is based on a rule engine to determine if the acceleration is greater than 5g. If so, the vehicle directly reports the collision data to the cloud server; otherwise, the vehicle performs a data discard operation, deleting the collected driving data.

[0036] In summary, it is clear that existing accident recognition systems rely solely on vehicle dynamics data (such as acceleration) to identify whether an accident has occurred, and cannot distinguish between real collisions and non-collision scenarios such as sudden braking and road bumps, resulting in a high false alarm rate (such as false alarms triggered by going over speed bumps).

[0037] Based on this, this application provides a vehicle collision handling method, which includes: acquiring vehicle status data; when the vehicle status data meets the first-level collision conditions, collecting vital sign data and audio / video data of the user in the vehicle; the first-level collision conditions are used to indicate the possibility of a collision; based on the vehicle status data and audio / video data, determining whether the second-level collision conditions are met; the second-level collision conditions are used to indicate whether a collision has occurred; when the second-level collision conditions are met, determining the degree of injury of the user based on the vehicle status data, vital sign data, and audio / video data. Compared with related technologies, this method acquires vehicle status data and collects the user's vital sign data and audio / video data when the first-level collision conditions are met, further combines the vehicle status data and audio / video data to determine whether the second-level collision conditions are met to confirm that a collision has occurred, and finally integrates multiple data to determine the degree of injury of the user, thereby achieving multi-dimensional collision judgment, reducing collision misjudgment, improving judgment accuracy, and assisting in rescue and accident analysis based on the assessment of user injury.

[0038] Figure 2 This is a schematic diagram illustrating the composition of a vehicle collision handling system provided in this application. Figure 2 As shown, the vehicle collision handling system adopts an edge-cloud collaborative architecture. The edge device performs data collection and simple data analysis, while the cloud device has powerful computing capabilities to perform more complex data calculations and analyses. This architecture can leverage the rapid response of the edge device and the powerful analysis capabilities of the cloud device to improve the efficiency and effectiveness of vehicle collision handling.

[0039] Specifically, for the edge computing layer, it first collects various vehicle data (such as physical signals like acceleration and angular velocity) and occupant multimodal data (such as occupant vital signs and in-vehicle audio data) in real time through vehicle sensors, providing basic data support for subsequent collision warnings. These sensors can sensitively capture subtle changes in the vehicle during driving, ensuring that abnormal situations can be detected in a timely manner at the moment of a collision.

[0040] Next, the edge-based system performs a Level 1 collision warning based on vehicle dynamics. By performing preliminary analysis of the data collected by the sensors, it quickly determines whether a collision has occurred. If a single axial acceleration greater than 3g is detected and lasts for 0.1s, or the combined acceleration of the three axes is greater than 4g (vector synthesis threshold), it is determined to be a suspected collision.

[0041] After a suspected collision is detected at the edge, the data storage module is automatically activated to cache data before and after the collision (for example, the system will synchronously record multimodal data cached 5 seconds before and 10 seconds after the suspected collision trigger) to ensure data integrity and availability. The data includes not only physical signals collected by vehicle sensors, but also occupant vital signs data (such as heart rate, respiratory rate, anxiety or post-traumatic stress levels) and in-vehicle occupant audio data.

[0042] Finally, the edge device performs secondary collision verification based on the multimodal raw data stored in the data storage module. By performing in-depth and detailed calculations on the multidimensional data, it confirms whether a collision has actually occurred, improves the accuracy and reliability of collision recognition, avoids false alarms and false negatives, and ensures that the system can make accurate judgments on real collision events.

[0043] For the cloud, if the edge device confirms a collision event through Level 1 collision warning and Level 2 collision verification, the cloud portion will first receive the multimodal raw data from the edge device. This raw data undergoes multimodal feature fusion processing based on a machine learning model. By processing temporal features (heart rate, acceleration changes over time) and processing video frame sequences, spatial features (such as occupant posture vectors) are extracted. This integrates data from different sources and types to help build a comprehensive and accurate collision scene recognition model.

[0044] The collision scene recognition model in the cloud confirms the collision, and a three-level occupant assessment is performed in the cloud to conduct a detailed assessment of the occupants' condition, including key information such as the risk of injury, the location of injury, and the severity of injury. By comprehensively analyzing factors such as the vehicle's condition at the time of the collision, occupant physiological indicators, and audio-visual characteristics inside the vehicle, the three-level occupant assessment function can provide accurate and crucial information for subsequent rescue, helping rescue personnel to understand the occupants' injuries in advance and thus formulate an appropriate rescue plan.

[0045] Subsequently, the cloud-based system outputs injury classification based on the results of the three-level occupant assessment, categorizing the severity of injuries to occupants and providing corresponding results. This classification output helps the system quickly take corresponding measures based on the injury level (for example, for minor injuries, accident data is uploaded to the OEM's cloud to assist in accident handling; for serious / critical injuries, the vehicle's emergency call system makes an emergency call to 120 and the traffic management system 122, transmitting relevant multimodal data). This helps rescue personnel quickly assess the severity of the accident, rationally allocate rescue resources, prioritize the treatment of seriously injured individuals, improve rescue efficiency, and minimize casualties from the accident.

[0046] Furthermore, for data identified as non-collision by the cloud-based collision scene recognition model, the cloud also has a data cleaning function to clean up this data, ensuring the validity and relevance of the data stored in the cloud. By removing useless or irrelevant information, the efficiency and accuracy of data processing are improved, allowing the model to focus more on the analysis and processing of actual collision events.

[0047] Optionally, since the vehicle collision handling system involves the transmission of user privacy content, user privacy protection can be designed at the edge.

[0048] For example, by blurring faces in the audio and video streams at the edge (using a Gaussian blur radius of 15px), only key nodes of body posture are retained, achieving data anonymization. At the same time, a strict access control mechanism is set up for the data. Emergency centers can only view the complete data with user authorization or by default authorization when the user is unconscious, and the data storage period is ensured not to exceed 72 hours. In addition, to protect user identity privacy, encrypted IMEI is bound to temporary rescue ID, effectively avoiding direct association between vehicle VIN code and user's real identity.

[0049] Scalable vehicle collision handling systems can upgrade existing vehicles in both hardware and software aspects.

[0050] For example, in terms of hardware reuse, cameras can reuse interfaces reserved in the ADAS system, vital sign sensors are connected to the body controller via the LIN bus, and microphone arrays and seat pressure sensors support modular installation to adapt to the interior layout of mainstream models, achieving efficient hardware reuse and flexible adaptation. In terms of software iteration, algorithm update packages are regularly pushed to the cloud, the vehicle-mounted system supports silent background upgrades, and each accident data is anonymized before being used for online training to continuously optimize collision recognition thresholds and injury assessment models, thereby constantly improving the system's intelligence and adaptability.

[0051] The vehicle collision handling method provided in this application will be described below with reference to specific embodiments and accompanying drawings:

[0052] Figure 3 This is a flowchart illustrating a vehicle collision handling method provided in this application. This method can be applied to a vehicle controller, also known as an on-board edge computing unit. Specifically, as... Figure 3 As shown, it includes the following:

[0053] S301. Obtain vehicle status data.

[0054] The vehicle status data includes acceleration information of the vehicle in three directions in three-dimensional space, rollover and sliding information, vehicle speed, gear position, seat belt status, and door opening and closing data, which can be obtained through onboard inertial sensors and vehicle status sensors.

[0055] For example, the vehicle data acquisition unit acquires vehicle status data by collecting and transmitting data from various vehicle sensors, including inertial sensors and vehicle status sensors. The inertial sensors include a three-axis accelerometer and a three-axis gyroscope, while the vehicle status sensors cover types such as vehicle speed, gear position, seat belt status, and door opening / closing. Each sensor operates at a specific sampling rate or working mode and is connected to the vehicle's infotainment system in a suitable manner to provide real-time feedback on the vehicle's operating status.

[0056] For example, the vehicle data acquisition unit is used to acquire vehicle status data, specifically including the following: A three-axis accelerometer measures the vehicle's acceleration information in three directions with a range of ±16g, an accuracy of 0.001g, and a sampling rate of 13Hz. A three-axis gyroscope detects the vehicle's rollover and slippage information with a range of ±2000° / s, an accuracy of 0.1° / s, and a sampling rate of 13Hz. Additionally, the vehicle speed sensor uses 1Hz sampling, the gear position sensor uses an event subscription mode, the seatbelt status switch uses contact sensing, and the door switch sensor uses a reed switch. These sensors, through the aforementioned parameter and mode settings, can provide real-time feedback on the vehicle's operating status. All this vehicle status information can be connected to the vehicle's infotainment system via the CAN bus.

[0057] S302. If the vehicle status data meets the conditions for a Level 1 collision, collect the vital signs data and audio / video data of the users in the vehicle.

[0058] The Level 1 collision condition is used to indicate the possibility of a collision.

[0059] In some embodiments, during the vehicle status data acquisition process, when the vehicle status data meets the Level 1 collision conditions, i.e., when a collision is possible, the vehicle data acquisition unit further acquires the vital signs data and audio / video data of the users in the vehicle. This process aims to provide more comprehensive information for emergency response after a collision, so that rescue measures can be taken in a timely manner.

[0060] For example, when the vehicle's current single-axis acceleration exceeds a certain threshold and remains so for a certain duration, and / or the combined acceleration of the three axes exceeds a certain threshold, the vehicle-mounted edge computing unit determines that a suspected collision has occurred, i.e., the first collision condition. Subsequently, the data acquisition unit collects the vital signs and audio / video data of the user in the vehicle.

[0061] For example, when the vehicle's current single-axis acceleration is greater than 3g and lasts for more than 0.1s and / or the combined acceleration of the three axes is greater than 4g, the edge computing unit on the vehicle's terminal will initially determine the current state as a suspected collision, and then activate the data acquisition unit to simultaneously record the occupant's vital signs data and audio and video data for 5 seconds before the trigger and 10 seconds after the trigger.

[0062] In one possible implementation, cockpit infrared cameras with wide-angle lenses (such as 120° field of view, 1080P resolution, and support for night vision mode) and built-in attitude recognition algorithms (based on a lightweight model) can collect user video data, such as capturing abnormal movements like head shifts and limb twitches in real time.

[0063] In one possible implementation, a four-microphone array (sampling rate 44.1kHz, signal-to-noise ratio >65dB) can be embedded in the roof to collect the user's audio data, such as using beamforming technology to locate the sound source, identify collision noises (metallic impact sound: energy change in the 800-5000Hz frequency band) and occupant calls for help (human voice frequency 50-4000Hz, intensity >70dB).

[0064] In one possible implementation, collecting vital sign data of a user in the vehicle includes: collecting the user's heart rate via a heart rate monitoring module installed in the vehicle's steering wheel; collecting the user's respiratory rate and body movement amplitude via a piezoresistive membrane installed on the vehicle seat; and monitoring changes in the user's skin resistance via a conductivity sensor installed in the vehicle's steering wheel.

[0065] In other words, collecting vital sign data of users in a vehicle can be achieved through various sensors. A capacitive heart rate monitoring module can be installed in the steering wheel to collect the user's heart rate; a seat pressure sensor array can be installed on the vehicle seat to collect the user's respiratory rate and body movement amplitude via a piezoresistive membrane; simultaneously, a conductivity sensor can be installed in the steering wheel to monitor changes in the user's skin resistance. These sensors work together and are connected to the vehicle's edge computing unit via an in-vehicle Ethernet (100Mbps) or CAN-FD bus to ensure real-time and comprehensive acquisition of vital sign data of users in the vehicle, providing data support for further accurate vehicle collision identification.

[0066] Among them, the CAN-FD bus, developed from the traditional Controller Area Network (CAN) bus, is a variable-rate CAN bus used in fields such as automotive and industrial automation that have high requirements for data transmission rate and data volume. It features increased data transmission rate, increased data field length, flexible data frame format, and compatibility with traditional CAN, and is widely used in automotive electronics and industrial automation.

[0067] For example, a capacitive heart rate monitoring module can be installed in the vehicle steering wheel. This module integrates silver / silver chloride electrodes and acquires the driver's heart rate (accuracy ±2 beats / min, sampling rate 5Hz) via contact impedance measurement. The collected heart rate data can be used to assess stress response. An array of seat pressure sensors can be installed in the vehicle seat to monitor occupant respiratory rate (calculated using chest and abdominal fluctuation cycles) and body movement amplitude via a piezoresistive membrane (resolution 0.1 kPa). A conductivity sensor can be installed in the vehicle steering wheel to measure changes in skin resistance using a weak current (resolution 0.1 μS) to detect occupant anxiety or post-traumatic stress levels.

[0068] S303. Based on vehicle status data and audio / video data, determine whether the conditions for a Level 2 collision are met.

[0069] The second-level collision condition is used to indicate whether a collision has occurred.

[0070] In one possible implementation, vehicle state data (such as acceleration and speed) and audio-visual data (such as images and sounds) are collected and processed by the edge computing unit on the vehicle's in-vehicle system using deep analysis and fusion algorithms. Collision probability is determined from abrupt acceleration changes in the vehicle state data and image and sound features in the audio-visual data. This is compared with secondary collision conditions; a high match indicates that the conditions are met, signaling a collision and improving vehicle safety performance.

[0071] In some embodiments, Figure 4 This is a flowchart illustrating a method for determining secondary collision conditions provided in this application. Figure 4 As shown, based on vehicle status data and audio / video data, the determination of whether the conditions for a Level 2 collision are met includes:

[0072] S401. Extract vehicle acceleration from vehicle status data.

[0073] S402. Short-time Fourier transform is used to extract the impact characteristic frequency band energy from the audio data.

[0074] The short-time Fourier transform is a tool for analyzing the time-frequency characteristics of a signal. By dividing the signal into short time segments and performing Fourier transforms on each segment, the spectral information of the signal at different time points can be obtained.

[0075] S403. Use optical flow and attitude classification models to obtain the user's head displacement from video data.

[0076] Optical flow is a computer vision technique used to calculate the motion of pixels in an image sequence. It estimates the motion vector of a pixel by analyzing changes in pixel intensity across consecutive image frames, thus forming an optical flow field. Pose classification models are used to identify and classify the poses of objects or people. They are trained on large amounts of pose-annotated data and extract features from images and videos using algorithms such as deep learning.

[0077] S404. When the vehicle acceleration is greater than the speed threshold, the energy in the impact characteristic frequency band is greater than the environmental noise energy, and the user's head displacement indicates that the displacement distance of the user's head relative to the steering wheel / seat is greater than the distance threshold, it is determined that the conditions for a Level 2 collision are met.

[0078] Specifically, short-time Fourier transform can identify collision-related audio, optical flow can calculate the occupant's head motion velocity, and combined with a posture classification model, it can determine whether an impact has occurred. By comprehensively analyzing acceleration, audio, and video data, collision type and collision intensity levels can be generated.

[0079] For example, the vehicle-mounted edge computing unit first preprocesses the multimodal data. Bandpass filtering (0.5-20Hz) is applied to the acceleration signal to remove high-frequency noise and extract the effective vehicle acceleration. Spectral features are extracted using short-time Fourier transform to identify collision-related frequency bands (such as the 800-2000Hz peak value of a metal impact). Optical flow is used to calculate the occupant's head motion velocity (threshold > 1.5m / s), and this is combined with a posture classification model (such as OpenPose) to determine if a collision has occurred. Then, based on the data preprocessing results, the vehicle-mounted edge computing unit fuses and determines whether a real collision event has occurred (e.g., vehicle acceleration exceeding the threshold, impact characteristic frequency band energy in the audio signal greater than 30dB of ambient noise, and video detection showing a relative displacement of the occupant's head relative to the steering wheel and / or seat greater than 10cm). Finally, the vehicle-mounted edge computing unit outputs the collision type as one of frontal, side, or rear-end collision, and the collision intensity level as one of mild, moderate, or severe.

[0080] S304. Under the condition of meeting the level 2 collision conditions, the degree of injury to the user shall be determined based on vehicle status data, vital sign data and audio and video data.

[0081] Once a vehicle meets the criteria for a Level 2 collision, i.e., after a real collision has occurred, vehicle status data, vital signs data, and audio-visual data are further integrated to accurately determine the extent of the user's injuries, facilitating subsequent rescue efforts.

[0082] Specifically, vehicle status data includes information such as acceleration, speed, and direction, reflecting the intensity of the collision and the degree of vehicle deformation; vital signs data covers indicators such as heart rate, respiratory rate, and skin conductance mutations, directly reflecting the user's physiological state; audio and video data captures images and sounds of injuries to occupants, providing an intuitive reference for the extent of injury. Comprehensive analysis of this data allows for a complete and accurate assessment of the user's injury severity, providing crucial information for subsequent emergency rescue or medical treatment.

[0083] For example, the vehicle-mounted edge computing unit determines that a collision has actually occurred based on vehicle status data and audio / video data, and then uploads the vehicle status data, vital sign data, and audio / video data related to the collision event collected by the vehicle data acquisition unit to the cloud. The cloud then determines the degree of injury to the user based on the vehicle status data, vital sign data, and audio / video data.

[0084] For example, the vehicle-mounted edge computing unit can determine the degree of injury to a user based on vehicle status data, vital sign data, and audio / video data.

[0085] In some embodiments, determining the degree of injury to a user based on vehicle status data, vital sign data, and audio / video data includes: inputting vehicle status data, vital sign data, and audio / video data into an injury assessment model and outputting an injury level; the injury level is used to reflect the degree of injury to the user; the injury level includes at least: minor injury level, serious injury level, and critical injury level.

[0086] For example, under the condition of a Level 2 collision, when determining the degree of user injury, the cloud first acquires vehicle status data, vital sign data (including heart rate, respiratory rate, skin conductance mutations, etc.), and audio and video data (video captured by the in-vehicle camera and audio recorded by the microphone). This data is input into an injury assessment model built on machine learning algorithms. The model, trained on a large amount of historical data, can identify different injury characteristic patterns. The model outputs an injury level, where a minor injury level indicates that the user has superficial injuries such as abrasions and contusions, and their bodily functions are basically normal; a serious injury level means that the user may have fractures, internal bleeding, etc., and their bodily functions are affected, requiring emergency treatment; a critical injury level indicates that the user is in a life-threatening state, with serious damage to vital organs, requiring immediate intensive care and resuscitation.

[0087] For example, a cloud-based injury assessment model is built using machine learning algorithms. Inputs include vehicle status data such as peak collision acceleration, duration, and collision angle, as well as vital sign data and audio / video data. The model then identifies abnormal indicators in the vital sign data, such as abnormal heart rate variability (ΔHR > 50 bpm), abnormal respiratory rate (greater than 25 bpm or less than 8 bpm), and skin conductance mutations greater than 2 μS. Based on audio / video features, it determines the duration of distress calls (e.g., distress calls lasting longer than 3 seconds) and identifies bloodstains on the cockpit screen (the average accuracy of the identification results across all test target categories reaches 92%). Finally, a comprehensive analysis categorizes the injury level into three grades: Grade 0, Grade 1, and Grade 2, representing minor injury, serious injury, and critical injury, respectively.

[0088] In some embodiments, the injury assessment model includes a CNN branch, an LSTM branch, and a fusion layer; the CNN branch is used to extract a first feature from audio and video data; the LSTM branch is used to extract a second feature from vehicle state data and vital sign data; and the fusion layer is used to weight and fuse the first and second features through an attention mechanism to output the injury level.

[0089] In one possible implementation, the injury assessment model comprises a CNN branch, an LSTM branch, and a fusion layer. The CNN branch is primarily used to extract spatial features (the first feature) from audio and video data; the LSTM branch is used to extract temporal features (the second feature) from vehicle state data (such as acceleration and angular velocity) and vital sign data (such as heart rate and respiratory rate); the fusion layer uses an attention mechanism to weight and fuse the spatial and temporal features, assigning different weights based on the importance and relevance of the features, ultimately outputting a precise injury level, thus providing a strong basis for subsequent rescue decisions.

[0090] Long Short-Term Memory (LSTM) networks are a variant of recurrent neural networks primarily used to address long-term dependency problems that are difficult for traditional recurrent neural networks to handle. By introducing memory units and three gating mechanisms (forget gate, input gate, and output gate), information can be selectively retained or discarded, effectively capturing dependencies in long sequences.

[0091] Convolutional Neural Networks (CNNs) are deep learning models for processing grid-structured data (such as images and audio). They extract local features through convolutional layers, reduce feature dimensionality using pooling layers, and perform classification or regression through fully connected layers. Their advantages include automatic feature extraction and translation invariance, making them excellent for applications such as image classification and object detection.

[0092] LSTM-CNN fusion network: A deep learning model combining convolutional neural networks and long short-term memory networks, primarily used for processing spatiotemporally correlated sequence data. CNN extracts spatial features, while LSTM captures temporal dependencies. Its advantages lie in its ability to fully utilize the spatiotemporal information of the data, its adaptability to various data types, and its good scalability. LSTM-CNN fusion networks can be applied to video analysis, weather forecasting, fault diagnosis, and other fields. In these areas, by combining the spatial feature extraction capabilities of CNNs with the time series modeling capabilities of LSTMs, LSTM-CNN fusion networks can achieve efficient processing and analysis of complex spatiotemporal data.

[0093] Attention mechanisms are computational methods that simulate human visual attention. They weight input features, enabling the model to focus more on features closely related to damage assessment. Specifically, attention mechanisms assign a weight value to each feature based on its relevance and importance; features with higher weight values ​​have a greater impact on the final damage rating output. In this way, the model can more accurately focus on key features, thereby improving the accuracy and reliability of damage rating assessments and ultimately outputting a precise damage rating.

[0094] For example, the CNN branch processes video frame sequences using a 3D convolutional network to extract spatial features of vehicle occupants, such as head collision trajectories. The LSTM branch processes temporal data such as heart rate and acceleration using a bidirectional LSTM network to capture the stress response trends of occupants. The fusion layer uses an attention mechanism to weight and fuse the spatial and temporal features of vehicle occupants, assigning different weights to spatial and temporal features based on their importance and relevance, and outputting a precise injury level.

[0095] Figure 5 This application provides a flowchart illustrating a method for taking corresponding measures based on the degree of injury. Figure 5 As shown, the method includes the following steps:

[0096] S501. In cases where the damage level is minor, the collision situation is reported to the cloud and repair suggestions are provided to the user; an accident report summary is displayed on the vehicle's display screen.

[0097] For example, when the injury assessment model determines the injury level to be minor, the edge vehicle can report detailed collision information to the cloud and provide repair suggestions to the user based on the collision details and vehicle damage data. The vehicle will display an accident report summary on its in-vehicle display screen so that the user can quickly understand the accident details and take subsequent actions.

[0098] For example, edge vehicles report detailed collision information to the cloud, including but not limited to key information such as the time, location, force of the collision, and damaged parts of the vehicle. Simultaneously, based on the collision details and vehicle damage data, the cloud can analyze and provide specific repair recommendations, such as suggesting inspection of exterior components, internal structures, or specific systems for damage, and recommending appropriate repair procedures or repair shops. Subsequently, the edge vehicle displays a summary of the accident report on its in-vehicle display screen, covering basic accident information, damage assessment results, and repair recommendations. Users can configure settings on the vehicle to cancel subsequent processes such as uploading data to the manufacturer.

[0099] Figure 6 This application provides a flowchart illustrating another method for taking corresponding measures based on the degree of injury. (For example...) Figure 6 As shown, the method includes the following steps:

[0100] S601. In the event of serious or critical injury, the automatic rescue function and vehicle emergency function are triggered.

[0101] The automatic rescue function includes: calling for help and reporting the location and level of injury, and sending accident-related information to the traffic management system; the in-vehicle application function includes: unlocking the car doors, turning on the hazard warning lights and cabin lighting, and reminding the user to stay awake through voice interaction.

[0102] For example, when the injury level is serious or critical, the automatic rescue function and the vehicle emergency function are triggered. The automatic rescue function will automatically call the local emergency rescue number (such as 120) and accurately report the specific location of the accident and the injury level. At the same time, it will send accident-related information such as the time, location, and vehicle information to the traffic management system so that rescue personnel can arrive at the scene quickly. The vehicle emergency function includes automatically unlocking the doors to facilitate entry for rescue personnel; activating hazard warning lights to alert other vehicles to give way; activating cabin lighting to ensure sufficient light inside the vehicle for rescue; and continuously reminding the user to stay alert through the voice interaction system.

[0103] For example, when the injury level is serious or critical, the vehicle at the edge of the emergency will first call the local emergency rescue number 120 through the vehicle's emergency call system. It will transmit the accident location (based on BeiDou high-precision positioning, accuracy <5m), collision type, and injury level to the rescue unit, and encrypt and transmit multimodal data slices to the medical cloud platform, allowing emergency personnel to formulate a rescue plan in advance. Simultaneously, it will send an accident warning to the traffic management system 122, along with data such as the accident time, location, vehicle information, and road environment video clips (10 seconds before and after the collision, with faces anonymized). In addition, the vehicle's emergency measures can also respond automatically, automatically unlocking the doors, turning on hazard warning lights and cabin lighting, and guiding occupants to remain conscious through voice interaction (e.g., "Please keep breathing, rescue is on its way").

[0104] The vehicle collision handling method provided in this application includes: acquiring vehicle status data; when the vehicle status data meets the first-level collision conditions, collecting the vital signs data and audio / video data of the user in the vehicle; the first-level collision conditions are used to indicate the possibility of a collision; based on the vehicle status data and audio / video data, determining whether the second-level collision conditions are met; the second-level collision conditions are used to indicate whether a collision has occurred; and when the second-level collision conditions are met, determining the degree of injury to the user based on the vehicle status data, vital signs data, and audio / video data. This method acquires vehicle status data and collects the user's vital signs data and audio / video data when the first-level collision conditions are met, further combines the vehicle status data and audio / video data to determine whether the second-level collision conditions are met to confirm that a collision has occurred, and finally integrates multiple data to determine the degree of injury to the user, thereby achieving multi-dimensional collision judgment, reducing collision misjudgment, improving judgment accuracy, and assisting in rescue and accident analysis based on the assessment of user injury.

[0105] In an exemplary embodiment, Figure 7 This is a schematic diagram illustrating the composition of a vehicle collision handling device provided in this application. Figure 7 As shown, the vehicle collision handling device includes an acquisition module 701 and a processing module 702. The acquisition module 701 is used to acquire vehicle status data; it is also used to collect vital sign data and audio / video data of the user in the vehicle when the vehicle status data meets the Level 1 collision conditions, the Level 1 collision conditions indicating the possibility of a collision; the processing module 702 is used to determine whether Level 2 collision conditions are met based on the vehicle status data and audio / video data, the Level 2 collision conditions indicating whether a collision has occurred; and the processing module 702 is also used to determine the degree of injury to the user based on the vehicle status data, vital sign data, and audio / video data when the Level 2 collision conditions are met.

[0106] In some embodiments, the processing module 702 is further configured to report the collision to the cloud and provide repair suggestions to the user when the injury level is minor; the processing module 702 is also configured to display an accident report summary on the vehicle display screen.

[0107] In some embodiments, the processing module 702 is further configured to trigger automatic rescue functions and vehicle emergency functions in the event of serious or critical injury. The automatic rescue functions include: calling for help and reporting the location and injury level, and sending accident-related information to the traffic management system; the vehicle emergency functions include: unlocking vehicle doors, turning on hazard warning lights and cabin lighting, and reminding the user to remain conscious via voice interaction.

[0108] In an exemplary embodiment, this application also provides an electronic device, which may be the vehicle collision processing device in the above method embodiments. Figure 8 This is a structural schematic diagram of a vehicle collision handling device provided in this application. Figure 8 As shown, the vehicle collision processing device may include: a processor 801 and a memory 802; the memory 802 stores instructions executable by the processor 801; when the processor 801 is configured to execute the instructions, it causes an electronic device or network device or manager to implement the method described in the foregoing method embodiments.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0114] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A vehicle collision handling method, characterized in that, The method includes: Obtain vehicle status data; If the vehicle status data meets the Level 1 collision conditions, the vital signs data and audio / video data of the user in the vehicle are collected; the Level 1 collision conditions are used to indicate whether there is a possibility of a collision. Based on the vehicle status data and the audio / video data, it is determined whether the secondary collision conditions are met; the secondary collision conditions are used to indicate whether a collision has occurred. If the conditions for a Level 2 collision are met, the degree of injury to the user is determined based on the vehicle status data, vital sign data, and audio / video data.

2. The method according to claim 1, characterized in that, The step of determining whether the conditions for a Level 2 collision are met based on the vehicle status data and the audio / video data includes: Extract vehicle acceleration from the vehicle status data; Short-time Fourier transform is used to extract impact feature frequency band energy from audio data; Optical flow and a pose classification model are used to obtain the user's head displacement from video data; If the vehicle acceleration is greater than a speed threshold, the energy of the impact characteristic frequency band is greater than the environmental noise energy, and the user head displacement indicates that the displacement distance of the user's head relative to the steering wheel / seat is greater than a distance threshold, then the conditions for a level two collision are determined to be met.

3. The method according to claim 1, characterized in that, The vital signs data of the users in the vehicle collected include: The user's heart rate is collected through a heart rate monitoring module installed in the vehicle's steering wheel; The user's breathing rate and body movement amplitude are collected by a piezoresistive membrane installed on the vehicle seat. Changes in the user's skin resistance are monitored by a conductivity sensor installed in the vehicle's steering wheel.

4. The method according to claim 3, characterized in that, The determination of the user's degree of injury based on the vehicle status data, vital sign data, and audio / video data includes: The vehicle status data, vital signs data, and audio / video data are input into the injury assessment model, and the injury level is output. The injury level is used to reflect the degree of injury to the user. The injury level includes at least: minor injury level, serious injury level, and critical injury level.

5. The method according to claim 4, characterized in that, The damage assessment model includes a CNN branch, an LSTM branch, and a fusion layer; The CNN branch is used to extract a first feature from the audio and video data; The LSTM branch is used to extract a second feature from the vehicle status data and the vital signs data; The fusion layer is used to weightedly fuse the first feature and the second feature through an attention mechanism to output the damage level.

6. The method according to claim 4, characterized in that, The method further includes: If the damage level is minor, the collision situation will be reported to the cloud and repair suggestions will be provided to the user. An accident report summary is displayed on the in-vehicle display screen.

7. The method according to claim 4, characterized in that, The method further includes: In the event of serious or critical injury, the automatic rescue function and the vehicle emergency function will be triggered. The automatic rescue function includes: calling a rescue hotline and reporting the location and injury level, and sending accident-related information to the traffic management system; The in-vehicle application functions include: unlocking the doors, turning on hazard warning lights and cabin lighting, and reminding the user to stay alert through voice interaction.

8. A vehicle collision handling device, characterized in that, The device includes: an acquisition module and a processing module; The acquisition module is used to acquire vehicle status data. The acquisition module is also used to collect the vital signs data and audio / video data of the user in the vehicle when the vehicle status data meets the Level 1 collision conditions; the Level 1 collision conditions are used to indicate whether there is a possibility of a collision. The processing module is used to determine whether the secondary collision conditions are met based on the vehicle status data and the audio and video data; the secondary collision conditions are used to indicate whether a collision has occurred. The processing module is also used to determine the degree of injury to the user based on the vehicle status data, vital sign data, and audio / video data, provided that the secondary collision conditions are met.

9. An electronic device, characterized in that, The controller includes: a processor and a memory; The memory stores instructions that the processor can execute; When the processor is configured to execute the instructions, the controller causes the controller to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes: computer software instructions; When the computer software instructions are executed in an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-7.

Citation Information

Cited By

  • Vehicle collision detection method, device, vehicle, server and system

    CN121725666A

  • Vehicle collision detection method, device, vehicle, server and system

    CN121725666B