Emergency call system, method and device based on AI vision and storage medium

Abstract

The invention provides an emergency call system, method and device based on AI vision and a storage medium, and relates to the technical field of active safety of intelligent network connection automobiles. The system comprises a camera module, a DMS module, an OMS module, an AI vision algorithm module, an AI vision application module and an eCall application module which work cooperatively. Aiming at the three defects that a traditional emergency call system depends on collision triggering, data dimensions are deficient and active protection is absent, DMS and OMS are innovatively and deeply integrated with the emergency call system, a multi-modal fusion decision mechanism is realized in combination with a layered AI visual architecture, and through deep cooperation of camera shooting, DMS, OMS, AI visual algorithm, AI visual application and an eCall application module, the multi-modal fusion decision mechanism of the emergency call system is realized. A full-link intelligent safety closed loop of sensing, decision making, control and rescue is constructed, early recognition, active intervention and accurate rescue of sudden diseases of drivers, serious fatigue disability and emergency physiological events of passengers are achieved, and meanwhile the rescue quality of collision accidents is improved.

Description

Technical Field

[0001] This application relates to the field of active safety technology for intelligent connected vehicles, specifically to an emergency call system, method, device, and storage medium based on AI vision. Background Technology

[0002] Current in-vehicle emergency call systems (eCall) are primarily triggered by collision sensors (accelerometers / airbag ECUs). When a vehicle reaches a preset physical impact threshold, it automatically sends a Minimum Set of Data (MSD) containing fixed data (such as vehicle location, VIN code, etc.) to a Public Safety Response Point (PSAP). While, according to a European Commission report, such systems reduce average rescue response time by 40% and accident fatality rates by 11%, ECU stands for Electronic Control Unit.

[0003] However, traditional eCall has significant technical shortcomings: its triggering mechanism is highly dependent on the physical collision threshold, resulting in complete underreporting of non-collision high-risk scenarios such as sudden driver illness (e.g., myocardial infarction, stroke), severe fatigue and disability, abnormal occupant behavior, and emergency physiological events (e.g., infant suffocation); at the same time, it lacks MSD data dimensions, missing key information such as occupant physiological status and occupant distribution, which seriously restricts the accurate allocation of rescue resources; and it lacks proactive protection, being limited to post-accident alarms and lacking the ability to predict and intervene in risks before accidents, thus failing to reduce the accident rate at the source.

[0004] With regulatory upgrades, the EU mandated DMS+eCall as standard equipment in new cars by 2022, the GSRII regulation required ADDW (Advanced Driver Distraction Detection System) by 2024, and Euro NCAP added OMS (On-Board Monitoring System) child safety monitoring in 2024. The national standard GB45672-2025, "On-Board Emergency Call System," will be implemented in 2027, and C-NCAP will incorporate DMS into its active safety scoring. Simultaneously, AI vision technology is rapidly developing, and DMS has evolved from fatigue monitoring to non-contact alcohol detection combined with vital sign analysis. OMS uses 3D posture recognition to achieve Child Leaving Warning (CPD). This trend of technological convergence is driving DMS / OMS towards an Integrated Cockpit Monitoring System (IMS). DMS stands for Driver Monitoring System. OMS stands for Occupant Monitoring System.

[0005] However, key contradictions in existing technologies are becoming increasingly apparent. On the one hand, data is fragmented; the visual modules (DMS / OMS) and the eCall system are deployed independently, forming data silos. For example, if the DMS detects a coma risk greater than 90%, it cannot activate the eCall. On the other hand, protocols are lagging; the OMS can identify infant suffocation, but the traditional MSD protocol does not support occupant posture abnormality scoring labels. Furthermore, control is disconnected, lacking a coordinated mechanism between vehicle active intervention (such as automatic deceleration) and rescue commands, resulting in a delay window from risk identification to rescue initiation. In addition, current compliance costs are high, and the separate systems lead to repeated certifications for automakers (such as the EU's separate certifications for DMS / OMS / eCall). Therefore, how to build an integrated eCall system that supports multimodal decision-making is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this application is to provide an AI vision-based emergency call system, method, electronic device, and storage medium that can achieve early identification, proactive intervention, and precise rescue of sudden driver illness, severe fatigue disability, and emergency physiological events of passengers, while improving the quality of rescue in collision accidents.

[0007] In a first aspect, embodiments of this application provide an emergency call system based on AI vision, comprising:

[0008] The camera module, DMS module, OMS module, AI vision algorithm module, AI vision application module, and eCall application module work together.

[0009] The camera module is used to capture image information of the driver and all passengers inside the vehicle from different perspectives, and send the driver's image information to the DMS module and the passenger's image information to the OMS module;

[0010] The DMS module is used to locally receive the driver's image information and locate the driver's facial region from the driver's image information using a target detection model; based on the driver's facial region, it outputs real-time time-series data of the driver's basic state.

[0011] The OMS module is used to locally receive image information of all occupants and extract the full-body key points of each occupant from the image information using a lightweight Transformer model; based on the full-body key points of the occupants, it outputs real-time time-series data of the occupants' basic state.

[0012] The AI ​​vision algorithm module is used to perform fusion modeling based on the driver's basic state data and the occupant's basic state data provided by the DMS module and the OMS module. It uses a temporal convolutional network to perform complex state detection and analysis of the driver's fatigue state, distraction behavior, and coma risk, and outputs the driver's fatigue level, distraction index, and coma risk probability value. Based on a pre-trained ResNet classification model, it distinguishes occupants into adult / child / infant types. According to the occupant type, it calls the corresponding 3D pose estimation algorithm to perform pose recognition and physiological state abnormality detection and analysis, and outputs the occupant type, pose abnormality score, and physiological state abnormality score for each occupant.

[0013] The AI ​​vision application module is used to achieve cross-camera occupant target association through the ReIDNet model and correct occupant positioning data by combining a spatial coordinate mapping model. Based on a temporal trend model, it calculates temporal thresholds for fatigue level, distraction index, and coma risk probability values ​​output by the AI ​​vision algorithm module to assess whether the driver has dangerous driving behavior. Based on a semantic risk model, it activates predefined semantic risk labels according to the occupant type, posture abnormality score, and physiological state abnormality score output by the AI ​​vision algorithm module to determine whether the occupant's physiological state is abnormal. It outputs a high-risk driving decision instruction based on the driver's dangerous driving behavior and an occupant first aid decision instruction based on the occupant's abnormal physiological state. When both dangerous driving behavior and abnormal physiological state coexist, an emergency situation decision instruction is generated. Based on the final decision instruction, it generates a vehicle intervention deceleration request signal and a light-on request signal, and transmits them to the vehicle control system interface through the in-vehicle bus to trigger active preventive measures. At the same time, it transmits the non-biological feature data of the driver and occupants obtained from the analysis to the eCall application module.

[0014] The eCall application module is used to convert the received non-biometric data of drivers and passengers into extended data of the MSD dataset; it triggers an alarm mechanism according to a preset risk level classification. When the risk level is less than a preset threshold, it actively reminds the vehicle owner through the vehicle control system interface; when the risk level is greater than or equal to the preset threshold, it actively reminds the vehicle owner through the vehicle control system interface, and establishes a voice call with PSAP through the cellular network based on the SIP protocol, and uploads the MSD dataset containing extended data to PSAP to achieve proactive intervention and establish an emergency call channel.

[0015] In some embodiments of this application, the camera module is a multi-path vision system composed of multiple dual-mode wide-angle cameras, used to capture infrared and visible light image information of the driver and all occupants inside the vehicle from different perspectives.

[0016] In some embodiments of this application, the driver's basic state includes the driver's eye opening angle, head yaw angle, and line of sight deviation angle.

[0017] In some embodiments of this application, the occupant basic state includes the occupant's posture angle, facial ROI coordinates, and expression vector.

[0018] In some embodiments of this application, the extended data includes at least one of the following: driver's fatigue level, distraction index, coma risk probability value and dangerous driving behavior, and occupant's seating area, location reliability, occupant type, abnormal posture score, abnormal physiological state score and occupant emergency information.

[0019] Secondly, embodiments of this application provide an emergency call method based on AI vision, including:

[0020] Receive image information of the driver and all occupants inside the vehicle, captured from different perspectives;

[0021] The driver's facial region is located from the driver's image information using a target detection model. Based on the driver's facial region, real-time temporal data of the driver's basic state is obtained.

[0022] The key points of each occupant's whole body are extracted from the occupant's image information using a lightweight Transformer model. Based on the key points of the occupant's whole body, real-time time series data of the occupant's basic state are obtained.

[0023] Based on the fusion modeling of driver and passenger basic state data, a temporal convolutional network is used to perform complex state detection and analysis of driver fatigue, distraction behavior, and coma risk, outputting the driver's fatigue level, distraction index, and coma risk probability value; based on the pre-trained ResNet classification model, passengers are distinguished into adult / child / infant types. According to the passenger type, the corresponding 3D pose estimation algorithm is called to perform pose recognition and physiological state abnormality detection and analysis, outputting the passenger type, pose abnormality score, and physiological state abnormality score for each passenger.

[0024] The ReIDNet model is used to associate occupant targets across cameras, and the occupant positioning data is corrected by combining a spatial coordinate mapping model. Based on a temporal trend model, temporal thresholds are calculated for the driver's fatigue level, distraction index, and probability of unconsciousness risk to assess whether the driver has engaged in dangerous driving behavior. Based on a semantic risk model, predefined semantic risk labels are activated according to the occupant's occupant type, abnormal posture score, and abnormal physiological state score to determine whether the occupant's physiological state is abnormal. A high-risk driving decision instruction is output based on the driver's dangerous driving behavior, and an occupant first aid decision instruction is output based on the occupant's abnormal physiological state. When both dangerous driving behavior and abnormal physiological state coexist, an emergency situation decision instruction is generated. Based on the final decision instruction, a vehicle intervention deceleration request signal and a light activation request signal are generated and transmitted to the vehicle control system interface via the in-vehicle bus to trigger active preventive measures.

[0025] The non-biometric data of drivers and passengers is converted into extended data of the MSD dataset; an alarm mechanism is triggered according to the preset risk level classification. When the risk level is less than the preset threshold, the vehicle owner is actively reminded through the vehicle control system interface; when the risk level is greater than or equal to the preset threshold, the vehicle owner is actively reminded through the vehicle control system interface. A voice call with PSAP is established through the cellular network based on the SIP protocol, and the MSD dataset containing the extended data is uploaded to PSAP to achieve proactive intervention and establish an emergency call channel.

[0026] In some embodiments of this application, image information of the driver and all occupants inside the vehicle, captured from different perspectives, is received, including:

[0027] It receives infrared and visible light image information of the driver and all occupants inside the vehicle, captured from different perspectives.

[0028] In some embodiments of this application, the extended data includes at least one of the following: driver's fatigue level, distraction index, coma risk probability value and dangerous driving behavior, and occupant's seating area, location reliability, occupant type, abnormal posture score, abnormal physiological state score and occupant emergency information.

[0029] Thirdly, this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in the second aspect.

[0030] Fourthly, this application provides a computer-readable storage medium having computer-readable instructions stored thereon, which can be executed by a processor to implement the method described in the second aspect.

[0031] Compared to existing technologies, the AI ​​vision-based emergency call system provided in this application addresses three major shortcomings of traditional emergency call systems: reliance on collision triggering, lack of data dimensions, and absence of proactive protection. It innovatively integrates DMS and OMS with the emergency call system, and combines a layered AI vision architecture to achieve a multimodal fusion decision-making mechanism. Through deep collaboration of camera, DMS, OMS, AI vision algorithms, AI vision applications, and eCall application modules, it constructs a full-link intelligent safety closed loop of "perception, decision-making, control, and rescue," enabling early identification, proactive intervention, and precise rescue of sudden driver illness, severe fatigue disability, and emergency physiological events of passengers, while improving the quality of rescue in collision accidents. Attached Figure Description

[0032] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0033] Figure 1 A schematic diagram of the structure of an AI vision-based emergency call system provided in an embodiment of this application is shown;

[0034] Figure 2 This illustration shows a structured diagram of extended data for a specific MSD dataset provided in an embodiment of this application;

[0035] Figure 3 A flowchart of an AI vision-based emergency call method provided in an embodiment of this application is shown;

[0036] Figure 4 A schematic diagram of an electronic device provided in an embodiment of this application is shown;

[0037] Figure 5 A schematic diagram of a T-BOX employing an AI vision-based emergency call method, as provided in an embodiment of this application, is shown. Detailed Implementation

[0038] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0039] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0040] Furthermore, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to those processes, methods, products, or apparatuses.

[0041] Figure 1 This is a schematic diagram of the structure of an AI vision-based emergency call system provided in an embodiment of this application, as shown below. Figure 1 As shown, the AI ​​vision-based emergency call system 10 provided in this application includes: a collaborative camera module 11, a DMS module 12, an OMS module 13, an AI vision algorithm module 14, an AI vision application module 15, and an eCall application module 16. The camera module is abbreviated as "camera" in the figure.

[0042] The camera module 11 is used to capture image information of the driver and all passengers in the vehicle from different perspectives, and send the driver's image information to the DMS module and the passenger's image information to the OMS module.

[0043] Specifically, the camera module 11 is a multi-path vision system composed of multiple RGB-IR dual-mode wide-angle cameras, covering all occupant areas inside the vehicle, and is used to capture IR (infrared) and RGB (visible light) image information of all occupants inside the vehicle from different perspectives.

[0044] The DMS module 12 is used to locally receive the driver's image information and locate the driver's facial region from the driver's image information using a target detection model; based on the driver's facial region, it outputs real-time time-series data of the driver's basic state.

[0045] Specifically, the driver's basic state includes the driver's eye opening degree, head yaw angle, and line-of-sight deviation angle. The driver monitoring system locates the driver's facial area through a target detection model and outputs real-time time-series data of the driver's eye opening degree (0-100%), head yaw angle (±180°), and line-of-sight deviation angle (X / Y axis vector).

[0046] The OMS module 13 is used to locally receive image information of all occupants and extract the full-body key points of each occupant from the image information of the occupants using a lightweight Transformer model; based on the full-body key points of the occupants, it outputs real-time time-series data of the basic state of the occupants.

[0047] Specifically, the occupant's basic state includes spatially structured data such as the occupant's attitude angles (pitch / yaw / roll), facial ROI coordinates (pixel-level positioning), and expression vectors.

[0048] The AI ​​vision algorithm module 14 is used to perform fusion modeling based on the driver's basic state data and the occupant's basic state data provided by the DMS module and the OMS module. It uses a temporal convolutional network (TCN) to perform complex state detection and analysis on the driver, including fatigue state (0-100% confidence), distraction behavior (dynamic recognition), and coma risk (0-100% confidence), and outputs the driver's fatigue level, distraction index, and coma risk probability value. Based on a pre-trained ResNet classification model, it distinguishes occupants into adult / child / infant types. According to the occupant type, it calls the corresponding 3D pose estimation algorithm to perform pose recognition and physiological state abnormality detection and analysis, and outputs the occupant type, pose abnormality score, and physiological state abnormality score for each occupant.

[0049] As can be seen, the AI ​​vision algorithm module 14 of this application can use a temporal convolutional network to analyze and output the driver's fatigue level, coma risk value and occupant abnormality identifier from the real-time temporal data of the driver's basic state, classify the occupant type (adult / child / infant) based on the neural network model, and call a dedicated detection model according to the occupant type.

[0050] The AI ​​vision application module 15 is used to achieve cross-camera occupant target association through the ReIDNet model and correct occupant positioning data by combining a spatial coordinate mapping model; based on the time-series trend model, it calculates time-series thresholds for fatigue level, distraction index, and coma risk probability values ​​output by the AI ​​vision algorithm module, performs sliding window calculation, and assesses whether the driver has dangerous driving behavior; based on the semantic risk model, it activates predefined semantic risk labels according to the occupant type, posture abnormality score, and physiological state abnormality score output by the AI ​​vision algorithm module to determine whether the occupant's physiological state is abnormal; it outputs a high-risk driving decision instruction based on the driver's dangerous driving behavior and an occupant first aid decision instruction based on the occupant's abnormal physiological state; when both the driver's dangerous driving behavior and the occupant's abnormal physiological state coexist, it generates an emergency situation decision instruction, i.e., when multiple abnormalities coexist, it generates and outputs an emergency situation instruction; based on the final decision instruction, it generates a vehicle intervention deceleration request signal and a light-on request signal, and transmits them to the vehicle control system interface through the in-vehicle bus to trigger active preventive measures; at the same time, it transmits the non-biological feature data of the driver and occupants obtained from the analysis to the eCall application module.

[0051] The eCall application module 16 is used to convert the received non-biological characteristic data of drivers and passengers into extended data of the MSD dataset; to trigger an alarm mechanism according to a preset risk level classification; when the risk level is less than a preset threshold, to actively remind the vehicle owner through the vehicle control system interface; when the risk level is greater than or equal to the preset threshold, to actively remind the vehicle owner through the vehicle control system interface; and to establish a voice call with PSAP through the cellular network based on the SIP protocol, and upload the MSD dataset containing extended data to PSAP, so as to realize active intervention and establish an emergency call channel.

[0052] In other words, when there is a low risk of anomalies, the system proactively alerts the driver using in-vehicle voice prompts and sensors integrated with the vehicle control system interface. When there is a high risk of anomalies, it also establishes a voice call with the PSAP via the cellular network using the SIP protocol and uploads the MSD (Missing Data Set) containing extended data to the PSAP to proactively intervene and establish an emergency call channel. Furthermore, in scenarios where a passive rescue automatically triggers an emergency call, the MSD dataset containing extended data is simultaneously uploaded to the PSAP to help improve the quality of rescue efforts.

[0053] Specifically, the extended data includes at least one of the following: driver's fatigue level, distraction index, probability of unconsciousness, and dangerous driving behavior; and occupant's seating area, location reliability, occupant type, abnormal posture score, abnormal physiological state score, and occupant emergency medical information. Figure 2The diagram shown is a structured illustration of extended data from a specific MSD dataset provided in this application. The AI ​​vision-based emergency call system of this application can report rich details about the driver and passengers in scenarios where emergency calls are automatically triggered by passive rescue, thus improving the quality of rescue efforts with more comprehensive information.

[0054] As can be seen, the AI ​​vision application module 15 of this application can perform target association and timing decision-making to generate vehicle control commands. Simultaneously, it transmits extended MSD (Missing Detection and Distress) data, such as high-risk driving and occupant emergency care, to the eCall application. The eCall application establishes a communication link with the PSAP (Powered Emergency Response Team) via the SIP protocol and transmits the extended MSD dataset through this link. The PSAP assesses the hazard level based on the details of the MSD dataset and autonomously selects to make an emergency call to the vehicle owner, shifting from passive rescue to proactive prevention. Furthermore, in scenarios where emergency calls are automatically triggered by passive rescue, more detailed MSD data can also improve the quality of rescue. This system overcomes the three major limitations of traditional eCall systems: reliance on collision triggering, lack of data dimensions, and absence of proactive protection. It achieves an intelligent safety closed loop in non-collision high-risk scenarios while simultaneously improving the quality of rescue in collision accidents.

[0055] The AI ​​vision-based emergency call system provided in this application is an intelligent closed-loop system that is protocol compatible (vSOMEIP service communication and eCall protocol), data fusion (extended MSD field), and control linkage (vehicle speed reduction + light warning), breaking through the dual constraints of technology silos and regulatory requirements.

[0056] Compared with existing technologies, the AI ​​vision-based emergency call system provided in this application has the following advantages:

[0057] 1. This application breaks through the traditional collision threshold triggering limitation and actively judges multimodal features such as sudden illness of the driver, severe fatigue and disability, and emergency physiological events of passengers through AI vision. It advances the eCall activation time from physical collision to before the accident occurs, covering traditional missed reporting scenarios such as sudden cardiac death and fatigue driving, and actively protects against accidents from the source.

[0058] 2. This application proactively intervenes after assessing the risk. When the system detects an anomaly and outputs decision instructions such as high-risk driving and occupant emergency rescue, it proactively generates standardized vehicle control signals and transmits them to the vehicle manufacturer's authorized interface via the in-vehicle bus to trigger proactive speed reduction and hazard warning flashes. Simultaneously, it initiates the eCall high-risk rescue process, effectively solving the problem of a delay window between risk identification and rescue initiation, and improving the rescue response speed.

[0059] 3. The extended MSD data provided in this application adds detailed reporting of drivers and passengers, providing a more intuitive description of the status information of occupants in the vehicle, effectively improving the information dimension, and facilitating the precise allocation of rescue resources in both collision and non-collision high-risk scenarios.

[0060] 4. This application uses a unified data transmission layer, which can meet the requirements of composite authentication in one go, effectively reducing the authentication cost of global deployment and avoiding the overhead of repeated testing and protocol adaptation of split systems.

[0061] In the above embodiments, an AI vision-based emergency call system is provided. Correspondingly, this application also provides an AI vision-based emergency call method. This AI vision-based emergency call method can be implemented through software, hardware, or a combination of both. Please refer to... Figure 3 As shown, this application provides an emergency call method based on AI vision, comprising:

[0062] S101, Receive image information of the driver and all occupants inside the vehicle captured from different perspectives;

[0063] S102. Locate the driver's facial region from the driver's image information using a target detection model, and obtain real-time temporal data of the driver's basic state based on the driver's facial region.

[0064] S103. Extract the full-body key points of each occupant from the image information of the occupants using a lightweight Transformer model, and obtain real-time time series data of the occupant's basic state based on the full-body key points of the occupants.

[0065] S104. Based on the driver's basic state data and the passenger's basic state data, a fusion model is performed. A temporal convolutional network is used to perform complex state detection and analysis of the driver's fatigue state, distraction behavior, and coma risk, and outputs the driver's fatigue level, distraction index, and coma risk probability value. Based on the pre-trained ResNet classification model, the passenger is distinguished into adult / child / infant types. According to the passenger type, the corresponding 3D pose estimation algorithm is called to perform pose recognition and physiological state abnormality detection and analysis, and outputs the passenger type, pose abnormality score, and physiological state abnormality score for each passenger.

[0066] S105. Cross-camera occupant target association is achieved through the ReIDNet model, and occupant positioning data is corrected by combining a spatial coordinate mapping model; based on the temporal trend model, temporal thresholds are calculated for the driver's fatigue level, distraction index, and coma risk probability value to assess whether the driver has dangerous driving behavior; based on the semantic risk model, predefined semantic risk labels are activated according to the occupant's occupant type, posture abnormality score, and physiological state abnormality score to determine whether the occupant's physiological state is abnormal; a high-risk driving decision instruction is output based on the driver's dangerous driving behavior, and an occupant first aid decision instruction is output based on the occupant's abnormal physiological state; when both the driver's dangerous driving behavior and the occupant's abnormal physiological state exist, an emergency situation decision instruction is generated; based on the final decision instruction, a vehicle intervention deceleration request signal and a light-on request signal are generated and transmitted to the vehicle control system interface through the in-vehicle bus to trigger active preventive measures;

[0067] S106. Convert the non-biological characteristic data of the driver and passengers into extended data of the MSD dataset; trigger the alarm mechanism according to the preset risk level classification; when the risk level is less than the preset threshold, actively remind the car owner through the vehicle control system interface; when the risk level is greater than or equal to the preset threshold, actively remind the car owner through the vehicle control system interface, and establish a voice call with PSAP through the cellular network based on the SIP protocol, and upload the MSD dataset containing extended data to PSAP to realize active intervention and establish an emergency call channel.

[0068] In some embodiments, step S101 includes receiving infrared and visible light image information of the driver and all occupants inside the vehicle captured from different perspectives.

[0069] In some embodiments, the driver's basic state includes the driver's eye opening, head yaw angle, and line of sight deviation angle.

[0070] In some embodiments, the occupant basic state includes the occupant's posture angle, facial ROI coordinates, and expression vector.

[0071] In some embodiments, the extended data includes at least one of the following: driver's fatigue level, distraction index, coma risk probability value and dangerous driving behavior, and occupant's seating area, location reliability, occupant type, abnormal posture score, abnormal physiological state score and occupant emergency information.

[0072] The AI ​​vision-based emergency call method and the AI ​​vision-based emergency call system provided in this application are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate, or implement.

[0073] This application also provides an electronic device corresponding to the method provided in the foregoing embodiments. The electronic device may be a vehicle terminal (T-BOX), mobile phone, laptop computer, tablet computer, desktop computer, etc., to execute the above-mentioned emergency call method based on AI vision.

[0074] Please refer to Figure 4 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 4 As shown, the electronic device 20 includes: a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected via the bus 202. The memory 201 stores a computer program that can run on the processor 200. When the processor 200 runs the computer program, it executes the AI ​​vision-based emergency call method provided in any of the foregoing embodiments of this application.

[0075] The memory 201 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 203 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0076] Bus 202 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs. After receiving an execution instruction, the processor 200 executes the program. The AI ​​vision-based emergency call method disclosed in any of the foregoing embodiments of this application can be applied to the processor 200, or implemented by the processor 200.

[0077] The processor 200 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 200 or by instructions in software form. The processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 201. The processor 200 reads the information in memory 201 and, in conjunction with its hardware, completes the steps of the above method.

[0078] For ease of understanding, the embodiments of this application provide, as follows: Figure 5 The diagram shows a T-BOX that uses the AI ​​vision-based emergency call method provided in this application.

[0079] The electronic device provided in this application embodiment and the AI ​​vision-based emergency call method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0080] This application also provides a computer-readable storage medium corresponding to the AI ​​vision-based emergency call method provided in the foregoing embodiments, wherein a computer program (i.e., a program product) is stored thereon, and the computer program, when run by a processor, executes the AI ​​vision-based emergency call method provided in any of the foregoing embodiments.

[0081] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0082] The computer-readable storage medium provided in the above embodiments of this application and the AI ​​vision-based emergency call method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application.

Claims

1. An emergency call system based on AI vision, characterized in that, include: The camera module, DMS module, OMS module, AI vision algorithm module, AI vision application module, and eCall application module work together. The camera module is used to capture image information of the driver and all passengers inside the vehicle from different perspectives, and send the driver's image information to the DMS module and the passenger's image information to the OMS module; The DMS module is used to locally receive the driver's image information and locate the driver's facial region from the driver's image information using a target detection model; Based on the driver's facial region, output real-time time-series data of the driver's basic state; The OMS module is used to locally receive image information of all occupants and extract the full-body key points of each occupant from the image information using a lightweight Transformer model; based on the full-body key points of the occupants, it outputs real-time time-series data of the occupants' basic state. The AI ​​vision algorithm module is used to perform fusion modeling based on the driver's basic state data and passenger's basic state data provided by the DMS module and the OMS module, and to use a temporal convolutional network to perform complex state detection and analysis of the driver's fatigue state, distraction behavior and coma risk, and output the driver's fatigue level, distraction index and coma risk probability value. Based on the pre-trained ResNet classification model, the occupants are distinguished as adults, children, and infants. According to the occupant type, the corresponding 3D pose estimation algorithm is called to perform pose recognition and physiological state abnormality detection and analysis, and output the occupant type, pose abnormality score, and physiological state abnormality score for each occupant. The AI ​​vision application module is used to achieve cross-camera occupant target association through the ReIDNet model and correct occupant positioning data by combining a spatial coordinate mapping model; based on the time-series trend model, it calculates time-series thresholds for the fatigue level, distraction index and coma risk probability value output by the AI ​​vision algorithm module to assess whether the marked driver has dangerous driving behavior. Based on the semantic risk model, predefined semantic risk labels are activated according to the occupant type, posture abnormality score and physiological state abnormality score output by the AI ​​vision algorithm module to determine whether there is an abnormality in the occupant's physiological state. Based on the driver's dangerous driving behavior, a decision instruction for high-risk driving is output; based on the abnormal physiological state of the passengers, a decision instruction for emergency rescue of the passengers is output. When both the driver exhibits dangerous driving behavior and the passengers are in an abnormal physiological state, an emergency decision instruction is generated. Based on the final decision command, a deceleration request signal and a light activation request signal for vehicle intervention are generated and transmitted to the vehicle control system interface via the in-vehicle bus to trigger active preventive measures. Simultaneously, the non-biological characteristic data of the driver and passengers obtained from the analysis are transmitted to the eCall application module; The eCall application module is used to convert the received non-biological characteristic data of drivers and passengers into extended data of the MSD dataset; The alarm mechanism is triggered according to the preset risk level classification. When the risk level is less than the preset threshold, the vehicle control system interface will actively remind the owner. When the risk level is greater than or equal to the preset threshold, the vehicle control system interface will actively remind the owner. In addition, a voice call will be established with PSAP through the cellular network based on the SIP protocol. The MSD dataset containing extended data will be uploaded to PSAP to realize active intervention and establish an emergency call channel.

2. The emergency call system based on AI vision according to claim 1, characterized in that, The camera module is a multi-channel vision system composed of multiple dual-mode wide-angle cameras, used to capture infrared and visible light image information of the driver and all occupants inside the vehicle from different perspectives.

3. The emergency call system based on AI vision according to claim 1, characterized in that, The driver's basic state includes the driver's eye opening, head yaw angle, and line of sight deviation angle.

4. The AI ​​vision-based emergency call system according to claim 1, characterized in that, The occupant's basic state includes the occupant's posture angle, facial ROI coordinates, and expression vector.

5. The AI ​​vision-based emergency call system according to claim 1, characterized in that, The extended data includes at least one of the following: driver's fatigue level, distraction index, probability value of coma risk and dangerous driving behavior, and passenger's seating area, location reliability, passenger type, abnormal posture score, abnormal physiological state score and passenger emergency information.

6. An emergency call method based on AI vision, characterized in that, include: Receive image information of the driver and all occupants inside the vehicle, captured from different perspectives; The driver's facial region is located from the driver's image information using a target detection model. Based on the driver's facial region, real-time temporal data of the driver's basic state is obtained. The key points of each occupant's whole body are extracted from the occupant's image information using a lightweight Transformer model. Based on the key points of the occupant's whole body, real-time time series data of the occupant's basic state are obtained. Based on the fusion modeling of driver's basic state data and passenger's basic state data, a temporal convolutional network is used to perform complex state detection and analysis of driver's fatigue state, distraction behavior and coma risk, and output the driver's fatigue level, distraction index and coma risk probability value. Based on the pre-trained ResNet classification model, the occupants are distinguished as adults, children, and infants. According to the occupant type, the corresponding 3D pose estimation algorithm is called to perform pose recognition and physiological state abnormality detection and analysis, and output the occupant type, pose abnormality score, and physiological state abnormality score for each occupant. The ReIDNet model is used to associate occupant targets across cameras, and the occupant positioning data is corrected by combining a spatial coordinate mapping model. Based on the time-series trend model, time-series thresholds are calculated for the driver's fatigue level, distraction index and coma risk probability value to assess whether the marked driver has dangerous driving behavior. Based on the semantic risk model, predefined semantic risk labels are activated according to the occupant's occupant type, posture abnormality score, and physiological state abnormality score to determine whether the occupant's physiological state is abnormal. Based on the driver's dangerous driving behavior, a decision instruction for high-risk driving is output; based on the abnormal physiological state of the passengers, a decision instruction for emergency rescue of the passengers is output. When both the driver exhibits dangerous driving behavior and the passengers are in an abnormal physiological state, an emergency decision instruction is generated. Based on the final decision command, a deceleration request signal and a light activation request signal for vehicle intervention are generated and transmitted to the vehicle control system interface via the in-vehicle bus to trigger active preventive measures. Transform non-biological data of drivers and occupants into extended data of the MSD dataset; The alarm mechanism is triggered according to the preset risk level classification. When the risk level is less than the preset threshold, the vehicle control system interface will actively remind the owner. When the risk level is greater than or equal to the preset threshold, the vehicle control system interface will actively remind the owner. In addition, a voice call will be established with PSAP through the cellular network based on the SIP protocol. The MSD dataset containing extended data will be uploaded to PSAP to realize active intervention and establish an emergency call channel.

7. The emergency call method based on AI vision according to claim 6, characterized in that, Receives image information of the driver and all occupants inside the vehicle, captured from different perspectives, including: It receives infrared and visible light image information of the driver and all occupants inside the vehicle, captured from different perspectives.

8. The emergency call method based on AI vision according to claim 6, characterized in that, The extended data includes at least one of the following: driver's fatigue level, distraction index, probability value of coma risk and dangerous driving behavior, and passenger's seating area, location reliability, passenger type, abnormal posture score, abnormal physiological state score and passenger emergency information.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method as claimed in any one of claims 6 to 8.

10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that can be executed by a processor to implement the method as described in any one of claims 6 to 8.

Images