Emergency ambulance intelligent terminal equipment based on multi-modal algorithm

Abstract

The invention provides intelligent emergency ambulance terminal equipment based on a multi-modal algorithm. The intelligent emergency ambulance terminal equipment comprises an intelligent video recognition module, a video format unification module, a multi-modal node recognition module, a medical equipment management system, an equipment information acquisition and transmission module and a voice control system. Through advanced computer vision and artificial intelligence technologies, the equipment realizes intelligent monitoring and management of conditions in the emergency ambulance, effectively improves the operation efficiency and safety of the emergency ambulance, ensures real-time controllability of the state of medical equipment, realizes voice operation control of medical personnel, and provides intelligent support for emergency work.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent vehicle equipment, and particularly relates to an intelligent terminal device applied to an emergency vehicle, which realizes intelligent management of the whole process of the emergency vehicle operation through integration of various advanced information technologies. BACKGROUND

[0002] The following problems exist in the traditional emergency vehicle in the process of emergency response and medical rescue:

[0003] 1. The operation safety in the emergency vehicle mainly depends on manual supervision, and lacks real-time intelligent monitoring means;

[0004] 2. The video data format is not unified, which is not conducive to subsequent analysis and processing;

[0005] 3. The identification of key time nodes depends on manual recording, which has errors and is low in efficiency;

[0006] 4. The medical equipment management is scattered, and it is difficult to grasp the equipment state and position in real time;

[0007] 5. The equipment data acquisition and transmission are not timely, which affects remote medical guidance;

[0008] 6. The operation interface is complex, and medical staff is inconvenient to operate in an emergency.

[0009] Therefore, it is necessary to develop an intelligent emergency vehicle terminal device which can comprehensively solve the above problems. SUMMARY

[0010] The present application aims to provide an intelligent emergency vehicle terminal device to solve the problems proposed in the background.

[0011] To achieve the above purpose, the present application provides the following technical scheme: an intelligent emergency vehicle terminal device, comprising:

[0012] An intelligent video recognition module adopts a deep learning algorithm to analyze video streams in the vehicle in real time, automatically detects operation behaviors affecting safe driving, and timely issues an alarm;

[0013] A video format unification module supports input of various video encoding formats, converts them into a unified video format, and facilitates storage and subsequent processing;

[0014] A multi-modal node recognition module integrates video data, positioning information and vehicle operation data, and accurately identifies key time nodes of the emergency vehicle operation by using a multi-modal fusion algorithm;

[0015] A medical equipment management system realizes electronic registration, state monitoring and intelligent reminding of medical equipment;

[0016] Device information acquisition and transmission module, real-time acquisition of medical device working state data, and transmission to command center through wireless network;

[0017] Voice control system, using natural language processing technology, allows medical staff to operate the system and vehicle-mounted equipment through voice commands.

[0018] Further, the intelligent video recognition module uses video recognition algorithm, which can detect safety hazards such as not wearing safety belt, improper placement of equipment, and blocked passage in real time.

[0019] Further, the multi-modal node recognition module uses a spatio-temporal attention mechanism, combining LSTM and CNN models, to achieve accurate stage division of the emergency vehicle operation process.

[0020] Further, the medical equipment management system includes RFID tag reader and environmental sensor, realizing comprehensive monitoring of equipment location and usage status.

[0021] Further, the device information acquisition and transmission module supports Bluetooth 5.0, Wi-Fi 6 and 5G, 4G communication technology, ensuring real-time and reliability of data transmission.

[0022] Further, the voice control system uses an end-to-end speech recognition model, supports multi-turn dialogue and context understanding, and realizes natural and smooth human-computer interaction. BRIEF DESCRIPTION OF DRAWINGS

[0023] [ Figure 1 ] System architecture diagram of the present application

[0024] [ Figure 2 ] Workflow diagram of multi-modal node recognition module

[0025] [ Figure 3 ] Structure block diagram of voice control system DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0027] As shown in Figure 1 The intelligent emergency vehicle terminal device mainly includes the following six modules:

[0028] Intelligent video recognition module: This module collects real-time video streams inside the vehicle through the vehicle-mounted camera and uses target detection algorithms to identify behaviors that affect safe driving, such as incorrect use of safety equipment, improper placement of medical equipment, and obstruction of emergency passages. Once an abnormal situation is detected, the driver and medical personnel are immediately alerted through audible and visual alarms.

[0029] Video format unification module: This module receives video signals from different cameras, supports multiple video encoding formats such as H.264, H.265, and MPEG-4, and uses high-performance encoding and decoding chips to convert them into a unified video format, balancing video quality and storage efficiency for subsequent centralized storage and intelligent analysis.

[0030] Multi-modal node recognition module: This module integrates results from video analysis, positioning data, vehicle speed sensor data, and vehicle door status signals, uses a spatio-temporal attention mechanism to fuse multi-source information, and uses a deep neural network model to accurately identify key time nodes of ambulance operations, including departure preparation completion, leaving the hospital, arriving at the accident scene, starting to load the patient, patient loading completion, driving to the hospital, arriving at the hospital, and on-duty during transit, and automatically records the timestamps and related environmental parameters of each node.

[0031] Medical equipment management system: This system includes RFID electronic tag readers and various environmental sensors (temperature, humidity, vibration, etc.) for electronic registration and management of on-board medical equipment. Each device is equipped with an RFID tag, and the system can monitor the usage status, storage location, and environmental conditions of the equipment in real time, automatically update records when the equipment is taken out or returned, and issue alerts when the equipment status is abnormal.

[0032] Device information acquisition and transmission module: This module collects real-time operational data of medical equipment through various sensors and interfaces, such as defibrillator battery status and monitor vital signs data, and transmits these data along with video data to the remote medical command center after encryption through built-in Bluetooth 5.0, Wi-Fi 6, 5G, and 4G communication modules, supports breakpoint resume and data integrity verification, and ensures reliable transmission of critical information.

[0033] Voice control system: This system uses an end-to-end deep learning speech recognition model, supports natural language understanding and multi-round dialogue, and medical personnel can query device status, retrieve medical record information, control video recording and transmission, and other operations through simple voice commands. The system also supports voiceprint recognition to ensure that only authorized personnel can use voice control functions.

[0034] This intelligent ambulance terminal device integrates multiple advanced information technologies to achieve intelligent management of the entire ambulance operation process, improving emergency response efficiency and safety, and providing faster and more professional medical service support for patients.

Claims

1. An intelligent ambulance terminal device based on a multi-modal algorithm, characterized by, Comprise: Intelligent video recognition module for real-time collection of in-vehicle video and intelligent recognition, detecting operating behaviors that affect the safe driving of the ambulance; Video format unification module for converting collected video data of different formats into a preset unified format; Multi-modal node recognition module integrating video data, vehicle state information, and location information, automatically identifying key time nodes in the operation process of the ambulance through multi-modal artificial intelligence algorithms; Medical equipment management system for electronic management of in-vehicle medical equipment, including equipment state monitoring and location tracking; Device information acquisition and transmission module for real-time acquisition of medical equipment data and transmission to the remote medical center through wireless networks; Voice control system allowing medical staff to operate the functions of the system through voice commands.

2. The intelligent ambulance terminal apparatus according to claim 1, characterized by, The intelligent video recognition module uses deep learning algorithms to identify unsafe behaviors such as incorrect use of safety equipment, improper placement of medical equipment, and blocked emergency passages.

3. The intelligent ambulance terminal apparatus of claim 1, wherein, The multi-modal node recognition module combines GPS positioning data, vehicle speed data, and video content analysis to accurately identify key nodes such as ambulance arrival at the scene, driving to the scene, arrival at the scene, patient loading, hospital delivery, and on-duty during transit.

4. The intelligent ambulance terminal apparatus of claim 1, wherein, The medical equipment management system includes RFID tag recognition and sensor networks to monitor equipment usage status and location information in real time.

5. The intelligent ambulance terminal apparatus of claim 1, wherein, The device information acquisition and transmission module supports multiple communication methods such as Bluetooth, Wi-Fi, and cellular networks to ensure real-time and reliable data transmission.

6. The intelligent ambulance terminal apparatus of claim 1, wherein, The voice control system uses natural language processing technology to understand and execute voice commands from medical staff, enabling intelligent control of terminal devices and on-board medical equipment.

Images