A multi-modal smart emergency response system and method

Abstract

A multimodal intelligent emergency rescue response system and method are disclosed. The system, centered on a dispatch center interface module, integrates eight modules including a user terminal, multimodal emergency calls, and location tracking. It deploys the "Life Green Card" health record management function via a WeChat mini-program. Users can call for help using voice, text, sign language, and other multimodal methods. After dual authentication, the system simultaneously collects centimeter-level location data, real-time physiological parameters, and traffic data. Based on a path planning algorithm that integrates A / B algorithm and real-time traffic conditions, it selects the optimal medical center and pushes the information to the 120 dispatch platform, traffic police, and hospitals, opening a green channel. Simultaneously, it coordinates remote consultations and dissemination of surrounding assistance, constructing comprehensive rescue support. This invention solves the problems of inefficient emergency information transmission, inaccurate location tracking, and poor inclusiveness in traditional emergency services, achieving "second-level response, precise dispatch, and full-process assistance" in pre-hospital emergency care, improving emergency response efficiency and success rates. It is suitable for all types of people, especially the elderly and users with special disabilities.

Description

Technical Field

[0001] This invention relates to the field of medical emergency information technology, specifically to a multimodal intelligent emergency response system and method. Background Technology

[0002] Currently, pre-hospital emergency care mainly relies on the traditional telephone emergency call model, which has many pain points: low information transmission efficiency, requiring dispatchers to spend time inquiring about key information such as the patient's basic information and medical history; insufficient positioning accuracy, making it difficult to accurately locate the patient's position in complex environments; poor service inclusiveness, failing to meet the emergency call needs of special groups such as the deaf and mute, and people with speech impairments; disconnected health records, preventing rescuers from obtaining key information such as the patient's underlying diseases and allergies in advance, affecting treatment efficiency; and lack of real-time health monitoring and professional guidance, making it easy for patients' conditions to worsen due to improper handling while waiting for rescue. Existing emergency care systems are multifunctional and singular, either focusing only on location dispatch or only providing health monitoring, failing to achieve multi-module collaborative linkage, and lacking sufficient integration with official dispatch platforms and hospital treatment systems, making it difficult to form a complete emergency service loop. Summary of the Invention

[0003] The purpose of this invention is to provide a multimodal intelligent emergency response system and method to solve the problems of inefficient emergency information transmission, inaccurate positioning, poor inclusiveness, and disconnected records in existing emergency response systems.

[0004] To solve the above problems, the technical solution of the present invention is as follows: A multimodal intelligent emergency response system includes a user terminal module, a multimodal emergency call module, an identity authentication and file matching module, a dispatch center interface module, a dynamic data synchronization module, a positioning module, a real-time health monitoring module, a remote consultation module, and a surrounding assistance dissemination module. The dispatch center interface module is the core hub of the system, and it communicates with the multimodal emergency call module, identity authentication and file matching module, dynamic data synchronization module, positioning module, real-time health monitoring module, remote consultation module, surrounding assistance dissemination module, as well as the external 120 dispatch platform, traffic police PDA mobile terminal, and target medical center. The user terminal module is communicatively connected to the multimodal emergency call module, the identity authentication and file matching module, the dynamic data synchronization module, the real-time health monitoring module, and the surrounding assistance dissemination module. The dynamic data synchronization module is also connected to external municipal population health databases, hospital diagnosis and treatment systems, and drug supply terminals. The remote consultation module also communicates with external hospital experts. The surrounding assistance dissemination module is also connected to a communication base station.

[0005] Furthermore, the real-time health monitoring module includes a control unit, a motion monitoring unit, and a biological monitoring unit; both the motion monitoring unit and the biological monitoring unit are communicatively connected to the control unit, and the control unit is communicatively connected to the dispatch center interface module and the user terminal module, respectively; the motion monitoring unit has a built-in gyroscope sensor and a linear accelerometer sensor, and the biological monitoring unit has a built-in heart rate sensor, body temperature sensor, and blood pressure sensor.

[0006] Furthermore, the positioning module has a built-in Beidou and GPS dual-mode positioning component and an electronic map module. The positioning module is connected to the dispatch center interface module in real time, and the dispatch center interface module is connected to the ambulance's on-board terminal for communication.

[0007] Furthermore, the remote consultation module includes a signal transmission unit and a video conferencing unit; the signal transmission unit is bidirectionally connected to the data flow interface of the dispatch center interface module, and the video conferencing unit is communicatively connected to the user terminal module and the external hospital expert terminal respectively.

[0008] Furthermore, the multimodal emergency call module includes a signal processing unit, a voice acquisition unit, a text input unit, an image transmission unit, a video call unit, a sign language interaction unit, and a one-click help unit; the voice acquisition unit, text input unit, image transmission unit, video call unit, sign language interaction unit, and one-click help unit are all communicatively connected to the signal processing unit, and the signal processing unit is communicatively connected to the user terminal module and the dispatch center interface module, respectively.

[0009] A multimodal intelligent emergency response method includes the following steps: S1: Users fill out health records via WeChat mini-program or their family members fill them out on their behalf. The system connects with the municipal population health database and hospital diagnosis and treatment system to automatically supplement and update users' age, medical records and electronic prescription information, and trigger card registration reminders for people who have just turned 80 years old. S2: After the user completes dual authentication with password and gesture, the system collects physiological parameters and motion status data in real time through wearable devices or external medical devices and stores them in the cloud database; S3: When a user initiates a request through the multimodal emergency call module, the system simultaneously obtains location information, traffic information from roadside cameras, and user health monitoring data; S4: Users can describe their symptoms using text, pictures, videos, or sign language emojis, and dispatchers or doctors can respond using text or video. S5: The system selects the optimal medical center based on location and traffic information, and packages and pushes user files, health data, and emergency call content to the 120 dispatch platform, traffic police PDA, and medical center to open a green channel for rescue. S6: Data from medical equipment inside the ambulance is transmitted to the expert's terminal in real time, and the expert provides treatment suggestions through the remote consultation module; S7: The system uses the user's location as the center to send the distress message to other users and emergency contacts within a preset range. If there is no response, the range is expanded and the message is repeatedly sent until the number of responses reaches the target. S8: After the emergency treatment is completed, the system automatically archives the emergency treatment process data, synchronizes the electronic prescription to the medication supply end, and pushes subsequent rehabilitation guidance and follow-up reminders to the user.

[0010] Furthermore, the authentication method in step S2 includes: the user inputs a real-time password, the system uses speech-to-text technology to perform keyword similarity matching with the password stored in the database, if the match is consistent, the system obtains a real-time gesture, compares it with a preset gesture, generates an authentication success signal, and starts health monitoring.

[0011] Furthermore, the positioning method in step S3 includes: obtaining the initial position through dual-mode positioning of Beidou and GPS, performing coordinate transformation and data fusion by combining electronic maps, and correcting positioning errors using image information collected by roadside cameras.

[0012] Furthermore, the method for selecting the optimal medical center in step S5 includes: the system acquiring information on medical centers within a preset range centered on the user's location, based on fusion A... The algorithm is a path planning algorithm that combines path length, traffic data, and weather data to calculate arrival time. The algorithm is based on Algorithm A and integrates real-time traffic correction coefficients, weather impact coefficients, and road priority coefficients to dynamically adjust the travel cost. After pruning and parallel computing optimization, it quickly selects the medical center with the shortest arrival time and corresponding treatment capabilities, and simultaneously pushes the optimal rescue route to the ambulance and traffic police PDA. The optimal route is dynamically updated every 30 seconds based on real-time traffic conditions during the journey.

[0013] Furthermore, the method for spreading distress calls in the surrounding area in step S7 includes: the communication base station sends distress information to rescue terminals within a preset range, with the user terminal as the center. After any rescue terminal responds, the distress call is spread to the preset range again with that terminal as the center. The operation is repeated until the number of responding rescue terminals reaches the target number. The rescue terminals include rescue organization communication terminals, emergency contact terminals, and surrounding user terminals.

[0014] The beneficial effects of this invention are as follows: 1. The multimodal emergency call mechanism breaks the limitations of single communication, and the voice, text, sign language and other methods work together to adapt to different groups of people, especially solving the emergency call problem for the deaf and mute and people with language impairments, and greatly improving the inclusiveness and coverage of emergency services.

[0015] 2. The combination of Beidou + GPS dual-mode positioning and roadside camera correction technology achieves centimeter-level accurate positioning. Coupled with an intelligent route planning algorithm that integrates real-time traffic conditions, weather and road priority, it dynamically optimizes the rescue route, significantly shortens the rescue response and travel time, and ensures efficient and accurate rescue.

[0016] 3. The dynamic synchronization and real-time monitoring of health records are linked to integrate key information such as the user's basic diseases and allergy history in advance, and provide real-time feedback on changes in physiological parameters, providing rescue personnel with accurate diagnosis and treatment basis, effectively reducing the risk of pre-hospital treatment and improving the safety of treatment.

[0017] 4. The dual linkage of remote consultation and surrounding assistance enables in-hospital experts to provide real-time guidance for pre-hospital treatment, while also quickly mobilizing surrounding rescue forces to respond first, building a comprehensive guarantee of "professional rescue + civilian support", maximizing the golden treatment time and improving the success rate of emergency treatment.

[0018] 5. Deployed based on WeChat mini-program, no additional download or installation is required. It supports family members in creating files and performing operations, adapts to the usage habits of the elderly, lowers the threshold for use, and forms a closed loop of emergency services with dynamic data synchronization and rehabilitation guidance functions, improving the user's medical experience.

[0019] 6. The various modules of the system are interconnected through standardized communication protocols, and can be seamlessly connected with external platforms such as the municipal population health database and hospital diagnosis and treatment systems, breaking down information silos, realizing the optimized allocation of emergency resources and multi-departmental collaboration, and improving the overall operational efficiency of the emergency medical system. Attached Figure Description

[0020] The invention will be further described below with reference to the accompanying drawings: Figure 1 This is a diagram showing the overall module connection relationship of the system of the present invention; Figure 2 This is an exploded view of the internal structure of the module of the present invention; Figure 3 This is a flowchart illustrating the execution of the path planning algorithm of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] A multimodal intelligent emergency response system includes a user terminal module, a multimodal emergency call module, an identity authentication and file matching module, a dispatch center interface module, a dynamic data synchronization module, a positioning module, a real-time health monitoring module, a remote consultation module, and a surrounding assistance dissemination module. The dispatch center interface module is the core hub of the system. It adopts a server (e.g., Huawei RH2288H V5), with a built-in Linux operating system and customized data integration algorithms. It communicates with the multimodal emergency call module, identity authentication and file matching module, dynamic data synchronization module, positioning module, real-time health monitoring module, remote consultation module, and surrounding assistance dissemination module via RJ45 gigabit network ports. It also communicates with the external 120 dispatch platform, traffic police PDA mobile terminal (e.g., Huawei Mate60 Pro), and target medical center HIS system through API interfaces to achieve standardized format conversion, priority sorting, and accurate distribution of multi-source data, with a system response delay of no more than 1 second. The user-end module is developed based on WeChat Developer Tools V2.34.4 and deployed on WeChat Mini Programs. It supports Android 9.0 and above. It communicates with the multimodal emergency call module, identity authentication and file matching module, dynamic data synchronization module, real-time health monitoring module, and surrounding help dissemination module through the Mini Program API. It is used to provide a visual operation interface for health record registration, emergency call method selection, and rescue progress viewing. The dynamic data synchronization module uses a data synchronization gateway (e.g., F5 BIG-IP LTM 1600) to communicate with external municipal population health databases, hospital diagnosis and treatment systems (such as Siemens Soarian), and drug supply management systems (such as Alibaba Health Pharmacy Management Platform) via HTTPS protocol. It supports dual modes of incremental data synchronization and full data backup, and communicates with user terminal modules and dispatch center interface modules respectively to update user health records and electronic prescription information in real time. The positioning module has a built-in Beidou (BDS-3) + GPS dual-mode positioning component (e.g., Ublox F9P) and an electronic map module (using Gaode Map SDK V9.8.0). It connects to the dispatch center interface module in real time via a UART interface. The dispatch center interface module communicates with the ambulance's onboard terminal (e.g., Hikvision DS-M5500) via a 4G / 5G network to obtain the user's accurate location information and surrounding traffic data. Combined with the electronic map, it enables offline caching and real-time traffic updates. The real-time health monitoring module includes a control unit, a motion monitoring unit, and a biological monitoring unit. Each unit transmits data via the I2C communication protocol. The control unit uses a microcontroller (e.g., STM32H743VI) with a built-in data cache chip. It communicates with the dispatch center interface module and the user terminal module via Bluetooth 5.2 and WiFi 6, respectively, to receive and process raw data. The motion monitoring unit incorporates a gyroscope sensor (e.g., MPU6050) and a linear accelerometer sensor (e.g., ADXL345), with a sampling rate of 100Hz and measurement ranges of ±2000° / s and ±16g, respectively, to collect user motion status data and construct a human motion trajectory model. The biological monitoring unit incorporates a heart rate sensor (e.g., MAX30102), a body temperature sensor (e.g., DS18B20), and a blood pressure sensor (e.g., MPX5700DP), with measurement accuracies of ±1bpm, ±0.5℃, and ±2mmHg, respectively, to collect the user's core physiological parameters in real time. The remote consultation module includes a signal transmission unit and a video conferencing unit. The signal transmission unit uses a 5G modem (e.g., Quectel RM500Q) and bidirectionally interfaces with the dispatch center interface module via PCIe 4.0. The video conferencing unit integrates Tencent Meeting SDK and DingTalk Meeting API, and communicates with the user terminal module and external hospital expert terminals (computer: Lenovo ThinkStation P620 workstation, mobile: iPad Pro 2024 model) respectively. It supports stable video calls at 30fps and is used to transmit data from medical equipment in the ambulance and real-time video streams. It can digitally record the consultation process. The surrounding distress call dissemination module establishes communication with surrounding rescue terminals through a communication base station that supports 4G / 5G dual-mode (e.g., Huawei BTS5900). It adopts a transmission method that combines broadcasting and point-to-point push, and communicates with the user terminal module and the dispatch center interface module respectively to expand the coverage of distress information. The multimodal emergency call module includes a signal processing unit, a voice acquisition unit, a text input unit, an image transmission unit, a video call unit, a sign language interaction unit, and a one-button emergency call unit. Each functional unit communicates with the signal processing unit via an SPI bus. The signal processing unit uses a digital signal processor (e.g., TI TMS320C6748) and communicates with the user terminal module and the dispatch center interface module via a USB-C interface. The voice acquisition unit uses a microphone array (e.g., Yamaha YMF820) and supports voice pickup within 3 meters. The text input unit supports Pinyin, handwriting input, and large font display for the elderly (font size ≥ 24 points). The image transmission unit supports 1080P image capture and H.265 format transmission. The video call unit supports adaptive two-way video with bandwidth of 1Mbps-10Mbps. The sign language interaction unit has a built-in library of 200+ commonly used first aid sign language emoticons and sign language videos. The one-button emergency call unit includes a physical trigger button with a travel of ≥ 2mm and a pressure of 500±100g, as well as a virtual button in a mini-program.

[0023] Furthermore, the motion monitoring unit of the real-time health monitoring module collects user motion data through a gyroscope sensor and a linear accelerometer sensor to construct a human motion trajectory model. When the similarity to a preset fall model reaches 95%, a collision alarm is automatically triggered. The biological monitoring unit collects physiological data through a heart rate sensor, a body temperature sensor, and a blood pressure sensor to generate a heart health coefficient. When the coefficient exceeds a preset threshold, an early warning is automatically triggered. Both the motion monitoring unit and the biological monitoring unit are communicatively connected to the control unit. The control unit synchronizes the filtered and noise-reduced effective data to the dispatch center interface module and the user terminal module.

[0024] Furthermore, the positioning module obtains the initial position through a dual-mode BeiDou and GPS positioning component, performs coordinate transformation and data fusion in conjunction with an electronic map module, and uses image information collected by roadside cameras (e.g., Hikvision DS-2CD7A26G0-IZS) to correct positioning errors, ultimately achieving centimeter-level positioning; the positioning module and the dispatch center interface module work together using a fusion A... The algorithm is a path planning algorithm based on real-time traffic data. It uses Algorithm A as its core framework and evaluates the function... Guided path search, where The actual driving cost from the starting point to the current node (integrating road condition correction coefficient, road priority coefficient, and weather impact coefficient). The algorithm estimates the cost from the current node to the destination (based on the Haversine formula and road topology correction). It collects real-time traffic data from multiple channels (once every 30 seconds), converting it into a traffic correction coefficient of 0.8-5.0. This coefficient, combined with weather influence coefficients (0.8-2.5) and road priority coefficients (0.5-1.5), dynamically adjusts the node's travel cost. Through pruning strategies and parallel computing optimization, the single path planning time does not exceed 1 second. It also supports dynamic path updates every 30 seconds, ensuring rescue vehicles arrive at the target location in the shortest possible time. The path planning algorithm, combined with weather data, filters the medical center with the shortest arrival time and pushes the optimal rescue route to the ambulance's onboard terminal and the traffic police's PDA mobile terminal.

[0025] Fusion A Path planning algorithms based on real-time traffic data include: (I) Core framework of the algorithm; A As a heuristic search algorithm, the algorithm evaluates a function. Guide the direction of path search, where: This represents the distance from the starting point (user's location) to the current node. The actual driving cost in this algorithm It not only includes the physical distance between nodes, but also incorporates dynamic parameters such as real-time traffic correction coefficient and road capacity coefficient; Indicates starting from the current node The estimated cost to the destination (target medical center) is based on the straight-line distance calculated using the Haversine formula, and corrected by combining the road network topology of the electronic map to ensure that the deviation between the estimated cost and the actual driving cost does not exceed 10%. The algorithm maintains the nodes to be explored using a priority queue, and selects nodes each time. The node with the smallest value is expanded until the destination node is found, thus achieving efficient path filtering.

[0026] (ii) Real-time traffic data fusion; 1. Traffic Data Sources and Preprocessing: Real-time traffic data is collected through multiple channels, including roadside cameras (such as traffic monitoring cameras and vehicle-mounted monitoring equipment), traffic management department API interfaces, and navigation map SDKs (such as Gaode Maps and Baidu Maps real-time traffic interfaces). The collection frequency is once every 30 seconds to ensure data timeliness. The collected data includes road congestion levels (smooth traffic, slow traffic, congested, severe congestion), average driving speed, road construction information, and traffic accident warnings.

[0027] 2. Dynamic adjustment of road condition weights: Converting real-time road conditions into road condition correction coefficients. The value ranges from 0.8 to 5.0. Unobstructed road sections (average vehicle speed > 60 km / h): Reduce the weight of driving costs; slow-moving sections (40km / h < average speed ≤ 60km / h): Maintain basic weighting; congested road sections (20km / h < average speed ≤ 40km / h): Increase the weighting of driving costs; severely congested road sections (average speed ≤20km / h): This significantly increases the weight of driving costs, guiding the algorithm to prioritize avoidance of certain obstacles. The corrected actual driving cost... This ensures that the algorithm prioritizes routes with good road conditions.

[0028] (III) Fusion of multi-dimensional auxiliary parameters; 1. Weather Data Adaptation: Real-time weather data (such as rainfall, snowfall, fog, high temperature, etc.) is obtained through the China Weather Network API and converted into weather impact coefficients. No severe weather: Light rain, light wind, and other minor weather influences: Moderate weather impacts include heavy rain, moderate snow, and dense fog. Heavy rain, blizzards, and strong typhoons severely impact the weather. Final node travel cost This enables the algorithm to prioritize roads with stable road conditions and strong drainage / anti-skid capabilities (such as urban arterial roads and highways) in severe weather.

[0029] 2. Road Priority Weighting: Set priority coefficients based on road type. Prioritize ensuring the passage efficiency of emergency vehicles: Dedicated emergency rescue channel: Urban expressways and highways: Main urban roads: Secondary roads: Side roads and residential roads: Combining priority-based driving costs This ensures that the algorithm prioritizes high-priority roads and reduces traffic congestion.

[0030] (iv) Algorithm execution flow; 1. Initialization phase: Input the starting point coordinates (user's centimeter-level positioning position), the ending point coordinates (target medical center location), and the electronic map road network topology data (including node, edge, and road attributes). Initialize the priority queue, the set of visited nodes, and the path cost matrix.

[0031] 2. Data Fusion Phase: Synchronously acquire real-time traffic data and weather data, and calculate traffic correction coefficients for each road segment. Weather Influence Coefficient and road priority coefficient And update the travel costs between nodes.

[0032] 3. Path search phase: Based on A Algorithm framework, with corrected driving costs and estimated costs Calculate the evaluation function Iteratively expand the nodes in the priority queue until the destination node is found, and output the initial planned path.

[0033] 4. Dynamic update phase: During the rescue vehicle's journey, real-time traffic data is collected again every 30 seconds. If the traffic conditions on the current route deteriorate (such as sudden congestion or traffic accidents) or a better route becomes available (such as improved traffic conditions on other routes), the route search process is re-executed to generate an updated optimal route, which is then pushed to the ambulance's onboard terminal and the traffic police's PDA mobile terminal to ensure that the route adapts to changes in traffic conditions in real time.

[0034] (v) Algorithm performance optimization; 1. Pruning strategy: During the search process The value exceeds the current optimal path Pruning nodes with a value of 1.5 times reduces invalid searches, improves algorithm execution efficiency, and ensures that the time for a single path planning operation does not exceed 1 second.

[0035] 2. Parallel computing: Multi-core parallel processing technology is used to divide the road network into regions and search and calculate nodes in different regions, further shortening the path planning response time.

[0036] Furthermore, the signal transmission unit of the remote consultation module ensures data transmission stability through a 5G industrial module (e.g., Huawei ME909S-821), and the video conferencing unit supports the access of medical devices in the ambulance (e.g., Philips IntelliVue MX800 ECG monitor) via Bluetooth 5.2 protocol, transmitting physiological data to the expert in real time. The expert can view the data, observe the user's condition, and provide treatment suggestions through the video conferencing unit.

[0037] Furthermore, the signal processing unit of the multimodal distress call module converts the raw data of each distress call unit into a standardized format, the voice acquisition unit adapts to users with clear speech to quickly transmit distress information, the text input unit solves the problem of inconvenience in voice distress calls, the image transmission unit helps the dispatch center to intuitively judge the danger, the video call unit enables face-to-face communication, the sign language interaction unit breaks down the barriers to distress calls for deaf and mute users, and the one-click help unit adapts to users with impaired consciousness and mobility to quickly initiate rescue requests. All units work together to achieve multimodal barrier-free distress calls.

[0038] A multimodal intelligent emergency response method includes the following steps: S1: Users fill out health records via WeChat mini-program or their family members fill them out on their behalf. The record fields include basic personal information, underlying diseases, allergy history, emergency contact, and past medical records. The system connects with the municipal population health database and hospital diagnosis and treatment system through API interface to automatically supplement and update user age, medical records and electronic prescription information. When the system detects that the user has just turned 80 years old, it will trigger a card registration reminder through a combination of mini-program pop-up and SMS. S2: Before a user initiates an emergency call, dual authentication is performed using both a password and a gesture: The user enters a 6-digit real-time password, which is then converted to text using Baidu AI Voice Recognition SDK V3.0. The system performs keyword similarity matching with the password stored in the database (matching threshold ≥90%). If a match is found, the system captures the real-time gesture using the phone's camera and compares it with preset gestures (such as "heart" or "raise hand"). If the comparison is successful, an authentication success signal is generated and health monitoring is initiated. The system collects physiological parameters and motion status data through wearable devices (such as Huawei Watch GT4) or external medical devices and stores them in the Alibaba Cloud OSS object storage cloud database. S3: When a user initiates a request through any unit of the multimodal emergency call module, the system simultaneously starts the positioning module, the real-time health monitoring module, and the roadside camera data interface; the positioning module obtains the initial position through dual-mode positioning of Beidou and GPS, performs coordinate transformation and data fusion in combination with electronic map, and uses image information collected by roadside cameras to correct positioning errors; the real-time health monitoring module uploads the latest physiological parameters; the roadside cameras provide on-site traffic congestion information; S4: Users can choose to describe their symptoms using text, images, videos, or sign language emojis. Text input supports voice-to-text input, images can be captured in real time or selected from the album, video calls support one-click access to the dispatch center, and sign language emojis can be quickly found through category search. After receiving information through the dispatch center terminal, dispatchers or doctors can respond via text or video. Pre-set sign language guidance videos can be sent to deaf and mute users. S5: The system uses a route planning algorithm to obtain information on medical centers within a preset range (default 10 kilometers) centered on the user's location. Based on the route length, road condition data, and weather data obtained through the China Weather Network API, it calculates the arrival time and selects the medical center with the shortest arrival time and corresponding treatment capabilities (e.g., for cardiovascular and cerebrovascular diseases, priority is given to matching the cardiology department of a tertiary hospital). Simultaneously, the system packages and pushes the user's profile, health data, and emergency call content to the 120 dispatch platform, traffic police PDA, and medical center to open a green channel for rescue. S6: Medical equipment in the ambulance connects to the remote consultation module via Bluetooth 5.2 protocol, transmitting physiological data to the expert in real time. The expert can view the data and observe the user's condition through the video conferencing unit, and provide treatment suggestions such as medication dosage and emergency operation. S7: The surrounding distress call dissemination module sends distress information, including location and symptom summary, to rescue terminals within a preset range (default 500 meters) centered on the user terminal via a communication base station. Rescue terminals include community first aid station walkie-talkies (e.g., Motorola GP328D+) and other rescue organization communication terminals, emergency contact terminals, and surrounding user terminals. If no response is received within a preset time (default 3 minutes), the dissemination range is automatically expanded to 1 kilometer, repeating the operation until the number of responding rescue terminals reaches the target number (default 3). S8: After the emergency rescue is completed, the system automatically archives data such as the call record, location trajectory, health data, consultation content, and treatment results into the user's health record. It synchronizes the electronic prescription to the drug supply end through the dynamic data synchronization module, generates personalized rehabilitation guidance (such as dietary suggestions and exercise plans) based on the user's condition, and pushes it to the user and emergency contacts through mini-program pop-ups and SMS, and reminds them of the follow-up time.

[0039] Furthermore, in step S2, identity authentication uses a dual verification mechanism to prevent malicious accidental triggering and waste of emergency resources, voice-to-text technology improves the convenience of password input, gesture verification further ensures the accuracy of user identity, and health monitoring initiated after successful authentication enables real-time collection of physiological data to provide data support for subsequent treatment.

[0040] Furthermore, step S3 integrates multi-source information including "location, health, and environment" to provide a comprehensive basis for dispatching decisions. Centimeter-level positioning solves the problems of "large errors and reliance on networks" in traditional positioning, ensuring that rescuers can quickly locate users even in complex environments (such as urban villages and underground parking garages).

[0041] Furthermore, step S5 uses intelligent algorithms to achieve "precise matching of medical resources + rapid clearing of rescue channels", shortening rescue response time, improving average dispatch efficiency, and avoiding treatment delays caused by mismatch of medical resources.

[0042] Furthermore, step S7 mobilizes surrounding rescue forces through multiple rounds of dissemination, building a dual guarantee of "professional rescue + civilian support," which is especially suitable for scenarios where professional rescue arrives slowly in remote areas, thus gaining golden treatment time for users.

[0043] Example: Scenario 1: Emergency treatment for an elderly person suffering from sudden high blood pressure An 82-year-old user had registered for a "Life Green Card" profile via a WeChat mini-program, recording their history of hypertension and penicillin allergy. One day, experiencing sudden dizziness at home, they triggered an emergency request via the mini-program's "one-click help" function. After successful identity verification, the real-time health monitoring module, using a smart bracelet, recorded a blood pressure of 180 / 110 mmHg (exceeding the threshold). The positioning module obtained a precise location (with an error of 3cm) and synchronized surrounding traffic conditions (no congestion). The dispatch center interface module integrated the data and pushed it to the nearest tertiary hospital and traffic police PDA. The traffic police coordinated to clear the way, and the ambulance departed within 3 minutes. En route, the remote consultation module established a video channel between the in-vehicle medical staff and a cardiology expert, who instructed on sublingual administration of antihypertensive medication. Simultaneously, the surrounding assistance dissemination module sent a distress message to neighbors within 500 meters, who promptly arrived to assist. The ambulance reached the scene within 10 minutes. After the emergency, the system archived the treatment data, synchronized the electronic prescription to the medication supply end, and provided rehabilitation guidance on a low-salt diet and regular blood pressure monitoring.

[0044] Scenario 2: First aid for accidental injury to a deaf-mute user A deaf user triggers an SOS by sending a sign language emoji for "arm bleeding" through the sign language interaction unit. The multimodal SOS module converts the emoji into a text description and transmits it to the dispatch center. The positioning module accurately pinpoints the user's location (in an underground parking garage, with an error of 5cm), and the real-time health monitoring module collects blood oxygen levels at 92% (slightly low). The dispatch center pushes the information to the 120 dispatch platform and the target hospital. The remote consultation module allows dispatchers to confirm the amount of bleeding and the cause of injury with the user via text. The surrounding assistance dissemination module sends SOS messages to users within a 1-kilometer radius. Two users with first aid certificates respond promptly and, under the remote guidance of experts, perform hemostasis and bandaging until the ambulance arrives.

[0045] This embodiment, through verification of specific component selection, communication protocols, algorithm logic, and application scenarios, fully demonstrates that the multimodal intelligent emergency response system and method of the present invention are practical and feasible. It can effectively solve problems such as inefficient transmission of traditional emergency information, inaccurate positioning, poor inclusiveness, and disconnected records, significantly improving emergency response efficiency and success rate. The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A multimodal intelligent emergency response system, characterized in that, It includes a user terminal module, a multimodal emergency call module, an identity authentication and file matching module, a dispatch center interface module, a dynamic data synchronization module, a positioning module, a real-time health monitoring module, a remote consultation module, and a surrounding assistance dissemination module; The dispatch center interface module is connected to the multimodal emergency call module, identity authentication and file matching module, dynamic data synchronization module, positioning module, real-time health monitoring module, remote consultation module, surrounding assistance dissemination module, as well as the external 120 dispatch platform, traffic police PDA mobile terminal, and target medical center. The user terminal module is communicatively connected to the multimodal emergency call module, the identity authentication and file matching module, the dynamic data synchronization module, the real-time health monitoring module, and the surrounding assistance dissemination module. The dynamic data synchronization module is also connected to external municipal population health databases, hospital diagnosis and treatment systems, and drug supply terminals. The remote consultation module also communicates with external hospital experts. The surrounding assistance dissemination module is also connected to a communication base station.

2. The multimodal intelligent emergency response system according to claim 1, characterized in that, The real-time health monitoring module includes a control unit, a motion monitoring unit, and a biological monitoring unit; both the motion monitoring unit and the biological monitoring unit are communicatively connected to the control unit, which is communicatively connected to the dispatch center interface module and the user terminal module, respectively; the motion monitoring unit has a built-in gyroscope sensor and a linear acceleration sensor, and the biological monitoring unit has a built-in heart rate sensor, body temperature sensor, and blood pressure sensor.

3. The multimodal intelligent emergency response system according to claim 1, characterized in that, The positioning module integrates BeiDou and GPS dual-mode positioning components and an electronic map module. The positioning module is connected in real-time to the dispatch center interface module, which in turn communicates with the ambulance's onboard terminal. The positioning module and dispatch center interface module work together using a fusion A... Algorithms and path planning algorithms based on real-time traffic data.

4. The multimodal intelligent emergency response system according to claim 1, characterized in that, The remote consultation module includes a signal transmission unit and a video conferencing unit; the signal transmission unit is bidirectionally connected to the data flow interface of the dispatch center interface module, and the video conferencing unit is communicatively connected to the user terminal module and the external hospital expert terminal respectively.

5. The multimodal intelligent emergency response system according to claim 1, characterized in that, The multimodal emergency call module includes a signal processing unit, a voice acquisition unit, a text input unit, an image transmission unit, a video call unit, a sign language interaction unit, and a one-click help unit. The voice acquisition unit, text input unit, image transmission unit, video call unit, sign language interaction unit, and one-click help unit are all communicatively connected to the signal processing unit, which is communicatively connected to the user terminal module and the dispatch center interface module, respectively.

6. A method for using the multimodal intelligent emergency response system according to any one of claims 1 to 5, applied to the system according to any one of claims 1 to 5. Includes the following steps: S1: Users fill out health records via WeChat mini-program or their family members fill them out on their behalf. The system connects with the municipal population health database and hospital diagnosis and treatment system to automatically supplement and update users' age, medical records and electronic prescription information, and trigger card registration reminders for people who have just turned 80 years old. S2: After the user completes dual authentication with password and gesture, the system collects physiological parameters and motion status data in real time through wearable devices or external medical devices and stores them in the cloud database; S3: When a user initiates a request through the multimodal emergency call module, the system simultaneously obtains location information, traffic information from roadside cameras, and user health monitoring data; S4: Users can describe their symptoms using text, pictures, videos, or sign language emojis, and dispatchers or doctors can respond using text or video. S5: The system selects the optimal medical center based on location and traffic information, and packages and pushes user files, health data, and emergency call content to the 120 dispatch platform, traffic police PDA, and medical center to open a green channel for rescue. S6: Data from medical equipment inside the ambulance is transmitted to the expert's terminal in real time, and the expert provides treatment suggestions through the remote consultation module; S7: The system uses the user's location as the center to send the distress message to other users and emergency contacts within a preset range. If there is no response, the range is expanded and the message is repeatedly sent until the number of responses reaches the target. S8: After the emergency treatment is completed, the system automatically archives the emergency treatment process data, synchronizes the electronic prescription to the medication supply end, and pushes subsequent rehabilitation guidance and follow-up reminders to the user.

7. The multimodal intelligent emergency response method according to claim 6, characterized in that, The authentication method in step S2 includes: the user inputs a real-time password, the system uses speech-to-text technology to match the password stored in the database with keywords, if the match is consistent, the system obtains a real-time gesture, compares it with a preset gesture, generates an authentication success signal and starts health monitoring.

8. The multimodal intelligent emergency response method according to claim 6, characterized in that, The positioning method in step S3 includes: obtaining the initial position through dual-mode positioning of Beidou and GPS, performing coordinate transformation and data fusion by combining electronic maps, and correcting positioning errors using image information collected by roadside cameras.

9. The multimodal intelligent emergency response method according to claim 6, characterized in that, The method for selecting the optimal medical center in step S5 includes: the system acquiring information on medical centers within a preset range centered on the user's location, based on fusion A The algorithm is a path planning algorithm that combines path length, traffic data, and weather data to calculate arrival time. The algorithm is based on Algorithm A and integrates real-time traffic correction coefficients, weather impact coefficients, and road priority coefficients to dynamically adjust the travel cost. After pruning and parallel computing optimization, it quickly selects the medical center with the shortest arrival time and corresponding treatment capabilities, and simultaneously pushes the optimal rescue route to the ambulance and traffic police PDA. The optimal route is dynamically updated every 30 seconds based on real-time traffic conditions during the journey.

10. The multimodal intelligent emergency response method according to claim 6, characterized in that, The method for spreading distress calls in the surrounding area in step S7 includes: the communication base station sends distress information to rescue terminals within a preset range, with the user terminal as the center. After any rescue terminal responds, the distress call is spread to the preset range again with that terminal as the center. The operation is repeated until the number of responding rescue terminals reaches the target number. The rescue terminals include rescue organization communication terminals, emergency contact terminals, and surrounding user terminals.

Images