Intelligent multifunctional rescue vehicle first-aid command method and system

Abstract

The application discloses a kind of intelligent multifunctional rescue vehicle first-aid command methods and systems, it is related to emergency medical technical field, including: receiving first-aid call event, screening dispatchable vehicle, determine optimal rescue vehicle and optimal driving path;According to the video data of rescue scene, the operation action of medical staff, rescue material and corresponding material state are identified, and the voice instruction of medical staff is extracted according to audio data;With timestamp as index, video recognition result, audio extraction result and patient vital sign data are aligned, voice instruction is bound with operation action and material state, the body coordinates of medical staff, material state and patient vital sign data are mapped to virtual rescue scene, and guidance data generated based on virtual rescue scene is received.Construct the whole process digital closed loop of dispatch-path-site-visualization-guidance coverage, realize the intelligentization, collaboration and traceability of first-aid whole process, and comprehensively improve first-aid quality and management level.

Description

Technical Field

[0001] This invention relates to the field of emergency medical technology, and in particular to an intelligent, multifunctional emergency rescue vehicle emergency command method and system. Background Technology

[0002] Ambulances are core equipment in hospital emergency settings, typically equipped with emergency medications, instruments, and life support tools for rapid intervention in patients experiencing sudden critical illness. With the development of the Internet of Things, artificial intelligence, and communication technologies, intelligent management of medical equipment and optimization of emergency procedures through sensing, identification, positioning, and data interaction have become important directions for improving emergency efficiency, ensuring the quality of care, and safeguarding medical safety.

[0003] While current emergency dispatch and on-site rescue technologies have incorporated digital methods such as image recognition, RFID (Radio Frequency Identification), voice interaction, and audio / video recording, achieving automation and informatization of some processes, the following shortcomings still exist:

[0004] The utilization of multimodal data on-site is insufficient, and the dynamic perception and real-time collaboration capabilities are weak. Although the existing system has audio and video acquisition and recording functions, it is mostly limited to post-event review and simple query. It lacks real-time recognition of operational actions, rescue materials and status in on-site videos, as well as effective extraction and parsing of voice commands in audio. Multi-source data has not achieved time alignment and semantic association, and cannot form a complete link of command-action-materials-vital signs, resulting in fragmented on-site status perception.

[0005] The lack of virtual-real integrated visualization decision support results in insufficient remote guidance and on-site collaboration. Existing solutions lack the ability to construct dynamic rescue scenarios using real-time audio and video data, making it difficult to intuitively present the rescue process and resource status. They also fail to provide on-site medical staff with intuitive decision guidance and resource direction through visualization and simulation. Similarly, the remote command terminal cannot provide precise guidance through dynamic virtual scenarios, and information transmission between the field and the back-end is not intuitive or timely, resulting in low efficiency in decision support and collaboration.

[0006] The level of intelligence in emergency resource dispatch is insufficient, and the ability to make comprehensive decisions from multiple dimensions is lacking. Existing dispatch methods are often limited to the management of a single ambulance. When an emergency call is made, the optimal emergency resources cannot be automatically located and dispatched, resulting in delayed emergency response, unreasonable resource allocation, and the need for manual requisition of medicines, leading to slow response.

[0007] The system modules are isolated, failing to form a complete closed-loop digital emergency rescue system. Existing improvements are mostly focused on single aspects, such as material expiration date identification, batch inventory, or voice recording. Modules such as dispatching, route planning, on-site perception, data fusion, virtual scenarios, and remote guidance are independent of each other, resulting in gaps in information transmission. The system has failed to construct a complete closed loop from emergency call to intelligent dispatching, route planning, on-site rescue, virtual and real visualization, and remote guidance, thus limiting the overall improvement of emergency rescue efficiency and quality. Summary of the Invention

[0008] To address the aforementioned issues, this invention proposes an intelligent multi-functional emergency vehicle command method and system, constructing a full-process digital closed loop covering dispatch, route, on-site, visualization, and guidance, thereby achieving intelligent, collaborative, and traceable emergency care throughout the entire process and comprehensively improving the quality and management level of emergency care.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides an intelligent, multi-functional emergency vehicle command method, comprising:

[0011] Based on the received emergency call events, determine the incident location and material resource needs, screen available vehicles, and comprehensively rank available vehicles from the dimensions of theoretical travel time, resource matching degree, regional load balance, predicted road condition delay time and vehicle priority to determine the optimal rescue vehicle.

[0012] Based on the location of the optimal rescue vehicle, the location of the incident, and the constraints and traffic weights of the passage route, the optimal driving route is determined to control the rescue vehicle's movement to the incident location.

[0013] Based on video data from the rescue scene, identify the actions of medical staff, rescue supplies and their corresponding status, and extract the voice commands of medical staff based on audio data;

[0014] Using timestamps as indexes, the video recognition results, audio extraction results, and patient vital signs data within the same time window are aligned. After binding voice commands with operational actions and material status, the limb coordinates of medical staff, material status, and patient vital signs data are mapped to the virtual rescue scene and visualized.

[0015] The generated virtual rescue scenario will be pushed out and displayed, and guidance data generated based on the virtual rescue scenario will be received.

[0016] As an alternative implementation method, key points of the limbs of medical staff and patients are extracted, and the key points of the limbs are drawn into a skeletal diagram according to the connection relationship of the human skeleton; and skeletal motion features are calculated, including: calculating the difference in coordinates of key points of limbs in adjacent frames to represent the direction and speed of limb movement; calculating the joint angles of skeletal connections to represent the bending state of the limbs; calculating the distance between the key points of the limbs of medical staff and patients to represent the interaction relationship of limbs; and identifying the operation actions of medical staff based on the key points of limbs and skeletal motion features.

[0017] As an alternative implementation method, key points of state characteristics of each rescued material are extracted, and the state of the material is determined by calculating the quantifiable geometric features between the key points of state characteristics; including: calculating the distance between any two key points of state characteristics and the ratio of the two distances; calculating the included angle formed by three key points of state characteristics; and calculating the area of ​​the polygon enclosed by multiple key points of state characteristics.

[0018] As an alternative implementation method, based on the coordinates of key points of the limbs of medical staff, the corresponding virtual medical staff model is driven by inverse kinematics to achieve synchronous mapping of posture and position.

[0019] Update the location and status of rescue equipment, medicines, and consumables in the virtual rescue scenario based on the status of supplies;

[0020] Patient vital signs data are bound to a virtual patient model and visualized through numerical floating annotations, waveforms, and changes in the model's appearance. At the same time, text annotations are added to operation nodes, with the annotation locations being non-critical areas of the simulation screen.

[0021] As an alternative implementation method, the conditions that the dispatchable vehicle meets are: the rescue vehicle is in standby or driving status, with no malfunctions or maintenance records; the quantity of materials in stock is greater than or equal to the quantity of materials required; the responsibility area of ​​the rescue vehicle is in the same area or adjacent to the incident location; the rescue vehicle currently has no unfinished emergency rescue missions, and the number of remaining dispatchable vehicles in its responsibility area is greater than or equal to 1.

[0022] As an alternative implementation method, the process of determining the optimal driving route includes:

[0023] The constraints on the travel route include: route priority, traffic restrictions, and time constraints. Route priority includes: emergency lanes > main roads within the hospital > general lanes > pedestrian lanes. Traffic restrictions include: areas where vehicles are prohibited from passing, the direction of travel on one-way lanes, and temporary control lanes. Time constraints include: setting the maximum allowed travel time for the route based on the emergency call level.

[0024] The passage weight includes: path length score, road condition score, and lane priority score. The overall passage weight is the sum of the path length score, road condition score, and lane priority score. The path with the highest score is the optimal travel path.

[0025] Secondly, the present invention provides an intelligent multi-functional emergency vehicle command system, comprising:

[0026] The dispatch module is configured to determine the location of the incident and the demand for supplies and resources based on the received emergency call events, and to screen dispatchable vehicles. The dispatchable vehicles are comprehensively ranked from the dimensions of theoretical travel time, resource matching degree, regional load balance degree, road condition prediction delay time and vehicle priority to determine the optimal rescue vehicle.

[0027] The planning module is configured to determine the optimal driving route based on the location of the optimal rescue vehicle, the location of the incident, and the constraints and traffic weights of the route, so as to control the rescue vehicle to travel to the incident location.

[0028] The recognition module is configured to identify the actions of medical staff, rescue supplies and their corresponding status based on video data from the rescue scene, and to extract the voice commands of medical staff based on audio data.

[0029] The mapping module is configured to use timestamps as indexes to align video recognition results, audio extraction results, and patient vital signs data within the same time window. After binding voice commands with operation actions and material status, it maps the limb coordinates of medical staff, material status, and patient vital signs data to the virtual rescue scene and visualizes them.

[0030] The push module is configured to push and display the generated virtual rescue scenario and receive guidance data generated based on the virtual rescue scenario.

[0031] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0032] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0033] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] This system enables multi-dimensional, comprehensive intelligent dispatching, significantly improving emergency response speed and the rationality of resource allocation. By comprehensively ranking dispatchable vehicles based on theoretical travel time, resource matching degree, regional load balance, predicted road condition delay time, and vehicle priority, the optimal emergency vehicle can be quickly determined. By combining path constraints and traffic weights to plan the optimal driving route, the system achieves precise dispatching and efficient allocation of emergency resources, shortening emergency arrival time and improving overall emergency response efficiency.

[0036] Real-time multimodal data recognition and fusion construct a complete on-site rescue perception chain. By recognizing medical staff's actions, rescue supplies, and their status through video, and extracting voice commands through audio, the video recognition results, audio extraction results, and patient vital signs data are time-aligned using timestamps as indexes. Voice commands are semantically bound to actions and supply status, forming a complete association between command, action, supply, and vital signs, enabling comprehensive, real-time, and structured perception of the rescue scene.

[0037] Constructing a virtual emergency rescue scenario that blends the real and virtual worlds enhances the capabilities of visualized decision-making and remote guidance. By mapping the limb coordinates of medical staff, the status of supplies, and the vital signs of patients onto the virtual emergency rescue scenario and visually representing them, the rescue process and resource status can be presented intuitively and dynamically. By pushing virtual scenarios and receiving remote guidance data, intuitive decision-making guidance and precise collaborative support are provided for on-site medical staff and remote command, reducing the risk of misoperation and improving the standardization and success rate of emergency rescues.

[0038] By integrating data and processes across multiple stages, a fully closed-loop digital emergency medical services system is formed. This system deeply integrates emergency call processing, intelligent dispatching, route planning, on-site perception, multi-source data fusion, virtual scenario visualization, and remote guidance. It eliminates information silos and process gaps, constructing a fully digital closed loop covering dispatching, routes, on-site conditions, visualization, and guidance. This achieves intelligent, collaborative, and traceable emergency medical services throughout the entire process, comprehensively improving the quality and management level of emergency medical services.

[0039] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0041] Figure 1The flowchart illustrates the intelligent multi-functional emergency rescue vehicle emergency command method provided in Embodiment 1 of the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0045] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0046] Example 1

[0047] This embodiment provides a smart, multi-functional emergency vehicle management method for interaction between the emergency vehicle terminal, the cloud server, and the hospital terminal. The emergency vehicle terminal establishes a two-way real-time data communication connection with the cloud server via a wireless network, and the cloud server establishes a one-way or two-way command and data connection with the hospital terminal via the hospital intranet or wireless network.

[0048] The emergency vehicle terminal is deployed inside and on the vehicle body to perform local perception, interaction, control and navigation. The cloud server has a hospital-wide cloud platform scheduling and command center unit for global resource management, intelligent decision-making, scheduling and data collaboration. The hospital terminal has an intelligent logistics supply unit for automated drug supply.

[0049] like Figure 1 As shown, the main steps include the following:

[0050] Based on the received emergency call events, determine the incident location and material resource needs, screen available vehicles, and comprehensively rank available vehicles from the dimensions of theoretical travel time, resource matching degree, regional load balance, predicted road condition delay time and vehicle priority to determine the optimal rescue vehicle.

[0051] Based on the location of the optimal rescue vehicle, the location of the incident, and the constraints and traffic weights of the passage route, the optimal driving route is determined to control the rescue vehicle's movement to the incident location.

[0052] Based on video data from the rescue scene, identify the actions of medical staff, rescue supplies and their corresponding status, and extract the voice commands of medical staff based on audio data;

[0053] Using timestamps as indexes, the video recognition results, audio extraction results, and patient vital signs data within the same time window are aligned. After binding voice commands with operational actions and material status, the limb coordinates of medical staff, material status, and patient vital signs data are mapped to the virtual rescue scene and visualized.

[0054] The generated virtual rescue scenario will be pushed out and displayed, and guidance data generated based on the virtual rescue scenario will be received.

[0055] In this embodiment, video and audio data of the rescue scene are collected in real time by cameras and microphone arrays deployed inside and around the vehicle body; by analyzing the video data in real time, the operation actions of medical staff, rescue materials and their status, patient vital signs data, etc. are identified; by analyzing the audio data in real time, key voice commands are identified; by fusing the video and audio analysis results, a virtual rescue scene is dynamically generated and rendered, which maps the personnel position, material status and key operation nodes in real time in a three-dimensional visualization form.

[0056] Specifically:

[0057] S1: Comprehensive data collection is achieved through fixed cameras on the emergency vehicle, panoramic cameras, and personal recorders worn by medical staff, covering patient status, medical staff operations, the use of emergency equipment, and the status of medications and consumables. Simultaneously, a microphone array is used to collect ambient sound and voice commands from medical staff.

[0058] S2: Based on video data from the rescue scene, identify the actions of medical personnel, rescue supplies and their corresponding status, and extract the voice commands of medical personnel based on audio data; including:

[0059] (1) Perform preprocessing such as anti-shake, noise reduction and image enhancement on the video data to solve the problem of blurry image caused by the bumpy ride of the rescue vehicle.

[0060] (2) Use deep learning object detection models (such as YOLOv8 / YOLOv9) to perform frame-level analysis on the preprocessed video data, segment the three core areas such as the patient treatment area, medical equipment area and medical operation area, and shield redundant areas such as the ambulance interior and the external environment.

[0061] At the same time, identify targets within the area, including patients, medical staff, and emergency supplies; emergency supplies include emergency equipment (such as defibrillators, monitors, infusion devices, etc.), medicines (such as adrenaline, atropine ampoules, etc.) and consumables (such as syringes, electrode pads, endotracheal tubes, etc.).

[0062] (3) Using a deep learning pose estimation algorithm, extract the coordinates of key points of the limbs (hands, elbows, shoulders, heads, torsos, etc.) of medical staff and patients, draw the coordinates of key points of the limbs into a skeleton diagram according to the connection relationship of the human skeleton, and obtain the dynamic skeleton frame by playing across frames.

[0063] The core connectivity relationships are as follows: Torso: left shoulder → right shoulder → left hip → right hip → left shoulder; Arms: left shoulder → left elbow → left wrist, right shoulder → right elbow → right wrist; Legs: left hip → left knee → left ankle, right hip → right knee → right ankle; This yields the dynamic human skeletal sequence (i.e., the set of key point coordinates in the time dimension).

[0064] Based on this, temporal feature modeling is performed on dynamic human skeletal sequences. Combined with temporal convolutional networks or 3D convolutional network models, the spatiotemporal features of continuous skeletons are learned to identify the operation actions of medical staff and the patient's status; including but not limited to: chest compressions (frequency / depth) by medical staff, endotracheal intubation, defibrillation operations (such as defibrillation electrode placement), medication operations (drug extraction, injection, etc.); and the patient's chest rise and fall, body position, limb movement, etc.

[0065] In this embodiment, standardization and denoising processes are performed to address issues such as coordinate offset (human body position movement), scale inconsistency (different human body heights), and noise (pose estimation error) in the extracted skeletal sequence. Specifically:

[0066] (a) Coordinate standardization to eliminate position / scale effects, including:

[0067] Translation normalization: Using the center of gravity of the human body (such as the average coordinates of key points on the torso) as the origin, subtract the coordinates of the center of gravity from the coordinates of all key points to eliminate the positional shift of the human body in the image.

[0068] Scaling normalization: Calculate the mean length of human bones (such as the length of the shoulder, elbow, and wrist), divide the coordinates of all key points by this mean, and eliminate scale differences between different human bodies.

[0069] (b) Temporal denoising; the skeleton sequence exhibits jitter in the temporal dimension (such as small shifts in keypoints between adjacent frames). Moving average filtering or Kalman filtering is used for denoising to preserve the temporal trend of the motion and eliminate inter-frame errors in pose estimation, including:

[0070] Moving average: For the coordinates of each key point, take the average value of the current frame ± n frames (n=2 / 3, adapted sampling rate).

[0071] Kalman filtering: Suitable for noisy scenarios, it predicts and updates the state of key point coordinates, improving the smoothness of the sequence.

[0072] (c) In addition to the original keypoint coordinates, skeletal motion features can be calculated as supplementary input to improve the model's action recognition capability; including:

[0073] Inter-frame difference: Calculates the difference in the coordinates of key points in adjacent frames to represent the direction and speed of limb movement;

[0074] Joint angles: Calculate the angles between skeletal connections (such as the angles between the shoulder, elbow, and wrist, and the angles between the hip, knee, and ankle) to characterize the flexion state of the limb (in medical procedures, joint angles are a core feature, such as the flexion angle of the arm during disinfection).

[0075] Skeletal distance: Calculates the Euclidean distance between key points of the limbs of medical staff and patients (such as the distance between the right wrist of medical staff and the arm of patients), representing the limb interaction relationship (such as limb contact between medical staff and patients).

[0076] In this embodiment, a basic model is first built using a Temporal Convolutional Network (TCN) or a 3D-Convolutional Neural Network (3D-CNN) to recognize medical actions and patient status. If higher accuracy is required, a dual-branch fusion model of TCN+3D-CNN is then built to take into account both temporal and spatial features.

[0077] The design of the temporal convolutional network for the skeletal sequence is as follows:

[0078] Input layer: limb keypoint coordinates and skeletal motion features;

[0079] Convolutional layer: 1D causal dilated convolution is used (the skeletal sequence is a 1D temporal feature), the kernel size is 3, and the dilation rate increases in powers of 2 (1, 2, 4, 8), and the receptive field is rapidly expanded;

[0080] Activation function: GELU activation function is used, which is smoother than ReLU activation function and is suitable for small medical datasets;

[0081] Pooling layer: Employs temporal pooling (such as average pooling / max pooling) to compress the time dimension and extract key temporal features;

[0082] Output layer: Employs a fully connected layer + Softmax multi-classification to output the probability distribution of medical and nursing action categories and patient status categories (dual-branch output, completing dual recognition in one inference).

[0083] (4) After identifying the rescue materials in the area, extract the key points of the state characteristics of each rescue material through key point detection / matching technology (such as the break point of the stopper of the ampoule, the scale of the syringe plunger, and the edge of the adhesive backing of the electrode). Finally, analyze the spatial distribution, geometric features, and matching relationship of the key points of the state characteristics to complete the judgment of the material status.

[0084] Specifically:

[0085] (4-1) Set the conditions for determining the status of rescued supplies; an example is as follows;

[0086] In emergency equipment, the status of a defibrillator includes: power on / off (screen on / off), electrode plates removed / not removed (slot empty), and electrode plates attached / not attached.

[0087] The monitor status includes: leads connected / not connected (whether the lead wires are attached to the patient), and screen display normal / abnormal.

[0088] The status of the infusion device includes: infusion tubing connected / not connected, drip chamber containing fluid / not containing fluid, and flow rate regulator on / off.

[0089] The status of the medicine includes: unopened (the stopper is intact and there are no breaks), opened (the stopper is broken / missing), and empty (there is no medicine in the bottle).

[0090] Among consumables, syringe status includes: no medication drawn (push rod returned to center, syringe empty), medication drawn (push rod pulled out, syringe with liquid column), and used (needle cap missing).

[0091] Electrode pad status includes: not pasted (adhesive backing intact, no wrinkles), pasted (adhesive backing missing / wrinkled, attached carrier), and used (gel layer detached).

[0092] The status of the endotracheal tube includes: unused (cloaca intact, no secretions) and used (cloaca has secretions, tube body has stains).

[0093] (4-2) Define the key points of the status characteristics of each type of rescue materials; an example is as follows:

[0094] (a) Key features of the state of the ampoule include: the top of the stopper A1, the connection point between the stopper and the body A2, and the upper edge of the liquid column inside the ampoule A3;

[0095] If the mask outlines of A1 and A2 are continuous, it means the bottle is unopened; if the connection is broken / missing, it means the bottle has been opened; if the distance between A3 and the bottom of the bottle is less than a set threshold, it means the bottle is empty, otherwise there is liquid.

[0096] (b) Key features of the syringe state include: B1, the contact point between the plunger and the syringe barrel, B2, the upper edge of the syringe liquid column, and B3, the tip of the needle cap;

[0097] If the distance between B1 and B2 is greater than the set threshold, it means that B1 has been extracted; otherwise, it has not been extracted. If B3 is missing (cannot be detected), it means that B3 has been used; otherwise, it has not been used.

[0098] (c) Key features of the electrode sheet include: adhesive edge points C1-C4; gel layer edge points C5-C8;

[0099] If the outlines of C1-C4 are smooth and without wrinkles, it means they are not pasted; if the outlines are wrinkled or incomplete, it means they have been pasted. If C5-C8 are blurry or detached, it means they have been used; otherwise, they are not used.

[0100] (d) Key features of the defibrillator electrode plates include: electrode plate slot edge points D1-D4, and electrode plate gel layer center D5;

[0101] If the key points of the electrode plate overlap with the key points D1-D4 of the card slot (there is no outline of the electrode plate within D1-D4), it means that it has not been removed; otherwise, it has been removed. If D5 is in contact with the human body / carrier, it means that it has been pasted; otherwise, it has not been pasted.

[0102] (4-3) For the detected rescue materials, the YOLOv8-Pose model is used to output the coordinates of key points of the state features, and the state of the materials is determined according to the state judgment conditions.

[0103] This includes determining the state by extracting quantifiable geometric features between key points; including:

[0104] Distance between key points: Calculate the Euclidean distance between any two key points; such as the distance between B1 and B2, to determine whether to extract the medicine.

[0105] Angles between key points: Calculate the angle formed by three key points and calculate the size of the angle by vector dot product; for example, the angle between A1, A2 and the midpoint of the bottle body to determine whether the bottle stopper is broken.

[0106] Proportional characteristics: Calculate the ratio of two distances; for example, the ratio of the height of the syringe liquid column to the total length of the syringe to determine the amount of liquid medicine drawn.

[0107] Area characteristics: Calculate the area of ​​the polygon enclosed by multiple key points; for example, the area enclosed by the edge points C1-C4 of the electrode backing adhesive to determine whether there are wrinkles (the area becomes smaller after pasting).

[0108] (5) For medical devices with digital interfaces, such as electrocardiogram monitors, ventilators, and infusion pumps, structured patient vital signs data, including real-time quantitative vital signs data such as electrocardiogram monitoring, blood pressure, blood oxygen, heart rate, and respiratory rate, as well as operation record data such as defibrillation, cardiopulmonary resuscitation, and medication automatically collected by vehicle-mounted emergency equipment, are directly acquired through network protocols (such as DICOM, Digital Imaging and Communications in Medicine, which is an international standard for medical images and related information). This type of data has the highest priority; and the timestamp is synchronized with the video stream through the general interface of medical devices to ensure that the image and data correspond one-to-one with no time difference.

[0109] For vital sign monitoring devices without digital interfaces, auxiliary information is collected through video data: First, the monitor screen is selected from the video frame by locating the region of interest; then, perspective correction and image enhancement are performed on the screen area to solve problems such as angle shift, reflection, and blur; then, the data is processed in three parallel paths: numerical information (heart rate, blood pressure, blood oxygen saturation, etc.) is identified through OCR (Optical Character Recognition), status icons (red alarm flashing, green normal indicator) are identified through image classification, and waveform categories (such as ventricular fibrillation waveform, ventricular tachycardia waveform, etc.) are identified through image analysis, and finally, the final auxiliary vital sign information is output.

[0110] (6) In terms of voice data processing, an end-to-end voice recognition model is used to convert audio data into voice text in real time. Named Entity Recognition (NER) technology is used to extract entity information such as drug name, dosage, and operation action from the voice text. Based on a pre-built domain dictionary containing standard rescue terminology, voice commands and types are determined, such as issuing medical orders, requesting assistance, and confirming operations.

[0111] Among them, the entity types and annotation specifications for rescue scenarios are defined, the entity types that the NER model needs to extract are clarified, and only entities directly related to instruction parsing are extracted in combination with the actual needs of rescue scenarios to avoid invalid entity extraction. At the same time, a unified entity annotation specification is formulated to provide a standard for model training and entity extraction.

[0112] For example: drug names, various drugs used in the rescue process, including generic names and standard aliases; such as adrenaline, atropine, lidocaine, dopamine;

[0113] Dosage values, the specific numerical values ​​for the dosage of the drug; such as 2, 5, 0.5;

[0114] Dosage units are standard units for drug dosage, including units of mass, volume, and concentration; such as mg, ml, μg / min, IU;

[0115] Operational actions refer to specific medical procedures performed by medical staff for medications / patients, including administration methods and resuscitation procedures; such as intravenous bolus injection, subcutaneous injection, intubation, defibrillation, chest compressions, requests, and confirmations.

[0116] The object of the operation, including the patient's body part and medical equipment; such as the precordial region, airway, defibrillator, and syringe.

[0117] Instruction intent words are the core vocabulary that directly reflects the type of voice instruction; such as medical orders, assistance, confirmation, issuance, request, and report.

[0118] Therefore, after extracting based on the above entity types, entity association is performed. The dose value + dose unit is associated with complete dose information, and the operation action + operation object is associated with complete operation information, ensuring the integrity of entity information; for example, the value 1 + unit mg → dose 1mg, the action intubation + object airway → operation airway intubation.

[0119] Then, a three-level progressive matching is performed with the domain dictionary:

[0120] First layer: Precise entity matching: The extracted entities are precisely matched against the standard entity dictionary, non-standard entities are converted into standard entities, and missing entity information is completed. Example: Extract the entity "push" → match the operation dictionary → complete as the standard action "intravenous injection"; extract the entity "renin" → match the drug dictionary → correct as the standard drug "adrenaline".

[0121] The second layer: Fuzzy text matching: The standardized complete text is fuzzily matched against the instruction template dictionary to determine if the text conforms to the standard template for resuscitation instructions, filtering out non-resuscitation instruction text (such as casual conversation or irrelevant dialogue). Example: Text "Patient's heart rate 120" → No instruction template match → Determined as an invalid instruction.

[0122] The third layer: Entity combination matching: The matched standard entities are combined by type and matched with the entity type combination of the instruction template to confirm whether the core entity combination of the instruction is complete, and to complete any missing non-core entities.

[0123] S3: All acquisition modules are connected to a unified time server, so video, audio and patient vital signs data are assigned a consistent timestamp during acquisition. All video and audio recognition results are also bound to the corresponding timestamp, laying the foundation for the spatiotemporal alignment of subsequent multimodal data.

[0124] Using timestamps as indexes, a time alignment buffer is established, and the video recognition results (operation actions, rescue supplies and supply status), audio analysis results (voice commands) and patient vital signs data within the same time window are aligned to form continuous multimodal data frames.

[0125] A video-audio semantic mapping rule base is constructed to define the correspondence between voice commands, operation actions, and material status. This is used to perform correlation verification on multimodal data frames, bind voice commands with operation actions and material status, form operation nodes with confidence, such as defibrillation preparation nodes and drug administration nodes, and mark them with timestamps.

[0126] For example, the voice command "prepare for defibrillation" should be mapped to related actions and material status such as "medical staff walk towards or are near the defibrillator" and "pick up or check the defibrillation electrode pads"; the voice command "administer adrenaline" should be mapped to a series of events such as "adrenaline vial detected", "syringe withdrawal action", and "approaching the patient's intravenous access".

[0127] S4: In the construction of virtual rescue scenarios, the coordinates of key points of the limbs of medical staff are identified, and inverse kinematics (IK) is used to drive the corresponding virtual medical staff model to achieve synchronous mapping of posture and position.

[0128] Based on the identified material status (e.g., picked up, used, discarded), update the location and status (e.g., full / empty, activated / deactivated) of rescue equipment, medicines, and consumables in the virtual rescue scenario.

[0129] Meanwhile, the patient's vital signs data are bound to the virtual patient model and visualized through numerical floating annotations, waveforms, and changes in the model's appearance (such as color changes and special effects). At the same time, operation nodes are labeled with text (such as cardiopulmonary resuscitation: 100 times / min, blood oxygen: 85%, continuously decreasing, left chest wall trauma, active bleeding). The labels are placed in non-critical areas of the simulation screen and do not obscure core diagnostic and treatment information.

[0130] Specifically:

[0131] (1) Mapping the limb coordinates of medical staff to a virtual medical model in a virtual rescue scenario, including:

[0132] (1-1) Construct a virtual medical care model that covers the joints (neck, shoulder, elbow, wrist, fingers, waist, hip, knee, ankle) of the rescue operation. The skeletal nodes of the virtual medical care model correspond one-to-one with the collected key points of the limbs to ensure accurate mapping of the solution data.

[0133] (1-2) Based on the physiological structure of the human body and the medical and nursing emergency operation procedures, set the range of motion threshold for each joint (such as elbow flexion 0-140°, wrist rotation -90°~90°) to avoid situations such as joint hyperextension and movement deformity in the virtual medical and nursing model that do not conform to clinical reality.

[0134] (1-3) Match the coordinates of key points of the limbs (such as the right wrist and right elbow joints) with the corresponding skeletal nodes of the virtual medical care model to determine the target endpoints (such as the hand and foot) and basic joints (such as the shoulder and hip) of the IK solution.

[0135] (1-4) For different body parts, construct corresponding IK solution chains (e.g., upper limb chain: shoulder → elbow → wrist → hand, lower limb chain: hip → knee → ankle → foot) and clarify the joint hierarchy of the solution chain.

[0136] (1-5) Using the three-dimensional coordinates of the target endpoint as constraints, iterate backward from the target endpoint to the basic joint, gradually adjusting the rotation angle of each joint so that the deviation between the end position of the solution chain and the target point coordinates is ≤ a set threshold (e.g., 1 mm), while also satisfying the joint range of motion threshold.

[0137] (1-6) Perform collaborative calculations on multiple IK calculation chains (such as left and right upper limbs, trunk, and lower limbs) of the whole body, consider the spatial constraints between each calculation chain, avoid limb crossing and clipping, and ensure the naturalness of the whole body posture.

[0138] (1-7) Using the coordinates of key points on the torso (such as the waist and abdomen) of medical staff as a reference, calculate the overall spatial position and rotation angle of the virtual medical model to achieve precise synchronization between the movement, turning and other actions of medical staff in the physical scene and the virtual medical model.

[0139] (1-8) The joint rotation angle and overall position coordinates obtained by IK calculation are transmitted to the virtual medical care model in real time to drive the update of the virtual medical care model's posture and position. At the same time, the posture is optimized in real time to improve the naturalness of the movement and the clinical realism, and avoid the mechanical feeling.

[0140] (2) Mapping the status of supplies to a virtual rescue scenario, including:

[0141] (2-1) Based on the function of the rescue materials, they are divided into medicines, consumables and rescue equipment. A virtual material model is built for each type of material. A unique RFID tag is attached to each physical rescue material and bound to the corresponding virtual material model with the same ID to achieve accurate matching between physical and virtual materials. It supports the individual identification of multiple materials of the same type (such as multiple adrenaline syringes).

[0142] (2-2) Define dynamically switchable state attributes for each virtual material model. The attributes correspond one-to-one with the state of the physical rescue materials, including basic attributes (position, rotation, scaling) and business attributes (usage status: idle / picked up / used / discarded; attribute status: full / empty / enabled / disabled / low battery), etc.

[0143] (2-3) Create differentiated visual designs for virtual material models in different states so that state transitions can be visually identified. For example: a full medicine bottle / infusion bag is marked with a textured image of medicine, while an empty one is transparent and without texture; a defibrillator is activated when the screen is on and the indicator light is green, and deactivated when the screen is black and the indicator light is off; consumables are discarded when they are stained or deformed.

[0144] (2-4) Based on the material status data, standardized update rules are formulated according to the principles of location priority, status synchronization, and attribute linkage to ensure that the update of the virtual material model is synchronized with the operation of physical rescue materials in real time.

[0145] (a) Location update rules.

[0146] Idle state: The virtual material model is fixed at the initial placement position of the physical rescue materials (such as material platform, rescue vehicle), and the position coordinates are consistent with the initial coordinates of the physical rescue materials.

[0147] Pick-up / Movement Status: When physical rescue supplies are picked up and moved by medical staff, the position coordinates of the virtual supplies model follow the coordinates of the physical rescue supplies in real time. If the physical rescue supplies are held in the hand, the position of the virtual supplies model will match the coordinates of the operating hand of the virtual medical staff model (e.g., when a syringe is held in the hand, it will match the pinching part of the fingers).

[0148] Usage status: When physical emergency supplies are in use (such as syringes for administering medication or defibrillators attached to the patient's chest wall), the position of the virtual supplies model is aligned with the corresponding part of the virtual patient / operating equipment and the position is locked (such as defibrillator electrodes that move synchronously with the virtual patient model after being attached to the patient's chest wall).

[0149] Abandoned / Placement Status: When physical rescue supplies are abandoned / placed in a designated location (such as a medical waste bin or operating table), the location coordinates of the virtual supplies model are updated to the physical abandonment / placement location, and the status is switched to the corresponding type.

[0150] (b) Status and attribute update rules.

[0151] Usage Status Linkage: When the usage status of physical rescue supplies changes (e.g., idle → picked up, picked up → used, used → discarded), the usage status of the virtual supply model switches synchronously and triggers corresponding visual effect changes.

[0152] Attribute status linkage: When the attribute status of physical rescue supplies changes (such as full → empty, activated → deactivated, low power), the attribute status of the virtual material model switches synchronously and triggers the corresponding visual effect change. The attribute status and usage status are linked and constrained (such as the syringe automatically switches to empty after use, and the infusion bag cannot switch to the usage status after it is empty).

[0153] Multi-state overlay: Supports the overlay display of multiple states of virtual material models (such as picking up + using + low battery defibrillator), and the visual effects of all overlay states are presented simultaneously to ensure the integrity of state information.

[0154] Rules for the disappearance / addition of supplies: When physical rescue supplies are discarded and removed from the rescue area (e.g., thrown into a medical waste bin), the virtual supply model disappears from the virtual scene; when new supplies are added to the physical scene (e.g., a new medicine bottle is taken out of the rescue vehicle), the virtual supply model is added to the corresponding position in the virtual scene according to the physical coordinates.

[0155] (3) Map the patient's vital signs data to the virtual rescue scenario, bind the real-time collected patient vital signs data and the patient's monitoring device ID to the virtual patient model with a unique ID, and add a vital signs data receiving and parsing module to the virtual patient model so that the virtual patient model can receive and parse vital signs data in real time.

[0156] include:

[0157] (3-1) Floating labeling of vital signs values.

[0158] For discrete numerical vital signs such as heart rate, blood pressure, blood oxygen, respiratory rate, and body temperature, a floating annotation format is used for display. The virtual patient model is bound to the annotation (moving with the entire virtual patient model but not shaking with limb movements), and it is deployed above the head / side of the torso of the virtual patient model (non-treatment area). The values ​​are updated in real time with the vital sign data. When the values ​​exceed the clinical normal range, the background of the annotation flashes red (e.g., blood oxygen <90%, heart rate >150 bpm), with a flashing frequency of 2Hz, to remind attention to abnormal vital signs.

[0159] (3-2) Dynamic display of vital signs waveforms.

[0160] For continuous waveform vital signs such as electrocardiogram, pulse, and respiration, a dynamic display is presented in the form of waveform diagrams. The waveform diagrams are deployed on the side / corner of the virtual scene (such as the right side / upper left corner), independent of the virtual patient model, and do not occupy the diagnosis and treatment area. When the waveform shows a clinically abnormal pattern (such as ventricular fibrillation, ventricular tachycardia, or a straight line), the waveform line flashes the corresponding color, and an abnormal text prompt is added next to the waveform diagram.

[0161] (3-3) Visual changes in the appearance of the virtual patient model.

[0162] For abnormalities in core vital signs and changes in disease progression, the changes in the appearance of the virtual patient model are presented intuitively, closely reflecting the physical manifestations of clinical conditions. The visual changes are only slightly modified and do not alter the anatomical structure of the model, ensuring clinical realism.

[0163] Core visual design changes:

[0164] Skin color changes: Adjust the skin color of the virtual patient model according to the blood oxygen saturation value (e.g., blood oxygen ≥95% → normal rosy, 90%-95% → slightly pale, <90% → obvious pale / cyanotic); adjust the skin color according to the body temperature value (body temperature >39℃ → flushed, <35℃ → pale / grayish);

[0165] Bleeding / Trauma Display: When the disease node includes bleeding or trauma (such as left chest wall trauma or limb bleeding), the corresponding part of the virtual patient model is displayed with trauma texture and dynamic bleeding effect (active bleeding is a slowly flowing blood texture, and the bleeding effect stops after hemostasis).

[0166] Vital signs and critical illness display: When critical illnesses such as cardiac arrest, ventricular fibrillation, and hypotensive shock occur, the virtual patient model shows visual effects such as slight limb stiffness and severe cyanosis / pale skin, which are consistent with the physical manifestations of clinical critical illness.

[0167] Visual change synchronization rule: The model's visual appearance changes are synchronized with the vital signs / condition nodes in real time. The visual effect is automatically restored after the values ​​return to normal or the condition is relieved.

[0168] (3-4) Key diagnosis / condition milestones are marked with text.

[0169] For diagnostic and treatment actions (cardiopulmonary resuscitation, intubation, defibrillation, etc.), disease milestones (bleeding, ventricular fibrillation, hypotension, etc.), and operation parameters (compression frequency / depth), text labels are used to display the information. The labels are placed next to the operation site / the disease site / in blank areas of the scene, without obstruction or interference with the operation.

[0170] This embodiment uses a 3D visualization engine to render updated virtual rescue scenes in real time, and supports multi-view observation (such as global top view, first-person view following specific medical staff) and key information annotation (such as floating display of vital sign values ​​and operation step labels); the rendered video stream is pushed in real time to the large screen and terminal devices of the hospital's cloud platform dispatch and command center through a low-latency network protocol.

[0171] In this embodiment, the emergency vehicle terminal also includes an identity recognition and security access module and an electronic lock control module. The identity recognition and security access module includes a card reader and / or a face recognition camera, used to verify the identity of medical personnel. After successful verification, it sends an unlocking signal to the electronic lock control module.

[0172] The electronic lock control module consists of an electronic lock and related drive circuits. It receives remote unlocking commands from the identity recognition and security access module or the cloud server to control the opening and locking of the cabinet door.

[0173] In this embodiment, the emergency vehicle terminal also includes a communication module, which integrates a 5G / 4G or Wi-Fi 6 communication chip to establish a stable, low-latency data transmission link with the cloud server.

[0174] In this embodiment, the emergency vehicle terminal also includes an item management module, which consists of an ultra-high frequency RFID reader and antenna installed in each medicine / equipment slot, and an RFID tag affixed to each medicine / equipment. This module is used for real-time, non-contact identification of the retrieval and return of materials, and uploads the material identification, quantity changes, and timestamp information in real time.

[0175] In this embodiment, after receiving the incident location and optimal driving route from the cloud server, the emergency vehicle terminal drives to the designated incident location;

[0176] After medical staff unlock the device through the identity recognition and security access module, they can obtain the current patient information from the cloud server through the communication module.

[0177] The cloud server integrates a patient information interface and interfaces with the Hospital Information System (HIS) and Electronic Medical Record (EMR). When a rescue event is triggered, it obtains the corresponding patient's medical records, allergy history, vital sign trend information, etc., and pushes them to the designated emergency vehicle terminal.

[0178] Inside the emergency vehicle terminal, based on patient information, drug inventory, and a built-in emergency knowledge base, several drugs most likely to be used in this emergency and their specific physical locations inside the vehicle are highlighted on the screen.

[0179] During the rescue, based on the drug retrieval sequence fed back by the item management module and the identified actions of medical staff, key timers, recommended next steps, or drug dosage conversion prompts pop up in the intelligent interactive display module of the rescue vehicle terminal.

[0180] In addition, a temporal neural network model is constructed, taking into inputs including time-series data of vital signs over a continuous period, sequences of executed actions, and basic patient information (obtainable from the electronic medical record system, such as age and medical history). The model outputs the current pathological prediction status (e.g., acute myocardial infarction, hemorrhagic shock, severe arrhythmia) and its probability of occurrence, binding the patient's vital sign data and condition prediction results to the virtual patient model. Furthermore, based on the condition prediction results and the current resuscitation stage, combined with resuscitation guidelines and a reinforcement learning model, suggestions for subsequent key measures and resource allocation prompts (e.g., calling for a cardiology consultation, preparing an intra-aortic balloon pump, etc.) are generated. These suggestions are then combined with expert opinions from the cloud server to refine the next steps, and transmitted to on-site medical personnel via a field data collection system.

[0181] In this embodiment, the intelligent interactive display module consists of one or more high-brightness touch screens, used to receive control commands, dynamically display digital simulation rescue scenarios, highlight recommended drug locations, key operation steps, timer reminders, patient information, and inventory status, etc.

[0182] Understandably, regarding data privacy protection, patient resuscitation videos are considered medical privacy and need to be protected against leaks through methods such as anonymization (blurring faces and hiding personal information), encrypted transmission, and access control (only doctors involved in the resuscitation within the hospital can view them).

[0183] Specifically:

[0184] When generating virtual simulation scenes, the system automatically removes the patient's face, appearance, personal belongings, and other personal identity information, retaining only the patient's condition-related signs and actions. The original video stream is automatically blurred and key information is masked before being stored.

[0185] This includes: Desensitizing identity information: Using facial blurring and body feature blurring technology, key identity features such as the patient's face, head, and neck are blurred, and the patient's clothing and personal belongings (such as mobile phones, wallets, and jewelry) are masked / blurred; the faces of medical staff are also slightly blurred to protect their privacy, retaining only the signs and procedures related to the patient's condition.

[0186] Scene information anonymization: Information that can identify specific locations in the video (such as ambulance number, landmarks outside the window, road signs, other people's information, etc.) is masked / blurred, and only scene elements related to the rescue are retained.

[0187] Data anonymization: Only medical quantitative data related to the patient's condition is retained, and any data that could be associated with the patient's identity (such as patient's name, ID number, mobile phone number, medical insurance number) is masked. All data is presented in an anonymized form (such as patient 1, vital sign data 1).

[0188] Patient's right to know: During pre-hospital emergency care, medical staff should inform patients / family members in a concise manner that "the rescue process will involve video capture and simulation scene generation, which will only be used for in-hospital rescue and will be subject to privacy desensitization and encryption throughout the process." In special circumstances (such as when the patient is in a coma), the data can be collected first, and the family can sign the informed consent form later.

[0189] During network transmission, a dual mobile network link is adopted, with 5G as the primary and 4G as the backup, supporting seamless network switching and avoiding transmission failure due to a single network interruption. The simulation scene file and the original video stream are encrypted end-to-end using AES-256 (an advanced encryption standard algorithm using a 256-bit key length). During transmission, access is only granted to servers related to the hospital's emergency response.

[0190] During the in-hospital application phase, dedicated diagnostic terminals (computers / large screens / tablets) in the hospital's resuscitation room / emergency department / ICU decode the received encrypted simulation files in real time and display them in a visual format. The original video stream is simultaneously received, encrypted, and stored on the hospital's backend server, with hierarchical access control. Only the hospital's resuscitation manager and quality control personnel have access to view the data. All data (simulation scenes, original videos, medical data, access logs) is stored on a dedicated local hospital server and a dedicated medical cloud platform. Storage on personal devices or public cloud platforms is strictly prohibited. The cloud server must pass Level 3 Information Security Protection Certification, and the local server is physically isolated.

[0191] This solution is designed around the high emergency response, mobility, and high privacy characteristics of pre-hospital emergency care. Its core is to achieve a highly efficient closed loop across the entire chain, from raw video acquisition to real-time processing, virtual simulation generation, encrypted transmission, and in-hospital reception. All nodes meet the requirements of mobile network adaptation (4G / 5G), low latency (≤3 seconds), and lightweight computing, and are compatible with the onboard hardware environment of ambulances.

[0192] In this embodiment, the cloud server stores and updates in real time a panoramic map of resources, including the location and status of all emergency vehicle terminals in the hospital, details of drug and equipment inventory, layout of each department, and passage information.

[0193] Upon receiving an emergency call, the system automatically calculates and assigns the optimal ambulance terminal based on the aforementioned panoramic map data, and simultaneously generates the optimal driving route and sends it to the optimal ambulance terminal.

[0194] Specifically:

[0195] (1) Obtain emergency call events; including the location of the incident, information of the person being rescued (age, gender, basic medical history), diagnosis of the condition (preliminary diagnosis of the condition provided by the caller, such as myocardial infarction, cerebral hemorrhage, traumatic shock, etc., symptom description such as difficulty breathing, confusion, etc.), call time (accurate to the second, as the starting point for response timing) and call level (such as Level 1 (critical and severe illness, requiring immediate dispatch), Level 2 (severe illness, requiring dispatch within 1 minute), Level 3 (mild illness, requiring dispatch within 3 minutes, the level affects the priority of vehicle allocation).

[0196] (2) Based on emergency call events, the system automatically analyzes the medicines, equipment and other resources required for emergency care through a pre-set disease-resource matching database, and clarifies the material resource needs; for example: myocardial infarction requires defibrillator, nitroglycerin, aspirin, oxygen inhalation device, etc.; cerebral hemorrhage requires mannitol, sphygmomanometer, suction device, hemostatic dressing, etc.; traumatic shock requires tourniquet, blood transfusion set, saline, first aid kit, etc.

[0197] (3) Based on the panoramic map of cloud resources, select vehicles that meet the following conditions from all rescue vehicles as dispatchable vehicles:

[0198] Condition qualified: The rescue vehicle is in standby or driving (able to turn back quickly), with no malfunctions and no maintenance records;

[0199] Resource requirements met: Inventory quantity ≥ Demand quantity;

[0200] Regional adaptation: The area of ​​responsibility of the rescue vehicle is in the same or adjacent area as the incident location to avoid excessively long response time caused by cross-regional dispatch;

[0201] Load controllable: The emergency vehicle currently has no unfinished emergency rescue tasks, and the number of remaining dispatchable vehicles in its area of ​​responsibility is ≥1, avoiding the depletion of resources in a single area.

[0202] (4) For the selected dispatchable vehicles, the weighted summation method is used to calculate the comprehensive weight based on the following dimensions. The weight of each dimension can be adjusted according to the actual needs of the hospital. The default weight allocation is as follows:

[0203] (4-1) Theoretical travel time (weight 30 points); Calculate the theoretical travel time of each dispatchable vehicle from its current location to the incident location. The shorter the time, the higher the score.

[0204] Specific calculation method:

[0205] Based on the vehicle's current coordinates and the location of the incident, stored in the cloud, calculate the straight-line distance between the two points.

[0206] Based on the normal traffic speeds of the lanes in the area to which the vehicle belongs (preset: normal speed of emergency lane is 15km / h, normal speed of main road is 10km / h, and normal speed of ordinary lane is 5km / h), calculate the theoretical travel time (time = distance / normal speed).

[0207] The shortest theoretical travel time among the dispatchable vehicles is used as the benchmark (30 points). The score for other vehicles is calculated as (benchmark time / theoretical time of the vehicle) × 30, and the score is rounded to one decimal place.

[0208] (4-2) Resource matching degree (weight 35 points): Compare the current vehicle inventory with the material resource demand. The higher the matching degree, the higher the score.

[0209] Specific calculation method:

[0210] Core essential resource matching score (20 points): 10 points are deducted for each missing core essential resource, until all points are deducted; 20 points are awarded if the quantity of all core resources meets the requirements; if the quantity is partially met (insufficient but can be used in an emergency), points are awarded proportionally (e.g., if the requirement is 5 units and the inventory is 3 units, the score = 20 × 3 / 5 = 12 points).

[0211] Auxiliary resource matching score (15 points): Auxiliary resource satisfaction rate = (number of auxiliary resource types that meet the needs / total number of auxiliary resource types) × 100%, score = satisfaction rate × 15, score is rounded to 1 decimal place;

[0212] Total resource matching score = core essential resource matching score + auxiliary resource matching score.

[0213] (4-3) Regional load balancing (weight 10 points): Based on the current task load of the area to which the vehicle belongs, the lower the load, the higher the score, to avoid the depletion of resources in a single area.

[0214] Specific calculation method:

[0215] Area load factor = (Number of vehicles performing tasks within the area / Total number of rescue vehicles within the area) × 100%;

[0216] Score = (1 - area load rate) × 10, and the score is rounded to one decimal place; if the area load rate is ≥80%, the score is ≤2 points; if the area load rate is ≤20%, the score is ≥8 points.

[0217] (4-4) Predicting delay time based on road conditions (weight 15 points): Based on real-time road condition data, predict the possible delay time of each candidate vehicle's route. The shorter the delay time, the higher the score.

[0218] Specific calculation method:

[0219] Based on the planned route from the vehicle's current location to the incident site, and combined with the congestion level of each road segment, the predicted delay time is calculated (no delay on unobstructed roads, 0.5-1 minute delay for light congestion, 1-3 minutes delay for moderate congestion, and more than 3 minutes delay for heavy congestion).

[0220] The shortest predicted road condition delay time among the dispatchable vehicles is used as the benchmark (15 points). The score for other vehicles is calculated as (benchmark delay time / predicted road condition delay time of the vehicle) × 15. If the predicted road condition delay time is 0, 15 points are awarded. The score is rounded to one decimal place.

[0221] (4-5) Vehicle Priority (Weight 10 points): Based on the emergency level and maintenance status of the vehicle, the vehicle priority is set. The higher the priority, the higher the score.

[0222] Specific standards:

[0223] Level 1 emergency vehicle (specifically used for critical and severe medical emergencies, with the most complete equipment): 10 points;

[0224] Level 2 emergency vehicle (standard ambulance, fully equipped): 8 points;

[0225] Backup emergency vehicles (equipped to be used for supplementary dispatch): 6 points;

[0226] If the vehicle has been maintained and calibrated within the past month, an extra point will be awarded (the total score shall not exceed 10 points).

[0227] (4-6) Comprehensive weight ranking: Calculate the comprehensive weight score of each dispatchable vehicle and rank them from high to low. If dispatchable vehicles with the same score are present, the vehicle with the shorter theoretical travel time shall be selected first. If the theoretical travel time is also the same, the vehicle with the higher resource matching degree shall be selected first. If both are the same, the vehicle with the lower regional load rate shall be selected first.

[0228] (5) After determining the optimal rescue vehicle, plan the optimal driving route and simultaneously send the vehicle assignment information and navigation route to the rescue vehicle terminal to guide the vehicle to rush to the scene quickly.

[0229] Route planning is based on a panoramic map of cloud resources, combined with factors such as real-time vehicle location, accident location coordinates, real-time traffic conditions, and lane priority, to plan the optimal driving route. The specific process is as follows:

[0230] (5-1) Determine the basic planning parameters:

[0231] Starting point parameters: current coordinates of the optimal rescue vehicle and vehicle direction of travel;

[0232] Destination parameters: Coordinates of the incident location, optimal parking location (such as the entrance of the department or the entrance of the passageway, avoiding densely populated areas);

[0233] Constraint parameters:

[0234] Access priority: Emergency access > Main roads within the hospital > Regular access > Pedestrian access (vehicles are prohibited from passing through);

[0235] Traffic restrictions: areas where vehicles are prohibited from passing, directions of travel on one-way traffic lanes, and temporary control lanes (avoid);

[0236] Time constraints: Based on the emergency call classification, set the maximum allowable passage time for the route (e.g., Level 1 call ≤ 5 minutes, Level 2 call ≤ 8 minutes, Level 3 call ≤ 12 minutes).

[0237] (5-2) Initial path generation:

[0238] Using Dijkstra's algorithm, multiple candidate paths are generated based on the coordinates of the starting point, the ending point, and the passage information. Paths that do not meet the constraint parameters (such as those containing prohibited areas or exceeding the maximum allowed passage time) are eliminated, and candidate paths that meet the conditions are retained.

[0239] (5-3) For the selected candidate paths, calculate the comprehensive traffic weight of each path in combination with real-time traffic conditions, and select the optimal path.

[0240] Specific calculation method:

[0241] Path length score (30 points): The shorter the path length, the higher the score. The shortest path is the baseline (30 points), and the score for other paths is (baseline length / path length) × 30.

[0242] Traffic Smoothness Score (50 points): Based on the congestion level of each road segment, the traffic smoothness rate is calculated as (length of smooth road segment / total length of route × 100%), and the score = smoothness rate × 50; if the route includes a heavily congested road segment, 20 points will be deducted directly;

[0243] Passage Priority Score (20 points): The higher the proportion of emergency passages and main roads in the route, the higher the score. Score = (Length of emergency passage + Length of main road) / Total length of route × 20;

[0244] The overall traffic weight is calculated as follows: path length score + traffic condition score + channel priority score. The path with the highest score is the optimal path.

[0245] (5-4) After the optimal route is generated, the traffic conditions along the route are monitored in real time. If any abnormalities occur, such as sudden changes in road conditions or passage closures, the route is replanned and optimized to ensure that the route remains optimal. The specific process is as follows:

[0246] Real-time monitoring: Collect real-time traffic data for each segment of the optimal route every 30 seconds and compare it with the traffic information during route planning;

[0247] If the following situations occur, the path is considered abnormal, and replanning will be triggered:

[0248] The congestion level of a certain section of the route has been upgraded to moderate or above, and the estimated delay time is ≥1 minute;

[0249] A passage in the route is temporarily closed (e.g., due to construction or an emergency).

[0250] The vehicle deviates from the planned path by ≥10 meters and fails to return to the planned path within 10 seconds;

[0251] Based on the latest traffic conditions, the comprehensive traffic weight of candidate routes is recalculated, and a new optimal route is selected. The new optimal route is sent to the emergency vehicle terminal in real time, and the driver is simultaneously reminded of the reason for the route adjustment (such as congestion ahead, and a new route has been planned for you).

[0252] In this embodiment, after the cloud server assigns the rescue vehicle, its signal processing and connection method is as follows: the dispatching command and navigation path are sent to the designated rescue vehicle terminal through the communication link. The rescue vehicle terminal parses the command and executes the movement task. At the same time, it prompts "automatically heading to the rescue site" through the intelligent interactive display module. During the movement, the rescue vehicle terminal transmits its own position and status back to the cloud server in real time for global view update.

[0253] In this embodiment, the cloud server also includes a data intelligent analysis module:

[0254] (1) Receive real-time inventory data, usage records and partially anonymized operation data uploaded from various terminals (such as emergency vehicles, rescue units and storage terminals);

[0255] Specifically, this includes: current stock, specifications, and expiration dates of medicines, medical consumables, and emergency equipment; usage time, quantity, and corresponding emergency event number of each type of material in actual emergency events; and partially anonymized medical personnel operation information; historical consumption records; seasonal climate information; and regional emergency event statistics frequency.

[0256] (2) The data intelligent analysis module uses machine learning models to analyze the consumption patterns of resources such as medicines and consumables based on the integrated historical and real-time data. Specifically, it automatically identifies the consumption trends of different materials by analyzing the correlation between historical usage records and external factors such as seasonal changes and regional emergency incidence rates. For example, peak drug usage corresponding to specific high-incidence disease seasons and characteristic changes in consumable demand during major public events.

[0257] (3) After analyzing the consumption patterns and current real-time inventory levels, predict the total demand for various resources and key demand time points within a specified future period (such as a week or a month);

[0258] Based on the forecast results, replenishment suggestions are automatically generated, including the types and quantities of materials to be replenished and the expected replenishment time, providing data support for resource scheduling and procurement planning.

[0259] In this embodiment, the data intelligence analysis module can monitor inventory status in real time and preset dynamic safety stock thresholds for each type of resource. These thresholds are set considering factors such as the regular procurement cycle of materials and the urgency level in emergency scenarios. The module compares the current inventory data with the preset thresholds in real time. Once the inventory level is found to be close to or below the safety line, an inventory warning mechanism is automatically triggered, prompting timely replenishment. Simultaneously, the module continuously monitors the expiration information of all resources, identifying and reminding users of materials nearing their expiration date, suggesting priority use or rotation, thereby effectively avoiding resource waste due to expired materials and ensuring that inventory resources are always in a safe and usable state. After the rescue operation, the module automatically aggregates time node logs, medication records, and operation step identification information from the terminal to generate a structured electronic rescue record that conforms to medical standards.

[0260] In this embodiment, the hospital terminal includes a pharmacy storage management system, a logistics robot scheduling system, and a smart medicine box.

[0261] Specifically:

[0262] (1) The pharmacy warehouse management system receives the drug replenishment request form sent by the cloud server and automatically completes the matching and pre-picking of the hospital pharmacy inventory.

[0263] (2) The logistics robot scheduling system schedules third-party logistics robots based on the replenishment application and the location of the target emergency vehicle, and plans its delivery route from the pharmacy to the station next to the target vehicle.

[0264] The logistics robot scheduling system receives information from the intelligent interactive display module and the data intelligent analysis module in real time. It receives information such as the current inventory of medicines, medical consumables, historical consumption records, and the statistical frequency of regional emergency events. Based on the predicted demand for various resources and key demand time nodes of the intelligent interactive display module, the logistics robot scheduling system establishes data contact with the ambulance in a timely manner to replenish medical resources.

[0265] The logistics robot scheduling system calculates the theoretical travel time from the current location to the replenishment location based on the resources required by the demand side, using this as a basic weight; it compares the matching degree between each demand side's list and the logistics robot; and, combined with predicted route data, it comprehensively considers various factors such as distance and the number of demand sides that are allocated resources at one time, to quickly select the fastest and most efficient route.

[0266] (3) The smart medicine box has an RFID tag that is compatible with the item management module of the emergency vehicle terminal. It is carried by a logistics robot and delivered to the emergency vehicle. After delivery, the medicine is replenished by medical staff or with the assistance of a robotic arm. After replenishment, the RFID information of the medicine box is read by the on-site terminal to complete the inventory data synchronization.

[0267] In this embodiment, the specific implementation of the -early warning-replenishment closed loop is as follows:

[0268] The item management module uploads real-time inventory data to the cloud server via the communication module each time medicines are retrieved or returned.

[0269] The data intelligence analysis module continuously monitors inventory. When the quantity of any medicine falls below the preset safety threshold, it automatically generates a replenishment request and triggers the pharmacy warehouse management system.

[0270] The pharmacy warehouse management system processes applications, and the logistics robot scheduling system assigns robots to load the corresponding smart medicine boxes for delivery.

[0271] After restocking is completed, the on-site terminal updates the local inventory and synchronizes it to the cloud server by reading the RFID tag of the smart pillbox or the restocked medicine again, thus completing the closed loop.

[0272] This embodiment addresses the three core pain points of raw surveillance video in terms of clinical usability, information transmission efficiency, and scenario adaptability, specifically targeting the unique scenarios of pre-hospital emergency care. Simply put: raw surveillance video records facts, while virtual simulation scenarios extract clinical value; the former is unprocessed raw material, while the latter is a customized diagnostic and treatment reference for in-hospital emergency care. The core difference lies in the mobile, complex, and highly emergency-critical nature of pre-hospital emergency care. Specific reasons include:

[0273] First, the original surveillance video contains a lot of noise, requiring hospital doctors to spend time filtering it, which can lead to missed opportunities for timely rescue.

[0274] The surveillance video inside the ambulance is a panoramic record, containing a large amount of redundant information unrelated to diagnosis and treatment: such as the ambulance's interior, non-critical operations by medical staff, vehicle movement, background environmental noise, etc. The video may even be blurry or obstructed due to vehicle jolting, insufficient lighting, or obstructed views (e.g., instruments blocking the patient's vital signs, or medical staff obscuring vital parts). In the critical few minutes of hospital resuscitation, doctors need the patient's core vital signs, changes in condition, and critical procedures, not the entire video feed. If the surveillance footage were directly transmitted, doctors would have to sift through the cluttered footage to "find information," a process that consumes valuable preparation time and could potentially cause them to miss crucial details about the patient's condition.

[0275] Virtual resuscitation scenarios are generated based on real resuscitation videos, recreating changes in the patient's vital signs, resuscitation procedures, and the progression of the disease. They retain core diagnostic and treatment elements (such as chest rise and fall, ECG monitoring data, CPR procedures performed by medical staff, and the site of traumatic bleeding) and present them in a clear, stable, and visual format, enabling efficient assessment without redundant information. Doctors can intuitively assess the severity of the patient's condition in advance, accurately prepare equipment (such as specific types of endotracheal intubation tubes and surgical supplies), allocate personnel (such as having specialists on standby in the resuscitation room in advance), and plan treatment pathways (such as directly transferring the patient to the interventional radiology room instead of a regular resuscitation room), thus reducing the time from admission to core treatment.

[0276] Second, virtual simulation scenarios can fill in the information gaps in the original video, enabling a "three-dimensional assessment" of the illness.

[0277] Pre-hospital emergency monitoring videos suffer from limited perspectives and fragmented data: for example, cameras can only capture images from 1-2 fixed angles, failing to fully present the patient's position and injury distribution; the videos can only record images and cannot simultaneously integrate core medical data such as electrocardiogram monitoring, blood pressure, and blood oxygenation. Hospital doctors can only see the images and not the quantitative signs, leading to judgment biases.

[0278] The core advantage of virtual simulation scenarios lies in the fusion of multi-source information: based on the original video, real-time data from medical equipment is overlaid through technical means, key disease nodes are annotated, and the patient's three-dimensional vital signs are restored. It can even perform high-definition restoration and perspective restoration on blurry images (such as restoring the distribution of trauma throughout the patient's body through a single-view video). For example, monitoring videos of myocardial infarction patients only show medical staff performing chest compressions, while virtual simulation scenarios can simultaneously display the compression frequency, dynamic changes in electrocardiogram waveforms, and fluctuations in blood oxygen saturation values ​​on the screen, allowing hospital doctors to achieve a three-dimensional assessment of the patient's condition based on "image + data," far superior to a single video image.

[0279] Third, virtual simulation scenarios can be adapted to in-hospital diagnosis and treatment scenarios, achieving precise cross-scenario connection.

[0280] The original surveillance video is presented from a monitoring perspective, which is completely different from the perspective of doctors in the hospital (such as the eye-level view next to the bed or the operating table). Doctors need to "switch perspectives" in their minds to match the patient's condition in the video with the hospital's emergency preparations. This process increases the cost of judgment. At the same time, the movement and bumps of the ambulance will cause the video to shake. Watching it for a long time can easily cause visual fatigue for doctors and affect the accuracy of their judgment.

[0281] The virtual simulation scene adjusts its presentation perspective according to the needs of in-hospital diagnosis and treatment, simulating the familiar diagnostic and treatment perspective of doctors. It also performs anti-shake processing and image stabilization to ensure that the doctor's visual experience closely matches the in-hospital emergency scene, achieving the effect of "seeing the simulation scene is like the patient has already arrived at the hospital." Furthermore, the virtual simulation scene supports image zooming and keyframe playback, allowing doctors to repeatedly review key points of the patient's condition (such as the moment of loss of consciousness or the patient's reaction after medication). In contrast, fast-forwarding and playback of original surveillance videos are relatively cumbersome and cannot meet the needs of refined analysis within the hospital.

[0282] Fourth, virtual simulation scenarios can solve privacy protection issues at the root, while the desensitization processing of original videos has inherent defects.

[0283] Patient resuscitation videos are highly sensitive medical privacy data. Directly transmitting raw surveillance videos, even with facial blurring, can still identify individuals through their physical characteristics, clothing, and personal belongings, posing a high risk of privacy breaches. Furthermore, the desensitization of raw videos is often done "after the fact," and in emergency pre-hospital care, there is simply no time to perform comprehensive privacy desensitization on the videos, which can easily lead to medical privacy disputes.

[0284] The generation process of virtual simulation scenarios is itself a privacy desensitization process: it's not a simple editing of the original video, but rather the transformation of the patient's core diagnostic and treatment characteristics into a virtual avatar through AI modeling. This completely removes the patient's personal identity information (facial, physical features, etc.), retaining only vital signs and operational information related to the condition, thus fundamentally avoiding the risk of privacy leaks. Furthermore, the data volume of virtual simulation scenarios is far smaller than that of the original video, making encrypted transmission and access control simpler. It also adapts to the mobile network environment of ambulances, avoiding delays caused by large file transfers.

[0285] Example 2

[0286] This embodiment provides an intelligent, multi-functional emergency vehicle command system, including:

[0287] The dispatch module is configured to determine the location of the incident and the demand for supplies and resources based on the received emergency call events, and to screen dispatchable vehicles. The dispatchable vehicles are comprehensively ranked from the dimensions of theoretical travel time, resource matching degree, regional load balance degree, road condition prediction delay time and vehicle priority to determine the optimal rescue vehicle.

[0288] The planning module is configured to determine the optimal driving route based on the location of the optimal rescue vehicle, the location of the incident, and the constraints and traffic weights of the route, so as to control the rescue vehicle to travel to the incident location.

[0289] The recognition module is configured to identify the actions of medical staff, rescue supplies and their corresponding status based on video data from the rescue scene, and to extract the voice commands of medical staff based on audio data.

[0290] The mapping module is configured to use timestamps as indexes to align video recognition results, audio extraction results, and patient vital signs data within the same time window. After binding voice commands with operation actions and material status, it maps the limb coordinates of medical staff, material status, and patient vital signs data to the virtual rescue scene and visualizes them.

[0291] The push module is configured to push and display the generated virtual rescue scenario and receive guidance data generated based on the virtual rescue scenario.

[0292] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0293] In further embodiments, the following is also provided:

[0294] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0295] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0296] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0297] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0298] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0299] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0300] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0301] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0302] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0303] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0304] It should be noted that all data acquisition is conducted in accordance with laws and regulations and with user consent, and the data is used legally.

[0305] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A smart, multi-functional emergency vehicle emergency command method, characterized in that, include: Based on the received emergency call events, determine the incident location and material resource needs, screen available vehicles, and comprehensively rank available vehicles from the dimensions of theoretical travel time, resource matching degree, regional load balance, predicted road condition delay time and vehicle priority to determine the optimal rescue vehicle. Based on the location of the optimal rescue vehicle, the location of the incident, and the constraints and traffic weights of the passage route, the optimal driving route is determined to control the rescue vehicle's movement to the incident location. Based on video data from the rescue scene, identify the actions of medical staff, rescue supplies and their corresponding status, and extract the voice commands of medical staff based on audio data; Using timestamps as indexes, the video recognition results, audio extraction results, and patient vital signs data within the same time window are aligned. After binding voice commands with operational actions and material status, the limb coordinates of medical staff, material status, and patient vital signs data are mapped to the virtual rescue scene and visualized. Using timestamps as indexes, a time alignment buffer is established to align video recognition results, audio analysis results, and patient vital signs data within the same time window, forming continuous multimodal data frames. Build video The audio semantic mapping rule base defines the correspondence between voice commands, operation actions, and material status. This is used to perform correlation verification on multimodal data frames, bind voice commands with operation actions and material status, and form operation nodes with confidence. Based on the coordinates of key points of the limbs of medical staff, inverse kinematics is used to drive the corresponding virtual medical staff model to achieve synchronous mapping of posture and position. Update the location and status of rescue equipment, medicines, and consumables in the virtual rescue scenario based on the status of supplies; The patient's vital signs data are bound to the virtual patient model and visualized through numerical floating annotations, waveforms, and changes in the model's appearance. At the same time, text annotations are added to the operation nodes, with the annotations located in non-critical areas of the simulation screen. The generated virtual rescue scenario will be pushed out and displayed, and guidance data generated based on the virtual rescue scenario will be received.

2. The intelligent multi-functional emergency vehicle emergency command method as described in claim 1, characterized in that, Extract key points of the limbs of medical staff and patients, and draw skeletal diagrams based on the connection relationship of human bones; It also calculates skeletal motion features, including: calculating the difference in coordinates of key points of limbs in adjacent frames to represent the direction and speed of limb movement; calculating the joint angles of skeletal connections to represent the bending state of limbs; calculating the distance between key points of limbs of medical staff and patients to represent the interaction relationship between limbs; and identifying the operation actions of medical staff based on key points of limbs and skeletal motion features.

3. The intelligent multi-functional emergency vehicle emergency command method as described in claim 1, characterized in that, Extract key points of state characteristics of each rescued material, and determine the state of the material by calculating quantifiable geometric features between key points of state characteristics; including: calculating the distance between any two key points of state characteristics and the ratio of the two distances; calculating the included angle formed by three key points of state characteristics; and calculating the area of ​​the polygon enclosed by multiple key points of state characteristics.

4. The intelligent multi-functional emergency vehicle emergency command method as described in claim 1, characterized in that, The conditions for dispatchable vehicles are as follows: the rescue vehicle is in standby or driving status, with no malfunctions or maintenance records; the quantity of supplies in stock is greater than or equal to the quantity of supplies required; and the area of ​​responsibility of the rescue vehicle is in the same area or adjacent to the incident location. The ambulance currently has no unfinished emergency rescue missions, and the number of available vehicles within its area of ​​responsibility is greater than or equal to 1.

5. The intelligent multi-functional emergency vehicle emergency command method as described in claim 1, characterized in that, The process of determining the optimal driving route includes: The constraints on the travel route include: route priority, traffic restrictions, and time constraints. Route priority includes: emergency lanes > main roads within the hospital > general lanes > pedestrian lanes. Traffic restrictions include: areas where vehicles are prohibited from passing, the direction of travel on one-way lanes, and temporary control lanes. Time constraints include: setting the maximum allowed travel time for the route based on the emergency call level. The passage weight includes: path length score, road condition score, and lane priority score. The overall passage weight is the sum of the path length score, road condition score, and lane priority score. The path with the highest score is the optimal travel path.

6. A smart multi-functional emergency vehicle command system, employing the smart multi-functional emergency vehicle command method as described in any one of claims 1-5, characterized in that, include: The dispatch module is configured to determine the location of the incident and the demand for supplies and resources based on the received emergency call events, and to screen dispatchable vehicles. The dispatchable vehicles are comprehensively ranked from the dimensions of theoretical travel time, resource matching degree, regional load balance degree, road condition prediction delay time and vehicle priority to determine the optimal rescue vehicle. The planning module is configured to determine the optimal driving route based on the location of the optimal rescue vehicle, the location of the incident, and the constraints and traffic weights of the passage route, so as to control the rescue vehicle to travel to the incident location. The recognition module is configured to identify the actions of medical staff, rescue supplies and their corresponding status based on video data from the rescue scene, and to extract the voice commands of medical staff based on audio data. The mapping module is configured to use timestamps as indexes to align video recognition results, audio extraction results, and patient vital signs data within the same time window. After binding voice commands with operation actions and material status, it maps the limb coordinates of medical staff, material status, and patient vital signs data to the virtual rescue scene and visualizes them. The push module is configured to push and display the generated virtual rescue scenario and receive guidance data generated based on the virtual rescue scenario.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-5.

9. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-5.

Images