Regional health emergency command voice question and answer interaction system and working method thereof

Abstract

The present application relates to the technical field of data processing, in particular to a regional health emergency command voice question and answer interaction system and working method, which analyzes and converts input original audio waveform into machine recognizable standardized text, generates structured dispatch instructions, respectively searches for relevant event information, and obtains unified and structured data objects through multi-source data fusion packaging; real-time analysis is performed on emergency information in the event information, multi-dimensional risk events and corresponding features are obtained, an encapsulated comprehensive risk assessment report is output and potential risk warning is pushed; optimal decision is made, an optimal dispatch scheme is obtained, control instructions are distributed and real-time data is collected to iteratively optimize the dispatch strategy. Through the natural interaction mode driven by voice, the present application can quickly issue instructions, shorten the response time, break the information silos, realize real-time integration and visualization of emergency resources, provide comprehensive information support for decision making, and realize automatic resource scheduling and path planning through multi-agent collaboration.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to a medical emergency command system. Background Technology

[0002] In the field of emergency medical command, an efficient and intelligent command system is crucial for timely rescue and saving lives. Existing emergency command systems mostly employ traditional graphical user interfaces (GUIs), requiring commanders to manually allocate resources using a mouse, keyboard, or touchscreen. However, these systems suffer from several significant problems: First, the problem of information silos is serious. Information on emergency resources such as ambulances, hospitals, and police forces is often stored in different systems and lacks a unified and real-time visualization. This makes it difficult for commanders to fully and timely grasp the overall situation on the scene when dealing with emergencies, and to quickly obtain all the necessary resource information, thus affecting the accuracy and timeliness of decision-making.

[0003] Secondly, the interaction efficiency is low. In critical emergency rescue situations, commanders need to manually query, select and issue instructions. The operation steps are cumbersome, the response delay is obvious, and it is easy to miss the best rescue time, which poses a serious threat to the patient's life safety.

[0004] Furthermore, insufficient collaboration capabilities exist. Most existing systems currently lack effective intelligent collaboration mechanisms, with resource scheduling and route planning relying primarily on human experience. This makes it difficult to achieve efficient coordination and collaboration in emergency rescue scenarios involving multiple departments, hindering the full utilization of each department's strengths and reducing overall emergency rescue efficiency.

[0005] Existing technologies disclose an ambulance-mounted remote intelligent emergency rescue system, which can acquire distress signal location and hospital emergency center location, but its device is installed inside the ambulance and cannot achieve large-scale multi-vehicle dispatching. Existing technologies also disclose an information network platform response system, which is installed in a pre-hospital emergency medical center to serve a single hospital and cannot serve large-scale multi-vehicle dispatching. Existing technologies also disclose a pre-hospital emergency command system including an emergency command center, vehicle positioning module, vital sign collection and transmission module, video monitoring module, and help terminal, but it also cannot serve large-scale multi-vehicle dispatching. Summary of the Invention

[0006] To address the above problems, the purpose of this invention is to provide a regional health emergency command voice question-and-answer interactive system; The present invention also aims to provide a working method for a regional health emergency command voice question-and-answer interactive system.

[0007] A regional emergency medical command voice-based question-and-answer interactive system, including: A speech parsing agent (21) is used to parse the raw audio waveform of the input and convert it into machine-readable normalized text; The instruction recognition agent (22) is connected to the speech parsing agent (21) to perform semantic understanding on the standardized text, recognize the user's intent, and generate structured scheduling instructions; The data extraction agent (23) is connected to the instruction recognition agent (22), and according to the structured scheduling instruction, it searches for relevant event information and obtains a unified, structured data object through multi-source data fusion and encapsulation. The risk warning intelligent agent (24) is connected to the data extraction intelligent agent (23) to perform real-time analysis of the emergency information in the event information, obtain multi-dimensional risk events and corresponding characteristics, output a packaged comprehensive risk assessment report through the risk assessment model and push potential risk warnings; The resource scheduling agent (25) is connected to the data extraction agent (23) and the risk warning agent (24). Based on the event information and the potential risk warning, it makes the optimal decision, obtains the optimal scheduling scheme, distributes control instructions, and collects real-time data to iteratively optimize the scheduling strategy.

[0008] The regional emergency medical command voice question-and-answer interaction system of the present invention includes, the voice parsing intelligent agent (21) comprising, The feature extraction unit (211) extracts features from the input raw audio waveform to obtain an acoustic feature sequence; The language recognition unit (212a) loads a language recognition model and predicts the language label of the original audio waveform or uses existing language category labeling. The acoustic event detection unit (212b) loads an acoustic event detection model to predict specific event labels present in the original audio waveform; The inverse text normalization unit (212c) provides control rules to perform inverse text normalization on the final recognized text to generate readable normalized text; The language recognition unit (212a), acoustic event detection unit (212b), and inverse text normalization unit (212c) perform parallel preprocessing during feature extraction; The task embedder (213) receives specific event labels and control rules output by the language recognition unit (212a), acoustic event detection unit (212b), and inverse text normalization unit (212c), and converts them into task embedding vectors that match the dimensions of the acoustic feature sequence. The self-attention network encoder (214) receives the joint feature sequence obtained by fusing the acoustic feature sequence with the task embedding vector, and outputs a deep encoded feature sequence containing context information; Connect the temporal classification decoding layer (215), receive the deep encoded feature sequence, and generate the main text sequence for speech recognition; The loss calculation unit (216) calculates the classification loss of language recognition loss based on the language label predicted by the language recognition unit (212a) and the real language label, and also obtains the connection time sequence classification loss based on the difference between the main text sequence and the real transcribed text. The model trainer (218) trains the model by minimizing the weighted sum of the language recognition loss and the connection temporal classification loss; The result output device (219) outputs the main text sequence and synchronously outputs or associates the language label or language category label, the specific event label, and the normalized text output by the language recognition unit. The original audio waveform is acquired through a voice input / output module (11).

[0009] The regional emergency medical command voice question-and-answer interaction system of the present invention includes, the instruction recognition intelligent agent (22) comprising, The semantic segmentation unit (221) is connected to the speech parsing agent (21), and is constructed based on the conditional random field model. It receives the normalized text, performs word segmentation and basic semantic unit division, and obtains the segmented semantic sequence. The entity recognition unit (222a) calls a pre-trained language model to identify and extract key entity objects from the semantic sequence; The action recognition unit (222b) identifies the core actions expressed in the semantic sequence based on the pre-trained language model; The relationship identification unit (222c) uses a graph neural network-based relationship extraction model to analyze and extract the semantic relationships or logical associations between the identified entity objects. The attribute recognition unit (222d) is connected to the entity recognition unit (222a), the action recognition unit (222b), and the relationship recognition unit (222c). It adopts a conditional random field model to identify attributes, states, or modifiers related to entity objects or core actions and outputs attribute recognition results. The instruction conversion unit (223) integrates, aligns and combines the obtained entity objects, core actions, semantic relationships or logical associations based on the logical architecture and data structure to generate the structured scheduling instructions.

[0010] The regional health emergency command voice question-and-answer interaction system of the present invention includes, the data extraction intelligent agent (23) comprising, The database pattern unit (231) is connected to the instruction recognition agent (22), receives the structured scheduling instruction, and maps and combines the entity objects, core actions, semantic relationships or logical associations contained in the structured scheduling into one or more SQL query statements that conform to the database syntax specifications according to the preset database pattern. Parallel processing engines, including: The ambulance search unit (232a) extracts a set of candidate vehicles from the real-time updated ambulance status database based on the location, status, and resource requirements contained in the SQL query statement. It uses a spatial index structure to perform fast nearest neighbor search and sorting based on the Euclidean distance or path planning distance between the current vehicle location and the target location to lock the optimal dispatch vehicle. The hospital resource search unit (232b) retrieves a set of candidate hospitals that meet the conditions from the hospital resource database based on the resource requirements contained in the SQL query statement. Using a spatial index structure, it performs spatial proximity calculations by combining the hospital location with the incident location or the patient's current location to select candidate hospitals. The location lookup unit (232c) calls a geocoding service to convert the text location into latitude and longitude coordinates based on the location description, distance judgment or regional inclusion relationship contained in the SQL query statement, and calculates the great circle distance between two points on the Earth's surface or determines the spatial relationship between the text location and a predetermined area. The indicator definition and calculation unit (232d) performs online aggregation, statistics and calculation based on the business indicator model to generate dynamic indicator data reflecting the regional emergency response situation. The multi-source data fusion and formatted output unit (233) aligns, associates and merges the optimal dispatch vehicle, alternative hospital, spatial relationship and dynamic indicator data, and encapsulates them into a unified and structured data object according to the agreed data interface specifications.

[0011] The regional health emergency command voice question-and-answer interactive system of the present invention includes, the risk warning intelligent agent (24) comprising, Risk factor polling unit (241) periodically polls to obtain real-time dynamic data related to risk. The real-time dynamic data includes real-time dynamic data of all ambulances, real-time occupancy rate of emergency resources of each hospital in the region, and creation events and status of all emergency events to be responded to. A multi-dimensional parallel risk identification unit identifies risk events and their corresponding characteristics based on the real-time dynamic data, including: The path overlap risk identification unit (242a) extracts the trajectory sequence of each ambulance that is performing a task or planning a route for all ambulances. It uses a path similarity calculation model to compare the similarity of the ambulance trajectories pairwise. When the distance between the trajectories of two ambulances is lower than a preset threshold and the task time windows of the two ambulances overlap, it is determined to be a risk of multiple ambulance routes overlapping. The coverage hole risk identification unit (242b) divides the command area into geographic grids. Based on the coverage model, it regards the currently available ambulances and the dynamic response range of the ambulances as the coverage set. If there are some geographic grids that cannot be covered by any available ambulances within the target response time, they are judged as coverage hole risks due to insufficient local ambulances. The resource saturation risk identification unit (242c) extracts the historical and real-time load data of the key emergency resources of each networked hospital, uses a resource load prediction model to predict the resource occupancy rate in the short term, and marks the networked hospital as having a risk of hospital emergency resource saturation for each networked hospital. The response timeout risk identification unit (242d) polls all emergency events obtained from the alarm receiving platform with the status of "waiting for dispatch", calculates the waiting time of each emergency event, and if the waiting market exceeds the preset first response threshold since its creation and still has not been associated with dispatching an ambulance, it immediately marks the emergency event as having a response delay risk of calling for help but no ambulance being dispatched. The comprehensive risk assessment and output unit (243) aggregates the risk events and corresponding features identified by the multi-dimensional parallel risk identification unit to form a risk feature vector at the current moment. The risk feature vector is input into a preset risk assessment model to perform weighted fusion and level assessment of the occurrence probability and potential impact of various risks. The unit outputs a structured comprehensive risk assessment report. The comprehensive risk assessment report lists the identified risks, their levels, confidence levels, and suggested areas of concern. The comprehensive risk assessment report is then encapsulated and output according to the prescribed data interface format.

[0012] The regional emergency medical command voice question-and-answer interactive system of the present invention includes, the resource scheduling intelligent agent (25) comprising, The scheduling request receiving and multi-source information synchronization acquisition unit (251) synchronously acquires multi-source real-time information according to the elements contained in the structured scheduling instruction. The multi-source real-time information includes multi-ambulance positioning and status information, real-time traffic condition data, and hospital emergency resource saturation information. The parallel multi-constraint candidate set generation and modeling unit performs parallel calculations based on the structured scheduling instructions and the multi-source real-time information to construct constraint models and candidate sets for optimization decisions, including: The available ambulance screening and cost modeling unit (252a) can screen out a set of candidate ambulances that meet the requirements of emergency medical missions from all ambulances on standby and those whose missions are about to end, and calculate the estimated response time for each candidate ambulance in the set of candidate ambulances. The receiving hospital screening and load modeling unit (252b) selects a set of candidate hospitals with the current ability to receive the corresponding injured and sick from hospitals in the region based on the hospital emergency resource saturation information, and calculates the estimated remaining reception capacity, specialty matching degree and subsequent load pressure for each candidate hospital in the candidate hospital set. The multi-objective joint optimization and scheduling scheme generation unit (253) integrates the candidate ambulance set, candidate hospital set, and cost and constraint model into the multi-objective optimization model to obtain a comprehensive optimal scheduling scheme. The scheduling scheme includes the specific ambulance to perform the task, the target hospital to which the injured person will be sent, and the specific driving route planned for the selected ambulance. The dispatch instruction execution and collaborative intervention unit (254) converts the generated dispatch scheme into executable control instructions and distributes the information to the on-board terminal of the specific ambulance, the emergency department of the target hospital, and the traffic command system of the specific driving route; The effect tracking and strategy iteration optimization unit (255) collects the actual execution data of this scheduling task, including the actual response time, the actual path travel time, and the actual reception status of the hospital. The actual execution data is input into the machine learning / reinforcement learning component for iterative optimization of the scheduling strategy, and the internal parameters and weights of the multi-objective optimization model are updated and optimized.

[0013] The working method of the regional emergency medical command voice question-and-answer interactive system includes the following steps: Step 1: Parse the input raw audio waveform and convert it into machine-readable normalized text; Step 2: Perform semantic understanding on the standardized text, identify the user's intent, and generate structured scheduling instructions; Step 3: According to the structured scheduling instructions, search for relevant event information respectively, and obtain a unified, structured data object through multi-source data fusion and encapsulation; Step 4: Perform real-time analysis on the emergency information in the event information to obtain multi-dimensional risk events and their corresponding characteristics. Output a packaged comprehensive risk assessment report through the risk assessment model and push potential risk alarms. Step 5: Make optimal decisions based on the event information and the potential risk alarms to obtain the optimal scheduling scheme, distribute control instructions, and collect real-time data to iteratively optimize the scheduling strategy.

[0014] The working method of the present invention, wherein step 1 includes, The acoustic feature sequence is obtained by extracting features from the input raw audio waveform; During feature extraction, the following preprocessing steps are performed in parallel: loading a language recognition model to predict the language label of the original audio waveform or using existing language category annotations; loading an acoustic event detection model to predict specific event labels present in the original audio waveform; and providing control rules to perform inverse text normalization on the final recognition output text to generate readable normalized text. Receive the specific event labels and control rules, and convert them into task embedding vectors that match the dimensions of the acoustic feature sequence; Receive the joint feature sequence obtained by fusing the acoustic feature sequence and the task embedding vector, and output a deep encoded feature sequence containing context information; Receive the deep encoded feature sequence and generate the main text sequence for speech recognition; The classification loss for language recognition is calculated based on the predicted language labels and the actual language labels, and the connection time-series classification loss is also calculated based on the difference between the main text sequence and the actual transcribed text. The model is trained by minimizing the weighted sum of the language recognition loss and the connection temporal classification loss; The main text sequence is output, and the language label or language category annotation, the specific event label, and the standardized text output by the language recognition unit are output synchronously or associated with it.

[0015] The working method of the present invention, wherein step 2 includes, Based on the conditional random field model, the normalized text is received and segmented into words and basic semantic units to obtain the segmented semantic sequence; The process involves calling a pre-trained language model to identify and extract key entity objects from the semantic sequence; identifying the core actions expressed in the semantic sequence based on the pre-trained language model; analyzing and extracting the semantic relationships or logical connections between the identified entity objects using a graph neural network-based relation extraction model; and using a conditional random field model to identify attributes, states, or modifiers related to the entity objects or core actions and outputting attribute identification results. Based on the logical architecture and data structure, the obtained entity objects, core actions, semantic relationships or logical associations are integrated, aligned and combined to generate the structured scheduling instructions.

[0016] The working method of the present invention, step 3 includes, Upon receiving the structured scheduling instruction, based on a preset database schema, the entity objects, core actions, semantic relationships, or logical associations contained in the structured scheduling instruction are mapped and combined into one or more SQL query statements that conform to the database syntax specifications. Based on the location, status, and resource requirements contained in the SQL query, a candidate vehicle set is extracted from the real-time updated ambulance status database. A spatial index structure is used to perform a fast nearest neighbor search and sorting based on the Euclidean distance or path planning distance between the vehicle's current location and the target location to pinpoint the optimal dispatch vehicle. Based on the resource requirements contained in the SQL query, a candidate hospital set meeting the criteria is retrieved from the hospital resource database. A spatial index structure is used, combined with the hospital's location and the incident location or the patient's current location, to perform spatial proximity calculations and select candidate hospitals. Based on the location description, distance judgment, or regional inclusion relationship contained in the SQL query, a geocoding service is invoked to convert the text location into latitude and longitude coordinates, calculate the great circle distance between two points on the Earth's surface, or determine the spatial relationship between the text location and a predetermined area. Online aggregation, statistics, and calculations are performed according to the business indicator model to generate dynamic indicator data reflecting the regional emergency response situation. The optimal dispatch vehicle, alternative hospitals, spatial relationships, and dynamic indicator data are aligned, correlated, and integrated, and encapsulated into a unified, structured data object according to the agreed data interface specifications.

[0017] Beneficial effects: This invention enables commanders to issue instructions quickly through a voice-driven natural interaction method, shortening response time, breaking down information silos, realizing real-time integration and visualization of emergency resources, providing comprehensive information support for decision-making, and achieving automatic resource scheduling and path planning through a multi-agent collaborative mechanism, forming a closed-loop command system, improving overall emergency response efficiency, and solving the problem of low efficiency in cross-hospital emergency command interaction in large areas. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the structure of the regional health emergency command voice question-and-answer interactive system of the present invention; Figure 2 This is a structural block diagram of a speech parsing intelligent agent according to a specific embodiment of the present invention; Figure 3 This is a structural block diagram of an instruction recognition intelligent agent according to a specific embodiment of the present invention; Figure 4 This is a structural block diagram of a data extraction intelligent agent according to a specific embodiment of the present invention; Figure 5 This is a structural block diagram of a risk warning intelligent agent according to a specific embodiment of the present invention; Figure 6This is a structural block diagram of a resource scheduling intelligent agent according to a specific embodiment of the present invention; Figure 7 This is a flowchart illustrating the working method of the regional health emergency command voice question-and-answer interactive system of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0022] Reference Figures 1 to 6 A regional emergency medical command voice question-and-answer interactive system, including: The speech parsing agent 21 is used to parse the input raw audio waveform and convert it into machine-readable normalized text; The instruction recognition agent 22 is connected to the speech parsing agent 21 to perform semantic understanding on the standardized text, recognize the user's intent, and generate structured scheduling instructions. Data extraction agent 23 is connected to instruction recognition agent 22. According to the structured scheduling instructions, it searches for relevant event information and obtains a unified, structured data object through multi-source data fusion and encapsulation. Risk warning intelligent agent 24 is connected to data extraction intelligent agent 23 to perform real-time analysis of emergency information in event information, obtain multi-dimensional risk events and corresponding characteristics, output a packaged comprehensive risk assessment report through risk assessment model and push potential risk alarms. Resource scheduling agent 25 is connected to data extraction agent 23 and risk warning agent 24. Based on event information and potential risk warnings, it makes optimal decisions, obtains the optimal scheduling scheme, distributes control commands, and collects real-time data to iteratively optimize the scheduling strategy.

[0023] To address the limitations of existing technologies in command efficiency due to their lack of multi-resource integration visualization, voice interaction, and multi-agent automatic collaboration capabilities, this invention adds multiple collaborative intelligent agent innovation modules to the basic modules. These modules are responsible for handling core business logic and decision-making. Employing a closed-loop design concept of perception-decision-command-feedback, it constructs an intelligent command device integrating a multimodal situational awareness screen, a voice interaction system, and a multi-agent collaborative framework. Using an architecture combining basic support with an intelligent core, the basic modules act like senses and limbs, handling input / output, data storage, computation, and real-time perception. The innovation modules, on the other hand, act as the brain and central nervous system, transforming raw information into intelligent analysis, recognition, early warning, and dispatch decisions. This enables efficient and intelligent voice question-and-answer and multi-directional interaction functions in regional health emergency command scenarios.

[0024] The speech parsing agent 21 is responsible for converting received voice commands into machine-readable text or structured data. The command recognition agent 22 performs semantic understanding on the parsed text, recognizing the user's intent and specific operational commands. The data extraction agent 23 quickly and accurately retrieves and extracts relevant information from backend data based on the recognized commands. The risk warning agent 24 performs real-time analysis of emergency-related data (such as patient condition, resource status, traffic, etc.), predicts and issues early warnings of potential risks. The resource scheduling agent 25 makes optimal decisions based on event circumstances and warning information, intelligently allocating emergency vehicles, personnel, equipment, and other resources.

[0025] This invention enables commanders to issue instructions quickly through a voice-driven natural interaction method, shortening response time, breaking down information silos, and achieving real-time integration and visualization of emergency resources. It provides comprehensive information support for decision-making, and through a multi-agent collaborative mechanism, it achieves automatic resource scheduling and path planning, forming a closed-loop command system, improving overall emergency response efficiency, and solving the problem of low efficiency in cross-hospital emergency command and interaction in large areas.

[0026] The basic module of this invention provides the necessary computational, data, interaction, and perception support for the intelligent agent module, including: The voice input / output module 11 is responsible for voice acquisition and synthesized broadcast through a speaker, enabling natural voice interaction with the commander. Voice acquisition uses a directional microphone array to capture the commander's voice, which is then processed for noise reduction before being sent to the recognition engine. Command parsing converts the voice into text and uses a deep learning-based Natural Language Understanding (NLU) model to extract command elements (such as action: dispatch, resource type: police, target: ambulance, Shanghai AXYZ). Voice interaction determines the commander's intent and parameters based on preset command templates (such as dispatching [resource] to execute [action] for [target]).

[0027] Data server module 12 is used to store and manage business data. It is the system's data warehouse. Business data may include patient information, ambulance location information, hospital resource information, and geographic information. The computing power service module 13 provides powerful computing capabilities for controlling the operation and optimization of various intelligent agent models, such as natural language processing and risk prediction models. Display module 14 provides a large-screen visualization of the command panorama, early warning information, resource distribution, and dispatch plans, enabling multi-directional interactive presentation. The multimodal situational awareness screen employs layered visualization technology, supporting the overlay and switching of different layers. The electronic map integrates GIS data at its core. Resource layers use different icons and colors to distinguish resource types, while the dynamic data layer displays real-time parameters (such as vehicle speed and hospital bed occupancy rate) in the form of labels. To maintain visual clarity, the icon design follows the principle of simplicity and easy identification, and key resources are highlighted (e.g., with a flashing effect).

[0028] The resource location data capture module 15 uses technologies such as GPS and the Internet of Things to track and acquire the dynamic location and status information of emergency resources such as ambulances and medical equipment in real time.

[0029] Reference Figure 2 The voice analysis agent 21 may include, The feature extraction unit 211 extracts features from the input raw audio waveform to obtain an acoustic feature sequence; The language recognition unit 212a loads a language recognition model to predict the language label of the original audio waveform or uses existing language category labels. Acoustic event detection unit 212b loads an acoustic event detection model to predict specific event labels present in the original audio waveform; The inverse text normalization unit 212c provides control rules to perform inverse text normalization on the final recognized text to generate readable normalized text; The speech recognition unit 212a, the acoustic event detection unit 212b, and the inverse text normalization unit 212c perform parallel preprocessing during the feature extraction process; The task embedder 213 receives specific event labels and control rules output by the language recognition unit 212a, the acoustic event detection unit 212b, and the inverse text normalization unit 212c, and converts them into task embedding vectors that match the dimensions of the acoustic feature sequence. The self-attention network encoder 214 receives the joint feature sequence obtained by fusing the acoustic feature sequence and the task embedding vector, and outputs a deep encoded feature sequence containing contextual information. Connect the temporal classification decoding layer 215, receive the deep encoded feature sequence, and generate the main text sequence for speech recognition; The loss calculation unit 216 calculates the classification loss of language recognition loss based on the language label predicted by the language recognition unit 212a and the real language label, and also obtains the connection time sequence classification loss based on the difference between the main text sequence and the real transcribed text. Model trainer 218 trains the model by minimizing the weighted sum of the language recognition loss and the connection-temporal classification loss; The output device 219 outputs the main text sequence and synchronously outputs or associates the language labels or language category annotations, specific event labels, and normalized text output by the language recognition unit. The original audio waveform is acquired through a voice input / output module 11.

[0030] The speech parsing agent 21 of this invention constructs a unified speech parsing agent integrating acoustic event detection, language recognition, and text normalization control through feature extraction, multi-task information embedding, self-attention network joint encoding, and multi-loss (language recognition loss, CTC loss) joint training, thereby achieving efficient and information-rich speech content parsing.

[0031] In a specific embodiment, the execution steps of the speech parsing agent 21 are as follows: Step S211: Speech signal input and front-end feature extraction; It receives the original audio waveform input corresponding to "start speaking" and extracts features such as filter banks, Mel frequency cepstral coefficients (MFCC), or time-frequency features to convert the original waveform into a high-dimensional acoustic feature sequence, providing standardized input for subsequent processing.

[0032] Step S212: Parallel task parameter and rule preprocessing; Simultaneously with feature extraction, three independent language recognition units 212a, acoustic event detection units 212b, and inverse text normalization units 212c are activated to process different prior tasks or rules: The language recognition unit 212a loads a language recognition model to predict the language label (such as Chinese or English) of the input audio, or uses existing language category labels; the acoustic event detection unit 212b loads an acoustic event detection model to predict specific event labels (such as applause, cough, knocking) in the audio; the inverse text normalization unit 212c sets whether to perform inverse text normalization operation on the final recognized output text (e.g., convert "one hundred" to "100") according to task requirements, and provides control rules or labels.

[0033] Step S213: Multi-task information fusion and embedding; The predicted labels or control rules from the speech recognition unit 212a, the acoustic event detection unit 212b, and the inverse text normalization unit 212c are input into the task embedder 213. The task embedder 213 transforms the discrete label or rule information into a continuous vector representation that matches the acoustic feature dimension, i.e., the task embedding vector, thereby realizing the vectorized fusion of task information.

[0034] Step S214: Joint feature encoding; The acoustic feature sequence extracted in step S211 is fused with the task embedding vector generated in step S213. For example, by splicing or adding, the fused joint feature sequence is input into the self-attention network encoder 214, such as the Transformer encoder structure. The self-attention network encoder 214 captures the long-term global dependencies within the sequence through the self-attention mechanism and outputs a deep encoded feature sequence rich in contextual information.

[0035] Step S215: Multi-objective joint training and loss calculation; During the model training phase, a multi-task joint loss function is designed to simultaneously optimize multiple objectives: among which, the language recognition loss is usually a classification loss (such as cross-entropy loss), calculated based on the language prediction results of the language recognition unit 212a and the real language labels, specifically optimizing the language recognition capability.

[0036] The connection-temporal classification (CTC) loss is calculated by feeding the feature sequence output by the self-attention network encoder 214 into a CTC decoding layer, which is responsible for generating the main text sequence for speech recognition. The difference between the generated text and the actual transcribed text is calculated to obtain the connection-temporal classification loss, thereby optimizing the core speech recognition task.

[0037] The overall training objective of the model is to minimize the weighted sum of the language recognition loss and the connection-temporal classification loss, thereby achieving end-to-end joint learning.

[0038] S216: Reasoning and Output; During the model inference (use) phase, the input audio is processed through the aforementioned steps. Ultimately, the system outputs the core recognized text result (generated via CTC path decoding), and can simultaneously output or correlate the language information predicted by the language recognition unit 212a, the event information detected by the acoustic event detection unit 212b, and the rules of the inverse text normalization unit 212c will be applied to the recognized text to complete the inverse text normalization as needed, generating the final readable normalized text.

[0039] Reference Figure 3 The instruction recognition agent 22 may include, The semantic segmentation unit 221 is connected to the speech parsing agent 21. It is constructed based on the conditional random field model, receives normalized text, performs word segmentation and basic semantic unit division, and obtains the segmented semantic sequence. The entity recognition unit 222a calls a pre-trained language model to identify and extract key entity objects from the semantic sequence; The action recognition unit 222b, based on a pre-trained language model, identifies the core actions expressed in a semantic sequence; The relation recognition unit 222c uses a relation extraction model based on graph neural networks to analyze and extract the semantic or logical relationships between the identified entity objects. The attribute recognition unit 222d is connected to the entity recognition unit 222a, the action recognition unit 222b, and the relationship recognition unit 222c. It adopts a conditional random field model to identify the attributes, states, or modifiers related to entity objects or core actions and outputs the attribute recognition results. The instruction conversion unit 223 integrates, aligns, and combines the obtained entity objects, core actions, semantic relationships, or logical associations based on the logical architecture and data structure to generate structured scheduling instructions.

[0040] The instruction recognition agent 22 of this invention realizes an end-to-end instruction understanding agent through the steps of serial speech conversion, semantic segmentation, parallel multi-dimensional recognition and comprehensive conversion. It adopts a processing architecture that combines hierarchical and parallel processing to ensure the accuracy, completeness and processing efficiency of instruction recognition.

[0041] The instruction recognition agent 22 transforms unstructured voice instructions into structured, executable scheduling instructions through a multi-stage, hierarchical processing flow. In a specific embodiment, the execution steps of the instruction recognition agent 22 are as follows: Step S221: Semantic unit segmentation; After obtaining the normalized text of the text instructions output by the speech parsing agent 21, the normalized text is input into the semantic segmentation unit 221 constructed based on the conditional random field model to perform word segmentation and basic semantic unit division on the continuous text, providing structured sequence data for subsequent fine-grained recognition. Step S222: Parallel multi-dimensional semantic recognition; The segmented semantic sequences are synchronously distributed to entity recognition unit 222a, action recognition unit 222b, relation recognition unit 222c, and attribute recognition unit 222d for parallel analysis. The entity recognition unit 222a calls a model fine-tuned based on a pre-trained language model (such as BERT or RoBERTa) to identify and extract key entity objects from the semantic sequence; the action recognition unit 222b is also based on the fine-tuned pre-trained language model to identify the core actions or operational intentions expressed in the semantic sequence; the relationship recognition unit 222c uses a relation extraction model based on graph neural networks (GNN) to analyze and extract the semantic relationships or logical connections between the identified entities; the attribute recognition unit 222d uses a conditional random field (CRF) model to identify attributes, states, or modifiers related to entities or actions.

[0042] Step S223: Instruction structure integration and output; The entity, action, relationship and attribute recognition results output by the above four parallel recognition units are uniformly sent to the instruction conversion unit 223. The instruction conversion unit 223 integrates, aligns and combines all these semantic elements according to the predetermined logical architecture and data structure to generate a complete and structured scheduling instruction that can be directly understood and executed by the downstream system or device.

[0043] Reference Figure 4 The data extraction agent 23 may include, The database schema unit 231 connects to the instruction recognition agent 22, receives structured scheduling instructions, and maps and combines the entity objects, core actions, semantic relationships or logical associations contained in the structured scheduling instructions into one or more SQL query statements that conform to the database syntax specifications, according to the preset database schema. Parallel processing engines, including: The ambulance search unit 232a extracts a set of candidate vehicles from the real-time updated ambulance status database based on the location, status, and resource requirements contained in the SQL query statement. It uses a spatial index structure to perform fast nearest neighbor search and sorting based on the Euclidean distance or path planning distance between the current vehicle location and the target location to lock the optimal dispatch vehicle. The hospital resource search unit 232b retrieves a set of candidate hospitals that meet the conditions from the hospital resource database based on the resource requirements contained in the SQL query statement. Using a spatial index structure, it performs spatial proximity calculations by combining the hospital location with the incident location or the patient's current location to select candidate hospitals. The location lookup unit 232c, based on the location description, distance judgment or regional inclusion relationship contained in the SQL query statement, calls the geocoding service to convert the text location into latitude and longitude coordinates, calculates the great circle distance between two points on the Earth's surface or judges the spatial relationship between the text location and the predetermined area. The indicator definition and calculation unit 232d performs online aggregation, statistics and calculation based on the business indicator model to generate dynamic indicator data reflecting the regional emergency response situation. The multi-source data fusion and formatted output unit 233 aligns, associates, and merges the optimal dispatch vehicle, alternative hospital, spatial relationship, and dynamic indicator data, and encapsulates them into a unified, structured data object according to the agreed data interface specifications.

[0044] The data extraction agent 23 achieves rapid and accurate mapping from high-level command intentions to low-level multi-source data through a process of "command conversion - parallel retrieval / computation - fusion output". It employs a combination of spatial indexing and geographic computing technology to ensure the real-time and accurate response to spatiotemporal elements in emergency scenarios, providing core data support capabilities for the entire regional health emergency command agent.

[0045] The data extraction agent 23 efficiently and accurately converts structured scheduling instructions into executable database queries and spatial calculations, and outputs the dynamic data required for emergency command and decision-making. In a specific embodiment, the implementation steps of the data extraction agent 23 are as follows: Step S231, receiving and querying structured instructions; The system receives structured scheduling instructions from upstream intelligent agent 22, calls semantic-SQL mapping algorithm, and maps and combines entities, actions, relationships and attributes contained in the structured scheduling into one or more SQL query statements that conform to the database syntax specification, according to the preset database pattern. Step S232: Parallel multipath data retrieval and computation; The SQL query is sent to the parallel processing engine, triggering the following four unit computation paths to simultaneously acquire and integrate emergency resources and spatial information: The ambulance search unit 232a is dispatched to perform dynamic search and matching of ambulances. Based on the location, status and resource requirements contained in the SQL query statement, the candidate vehicle set is extracted from the real-time updated ambulance status database. Then, using a spatial index structure (such as KD Tree), a fast nearest neighbor search and sorting is performed based on the Euclidean distance or path planning distance between the current vehicle location and the target location to lock the optimal dispatch vehicle. The hospital resource search unit 232b performs hospital resource search and matching. Based on the resource requirements such as hospital type, specialty capabilities, and available beds in the instruction, it retrieves a set of candidate hospitals that meet the conditions from the hospital resource database. It also uses a spatial index structure (such as KD Tree) and combines the hospital location with the location of the incident or the patient's current location to perform spatial proximity calculations, and selects candidate hospitals with available resources and suitable geographical locations. The location lookup unit 232c performs geographic location parsing and spatial calculation. For instructions involving location description, distance judgment, or regional inclusion relationship, it calls the geocoding service to convert the text location into latitude and longitude coordinates. Then, it uses the Haversine formula to accurately calculate the great circle distance between two points on the Earth's surface, or uses geometric algorithms to determine the spatial relationship between points and regions (such as responsibility areas and control areas), providing core geographic information for scheduling and route planning.

[0046] The indicator definition and calculation unit 232d performs dynamic calculation of command indicators: based on predefined business indicator models, such as average response time, resource coverage, and task load balance, it extracts relevant basic data in real time, performs online aggregation, statistics and calculation, and generates key performance indicators that reflect the regional emergency response situation.

[0047] Step S233: Multi-source data fusion and formatted output; The ambulance information, alternative hospital information, spatial relationship data, and dynamic indicator data output from the four parallel paths are aligned, correlated, and integrated. According to the data interface specifications agreed upon by the downstream large-screen visualization module and decision-making module, they are encapsulated into a unified, structured data object (such as JSON or Protocol Buffer format) and output to the downstream module to drive the interactive visualization display of the large screen and assist command and decision-making.

[0048] Reference Figure 5 The risk warning intelligent agent 24 may include, Risk factor polling unit 241 periodically polls to obtain real-time dynamic data related to risks. The real-time dynamic data includes real-time dynamic data of all ambulances, real-time occupancy rate of emergency resources in various hospitals in the region, and creation events and status of all emergency events to be responded to. The multi-dimensional parallel risk identification unit identifies risk events and their corresponding characteristics based on real-time dynamic data, including: The path overlap risk identification unit 242a extracts the trajectory sequence of each ambulance that is performing a task or planning a route for all ambulances. It uses a path similarity calculation model to compare the similarity of the ambulance trajectories pairwise. When the distance between the trajectories of two ambulances is lower than a preset threshold and the task time windows of the two ambulances overlap, it is determined to be a risk of multiple ambulance routes overlapping. Coverage hole risk identification unit 242b divides the command area into geographic grids. Based on the coverage model, it considers the currently available ambulances and the dynamic response range of the ambulances as the coverage set. If there are some geographic grids that cannot be covered by any available ambulances within the target response time, they are judged as coverage hole risks due to insufficient local ambulances. The resource saturation risk identification unit 242c extracts historical and real-time load data of key emergency resources for each networked hospital, uses a resource load prediction model to predict the resource occupancy rate in the short term, and marks the networked hospital as having a risk of emergency resource saturation for each hospital. The response timeout risk identification unit 242d polls all emergency events obtained from the alarm receiving platform that are in the status of "waiting for dispatch", calculates the waiting time of each emergency event, and if the waiting time exceeds the preset first response threshold since its creation and no ambulance has been dispatched, the emergency event is immediately marked as having a response delay risk of calling for help but no ambulance being dispatched. The comprehensive risk assessment and output unit 243 aggregates the risk events and corresponding characteristics identified by the multi-dimensional parallel risk identification unit to form a risk feature vector at the current moment. The risk feature vector is then input into a preset risk assessment model to perform weighted fusion and level assessment of the probability of occurrence and potential impact of various risks. A structured comprehensive risk assessment report is then output, which lists the identified risks, their levels, confidence levels, and suggested areas of concern. The comprehensive risk assessment report is then packaged and output according to the prescribed data interface format.

[0049] The risk warning intelligent agent 24 of this invention achieves automated, multi-dimensional, and real-time perception and assessment of potential risks in regional emergency medical networks through a process of "data polling → parallel specialized calculation (path / coverage / resources / timeout) → comprehensive evaluation → visual push". It combines trajectory analysis, coverage models, time series prediction, and machine learning evaluation technologies, providing key technical support for improving the robustness and decision-making foresight of regional health emergency command systems.

[0050] The risk warning intelligent agent automatically and in real-time senses and assesses potential risks in the regional emergency medical network and outputs visual warning information to the command center screen. A specific embodiment includes the following steps: Step S241: Real-time polling of risk factors; The timed polling engine is activated to periodically (e.g., at 1-minute intervals) actively acquire real-time dynamic data related to risks from the system's data bus, vehicle terminals, hospital information systems (HIS), and emergency call platforms. This data includes, but is not limited to: the real-time location, status, tasks, and routes of all ambulances; the real-time occupancy rate of emergency resources in each hospital within the region; and the creation time and status of all pending emergency events.

[0051] Step S242: Multi-dimensional parallel risk identification; The real-time data obtained through polling is input into multiple parallel-running specialized computing models to identify specific risk patterns, including... The path overlap risk identification unit 242a extracts the trajectory sequence of all ambulances that are performing tasks or planning routes. It uses a path similarity calculation model, especially the Fréchet distance algorithm, to compare the similarity of the ambulance trajectories pairwise. When the Fréchet distance between the trajectories of two vehicles is lower than a preset threshold and their task time windows overlap, it is determined to be a risk of multiple ambulance routes overlapping, indicating that there may be resource aggregation or path conflict. The coverage hole risk identification unit 242b divides the command area into geographical grids. Based on the coverage model, especially the modeling concept of ensemble coverage problems, it considers the currently available (non-mission) ambulances and their dynamic response ranges as the "coverage set". It calculates whether, at any given time, there are any parts of the grid that cannot be covered by any available ambulances within the target response time (e.g., 8 minutes). If so, it is determined to be a coverage hole risk of "local ambulance shortage".

[0052] The resource saturation risk identification unit 242c extracts historical and real-time load data of key emergency resources (such as resuscitation beds and ventilators) for each networked hospital. Using a resource load prediction model, particularly employing the ARIMA time series prediction algorithm, it predicts the resource occupancy rate for the next short period (e.g., the next 2 hours). If the predicted value exceeds a set saturation threshold (e.g., 85%), the hospital is marked as having a "hospital emergency resource saturation" risk.

[0053] The response timeout risk identification unit 242d polls all emergency events obtained from the alarm receiving platform that are in the "pending dispatch" status, calculates the waiting time of each event, and if it exceeds the preset first response threshold (e.g., 10 minutes) since its creation and still has not been associated with an ambulance dispatch, then the event is immediately marked as having a response delay risk of "calling for help for more than 10 minutes without dispatch".

[0054] Step S243, Comprehensive Risk Assessment and Output; The comprehensive risk assessment and output unit 243 aggregates all risk events and their characteristics (including type, location, involved parties, and severity quantification) identified in parallel step S242 to form a risk feature vector at the current moment. This risk feature vector is then input into a preset risk assessment model. This risk assessment model is preferably a decision tree or ensemble learning model based on machine learning, used to weight and fuse the probability of occurrence and potential impact of various risks and to assign a level of risk. Finally, the risk assessment model outputs a structured comprehensive risk assessment report, clearly listing the identified risks, their levels, confidence levels, and recommended areas of concern.

[0055] Finally, risk warning information is pushed out. The structured comprehensive risk assessment report is packaged according to the data interface format specified by the downstream large screen interaction module and pushed to the command center large screen in real time and proactively through the system's internal message queue. This triggers visual components such as maps, panels, and lists to highlight, flash, change color, or pop-up alarms, thus completing a complete closed loop from risk perception to warning presentation.

[0056] The Risk Warning Intelligent Agent 24 enables automatic scanning and quantitative assessment of risks across the entire chain of regional emergency medical services networks. Its core technology integrates trajectory similarity calculation, set coverage modeling, time series prediction, and multi-factor decision-making models, enabling it to proactively identify vulnerabilities in system operation and provide crucial risk avoidance basis for command and decision-making.

[0057] Reference Figure 6 The resource scheduling agent 25 may include, The scheduling request receiving and multi-source information synchronization acquisition unit 251 synchronously acquires multi-source real-time information based on the elements contained in the structured scheduling instructions. The multi-source real-time information includes the location and status information of multiple ambulances, real-time traffic data, and hospital emergency resource saturation information. The parallel multi-constraint candidate set generation and modeling unit performs parallel computations based on structured scheduling instructions and multi-source real-time information to construct constraint models and candidate sets for optimization decisions, including: Available ambulance screening and cost modeling unit 252a can screen out a set of candidate ambulances that meet the requirements of emergency rescue missions from all ambulances on standby and those whose missions are about to end, and calculate the estimated response time for each candidate ambulance in the candidate ambulance set. The receiving hospital screening and load modeling unit 252b, based on the saturation information of hospital emergency resources, selects a set of candidate hospitals in the region that currently have the ability to receive the corresponding injured and sick, and calculates the estimated remaining reception capacity, specialty matching degree and subsequent load pressure for each candidate hospital in the candidate hospital set; The multi-objective joint optimization and scheduling scheme generation unit 253 integrates the candidate ambulance set, the candidate hospital set, and the cost and constraint model into the multi-objective optimization model to obtain a comprehensive optimal scheduling scheme. The scheduling scheme includes the specific ambulance to perform the task, the target hospital to which the injured person will be transported, and the specific driving route planned for the selected ambulance. The dispatch instruction execution and collaborative intervention unit 254 converts the generated dispatch plan into executable control instructions and distributes the information to the on-board terminal of the specific ambulance, the emergency department of the target hospital, and the traffic control system of the specific driving route; The effect tracking and strategy iteration optimization unit 255 collects the actual execution data of this scheduling task, including the actual response time, actual path travel time, and actual hospital reception status. The actual execution data is input into the machine learning / reinforcement learning component for iterative optimization of the scheduling strategy, updating and optimizing the internal parameters and weights of the multi-objective optimization model.

[0058] The resource scheduling intelligent agent 25 of this invention achieves intelligent scheduling of emergency resources in dynamic and complex environments through a closed-loop process of "request reception and information synchronization → parallel modeling → joint optimization → execution intervention → learning optimization". Its core lies in unifying the modeling of multi-source heterogeneous information such as multiple ambulance locations, real-time traffic, and hospital resources, and using multi-objective optimization and machine learning techniques to automatically generate and execute globally optimal scheduling decisions while satisfying multiple real-world constraints. It is a key intelligent hub for improving the efficiency and effectiveness of regional emergency command systems.

[0059] Resource scheduling agent 25 receives structured emergency medical instructions and, through the fusion of multi-source information and multi-objective optimization, generates and executes the optimal ambulance dispatching and task assignment scheme. A specific embodiment of resource scheduling agent 25 is as follows: Step S251: Receive structured scheduling request and synchronously acquire multi-source information; The scheduling request receiving and multi-source information synchronization acquisition unit 251 receives a structured scheduling instruction generated by the upstream instruction recognition module. This instruction includes elements such as event location, injury, and resource requirements. It synchronously acquires real-time information from three aspects from the system's data bus: "multiple ambulance positioning" and status information from the vehicle GPS, real-time "traffic conditions" data from the traffic information system, and "hospital emergency resource saturation" information from the hospital information system.

[0060] Step S252: Parallel generation and modeling of multi-constraint candidate sets; Based on the received instructions and real-time information, the following calculations are performed in parallel to construct a constraint model and candidate set for optimization decisions: Available ambulance screening and cost modeling unit 252a, based on "multi-ambulance location" information, uses the core logic of "multi-objective optimization algorithm and assignment problem algorithm" to screen a set of candidate vehicles that meet the basic requirements of emergency rescue missions (such as equipment type) from all ambulances on standby and those whose missions are about to end. It calculates the estimated response time for each candidate vehicle. The estimated response time is initially estimated by combining the "shortest path algorithm" with real-time traffic conditions and is used as a key decision cost. The receiving hospital screening and load modeling unit 252b, based on "hospital emergency resource saturation" information, uses "resource allocation and load balancing algorithms and queuing theory models" to screen a set of hospitals within the region that currently have the capacity to receive the corresponding injured and sick patients, i.e., "finding unsaturated hospitals." It calculates the estimated remaining capacity, specialty matching degree, and subsequent load pressure for each candidate hospital, serving as key constraints for system equilibrium.

[0061] Step S253: Multi-objective joint optimization and scheduling scheme generation; This is the core decision-making step; the multi-objective joint optimization and scheduling scheme generation unit 253 will... The candidate ambulance set and candidate hospital set generated in step S232, along with their respective cost and constraint models, are integrated into a unified multi-objective optimization model. This multi-objective optimization model operates within the framework of a "shortest path / optimal matching algorithm under resource constraints," aiming to simultaneously minimize the overall response time, balance the resource load of each hospital, and achieve efficient ambulance utilization, ultimately solving for a comprehensively optimal scheduling scheme. This scheduling scheme explicitly specifies: the specific ambulance performing the task, the target hospital to which the injured person will be transported, and the specific route planned for the selected ambulance.

[0062] Step S254: Execution of scheduling instructions and coordinated intervention; The dispatch instruction execution and collaborative intervention unit 254 transforms the generated dispatch plan into specific, executable control instructions and distributes them to the corresponding systems: it sends the task instructions and planned routes to the on-board terminals of designated ambulances; it sends the estimated arrival information of patients and the pre-notification of injuries to the emergency department of the target hospital; if there are severely congested sections in the planned route, it can automatically trigger the "route planning / notification to traffic police" collaborative process and send a coordination request to the traffic command system through the interface.

[0063] Step S255: Effect tracking and strategy iteration optimization; The effect tracking and strategy iteration optimization unit 255 initiates the "effect tracking" mechanism, continuously collecting actual execution data for this scheduling task, including actual response time, actual path travel time, and actual hospital reception status. This data forms a closed-loop feedback loop, which is input into the "Machine Learning / Reinforcement Learning for Iterative Optimization of Scheduling Strategies" component. By learning from historical decisions and subsequent results, the internal parameters and weights of the multi-objective optimization model in step S253 are continuously updated and optimized, thereby achieving adaptive and continuous improvement of the scheduling strategy.

[0064] Reference Figure 7 The working method of the regional health emergency command voice question-and-answer interactive system includes the following steps: Step 1: Parse the input raw audio waveform and convert it into machine-readable normalized text; Step 2: Perform semantic understanding on the standardized text, identify the user's intent, and generate structured scheduling instructions; Step 3: Based on the structured scheduling instructions, search for relevant event information and encapsulate it through multi-source data fusion to obtain a unified, structured data object; Step 4: Perform real-time analysis of emergency information in the event information to obtain multi-dimensional risk events and their corresponding characteristics. Output a packaged comprehensive risk assessment report through the risk assessment model and push potential risk alerts. Step 5: Make optimal decisions based on event information and potential risk alarms to obtain the optimal scheduling scheme, distribute control instructions, and collect real-time data to iteratively optimize the scheduling strategy.

[0065] The present invention has the following advantages: Breaking through the limitations of single-unit command: Previous inventions were often limited to the command of a single ambulance or a single emergency center, while this invention focuses on the allocation and dispatch of emergency resources at the city level, involving multiple ambulances, multiple emergency centers, and more. It also provides early warnings of potential medical congestion caused by overlapping routes of multiple ambulances, as well as localized ambulance shortages and saturation of emergency hospitals, thereby improving the emergency response capacity and management precision of the entire city.

[0066] Improved command efficiency: Through natural voice interaction, the time required for traditional manual operations has been reduced from several minutes to seconds (test data shows that the time for issuing instructions has been reduced from an average of 3-5 minutes to 10-15 seconds), greatly improving the speed of emergency response.

[0067] More comprehensive decision-making: The multimodal situational awareness screen integrates various types of resource information, helping commanders to fully grasp the situation on the ground and reduce decision-making biases caused by missing information.

[0068] Enhanced collaboration capabilities: The multi-agent automatic collaboration mechanism reduces the cost of manual coordination and improves the accuracy of resource scheduling (actual tests show that the accuracy of resource scheduling has increased to over 98%).

[0069] The system boasts strong compatibility: Through standardized interface design, the system can flexibly connect to various existing terminals (such as police communication devices and vehicle-mounted equipment), protecting existing investments.

[0070] The description and accompanying drawings provide typical embodiments of specific structures for specific implementations. Other modifications are possible based on the spirit of the invention. While the above-described invention presents preferred embodiments, these are not intended to be limiting.

[0071] For those skilled in the art, various changes and modifications will undoubtedly be apparent after reading the above description. Therefore, the appended claims should be construed as covering all changes and modifications that encompass the true intent and scope of the invention. Any and all equivalent scope and content within the scope of the claims should be considered to remain within the intent and scope of the invention.

Claims

1. A regional emergency medical command voice question-and-answer interactive system, characterized in that, include, A speech parsing agent (21) is used to parse the raw audio waveform of the input and convert it into machine-readable normalized text; The instruction recognition agent (22) is connected to the speech parsing agent (21) to perform semantic understanding on the standardized text, recognize the user's intent, and generate structured scheduling instructions; The data extraction agent (23) is connected to the instruction recognition agent (22), and according to the structured scheduling instruction, it searches for relevant event information and obtains a unified, structured data object through multi-source data fusion and encapsulation. The risk warning intelligent agent (24) is connected to the data extraction intelligent agent (23) to perform real-time analysis of the emergency information in the event information, obtain multi-dimensional risk events and corresponding characteristics, output a packaged comprehensive risk assessment report through the risk assessment model and push potential risk warnings; The resource scheduling agent (25) is connected to the data extraction agent (23) and the risk warning agent (24). Based on the event information and the potential risk warning, it makes the optimal decision, obtains the optimal scheduling scheme, distributes control instructions, and collects real-time data to iteratively optimize the scheduling strategy.

2. The regional emergency medical command voice question-and-answer interactive system according to claim 1, characterized in that, The speech parsing agent (21) includes, The feature extraction unit (211) extracts features from the input raw audio waveform to obtain an acoustic feature sequence; The language recognition unit (212a) loads a language recognition model and predicts the language label of the original audio waveform or uses existing language category labeling. The acoustic event detection unit (212b) loads an acoustic event detection model to predict specific event labels present in the original audio waveform; The inverse text normalization unit (212c) provides control rules to perform inverse text normalization on the final recognized text to generate readable normalized text; The language recognition unit (212a), acoustic event detection unit (212b), and inverse text normalization unit (212c) perform parallel preprocessing during feature extraction; The task embedder (213) receives specific event labels and control rules output by the language recognition unit (212a), acoustic event detection unit (212b), and inverse text normalization unit (212c), and converts them into task embedding vectors that match the dimensions of the acoustic feature sequence. The self-attention network encoder (214) receives the joint feature sequence obtained by fusing the acoustic feature sequence with the task embedding vector, and outputs a deep encoded feature sequence containing context information; Connect the temporal classification decoding layer (215), receive the deep encoded feature sequence, and generate the main text sequence for speech recognition; The loss calculation unit (216) calculates the classification loss of language recognition loss based on the language label predicted by the language recognition unit (212a) and the real language label, and also obtains the connection time sequence classification loss based on the difference between the main text sequence and the real transcribed text. The model trainer (218) trains the model by minimizing the weighted sum of the language recognition loss and the connection temporal classification loss; The result output device (219) outputs the main text sequence and synchronously outputs or associates the language label or language category label, the specific event label, and the normalized text output by the language recognition unit. The original audio waveform is acquired through a voice input / output module (11).

3. The regional emergency medical command voice question-and-answer interactive system according to claim 1, characterized in that, The instruction recognition agent (22) includes, The semantic segmentation unit (221) is connected to the speech parsing agent (21), and is constructed based on the conditional random field model. It receives the normalized text, performs word segmentation and basic semantic unit division, and obtains the segmented semantic sequence. The entity recognition unit (222a) calls a pre-trained language model to identify and extract key entity objects from the semantic sequence; The action recognition unit (222b) identifies the core actions expressed in the semantic sequence based on the pre-trained language model; The relationship identification unit (222c) uses a graph neural network-based relationship extraction model to analyze and extract the semantic relationships or logical associations between the identified entity objects. The attribute recognition unit (222d) is connected to the entity recognition unit (222a), the action recognition unit (222b), and the relationship recognition unit (222c). It adopts a conditional random field model to identify attributes, states, or modifiers related to entity objects or core actions and outputs attribute recognition results. The instruction conversion unit (223) integrates, aligns and combines the obtained entity objects, core actions, semantic relationships or logical associations based on the logical architecture and data structure to generate the structured scheduling instructions.

4. The regional emergency medical command voice question-and-answer interactive system according to claim 1, characterized in that, The data extraction agent (23) includes, The database pattern unit (231) is connected to the instruction recognition agent (22), receives the structured scheduling instruction, and maps and combines the entity objects, core actions, semantic relationships or logical associations contained in the structured scheduling instruction into one or more SQL query statements that conform to the database syntax specifications according to the preset database pattern. Parallel processing engines, including: The ambulance search unit (232a) extracts a set of candidate vehicles from the real-time updated ambulance status database based on the location, status, and resource requirements contained in the SQL query statement. It uses a spatial index structure to perform fast nearest neighbor search and sorting based on the Euclidean distance or path planning distance between the current vehicle location and the target location to lock the optimal dispatch vehicle. The hospital resource search unit (232b) retrieves a set of candidate hospitals that meet the conditions from the hospital resource database based on the resource requirements contained in the SQL query statement. Using a spatial index structure, it performs spatial proximity calculations by combining the hospital location with the incident location or the patient's current location to select candidate hospitals. The location lookup unit (232c) calls a geocoding service to convert the text location into latitude and longitude coordinates based on the location description, distance judgment or regional inclusion relationship contained in the SQL query statement, and calculates the great circle distance between two points on the Earth's surface or determines the spatial relationship between the text location and a predetermined area. The indicator definition and calculation unit (232d) performs online aggregation, statistics and calculation based on the business indicator model to generate dynamic indicator data reflecting the regional emergency response situation. The multi-source data fusion and formatted output unit (233) aligns, associates and merges the optimal dispatch vehicle, alternative hospital, spatial relationship and dynamic indicator data, and encapsulates them into a unified and structured data object according to the agreed data interface specifications.

5. The regional emergency medical command voice question-and-answer interactive system according to claim 1, characterized in that, The risk warning intelligent agent (24) includes, Risk factor polling unit (241) periodically polls to obtain real-time dynamic data related to risk. The real-time dynamic data includes real-time dynamic data of all ambulances, real-time occupancy rate of emergency resources of each hospital in the region, and creation events and status of all emergency events to be responded to. A multi-dimensional parallel risk identification unit identifies risk events and their corresponding characteristics based on the real-time dynamic data, including: The path overlap risk identification unit (242a) extracts the trajectory sequence of each ambulance that is performing a task or planning a route for all ambulances. It uses a path similarity calculation model to compare the similarity of the ambulance trajectories pairwise. When the distance between the trajectories of two ambulances is lower than a preset threshold and the task time windows of the two ambulances overlap, it is determined to be a risk of multiple ambulance routes overlapping. The coverage hole risk identification unit (242b) divides the command area into geographic grids. Based on the coverage model, it regards the currently available ambulances and the dynamic response range of the ambulances as the coverage set. If there are some geographic grids that cannot be covered by any available ambulances within the target response time, they are judged as coverage hole risks due to insufficient local ambulances. The resource saturation risk identification unit (242c) extracts the historical and real-time load data of the key emergency resources of each networked hospital, uses a resource load prediction model to predict the resource occupancy rate in the short term, and marks the networked hospital as having a risk of hospital emergency resource saturation for each networked hospital. The response timeout risk identification unit (242d) polls all emergency events obtained from the alarm receiving platform with the status of "waiting for dispatch", calculates the waiting time of each emergency event, and if the waiting market exceeds the preset first response threshold since its creation and still has not been associated with dispatching an ambulance, it immediately marks the emergency event as having a response delay risk of calling for help but no ambulance being dispatched. The comprehensive risk assessment and output unit (243) aggregates the risk events and corresponding features identified by the multi-dimensional parallel risk identification unit to form a risk feature vector at the current moment. The risk feature vector is input into a preset risk assessment model to perform weighted fusion and level assessment of the occurrence probability and potential impact of various risks. The unit outputs a structured comprehensive risk assessment report. The comprehensive risk assessment report lists the identified risks, their levels, confidence levels, and suggested areas of concern. The comprehensive risk assessment report is then encapsulated and output according to the prescribed data interface format.

6. The regional emergency medical command voice question-and-answer interactive system according to claim 1, characterized in that, The resource scheduling agent (25) includes, The scheduling request receiving and multi-source information synchronization acquisition unit (251) synchronously acquires multi-source real-time information according to the elements contained in the structured scheduling instruction. The multi-source real-time information includes multi-ambulance positioning and status information, real-time traffic condition data, and hospital emergency resource saturation information. The parallel multi-constraint candidate set generation and modeling unit performs parallel calculations based on the structured scheduling instructions and the multi-source real-time information to construct constraint models and candidate sets for optimization decisions, including: The available ambulance screening and cost modeling unit (252a) can screen out a set of candidate ambulances that meet the requirements of emergency medical missions from all ambulances on standby and those whose missions are about to end, and calculate the estimated response time for each candidate ambulance in the set of candidate ambulances. The receiving hospital screening and load modeling unit (252b) selects a set of candidate hospitals with the current ability to receive the corresponding injured and sick from hospitals in the region based on the hospital emergency resource saturation information, and calculates the estimated remaining reception capacity, specialty matching degree and subsequent load pressure for each candidate hospital in the candidate hospital set. The multi-objective joint optimization and scheduling scheme generation unit (253) integrates the candidate ambulance set, candidate hospital set, and cost and constraint model into the multi-objective optimization model to obtain a comprehensive optimal scheduling scheme. The scheduling scheme includes the specific ambulance to perform the task, the target hospital to which the injured person will be sent, and the specific driving route planned for the selected ambulance. The dispatch instruction execution and collaborative intervention unit (254) converts the generated dispatch scheme into executable control instructions and distributes the information to the on-board terminal of the specific ambulance, the emergency department of the target hospital, and the traffic command system of the specific driving route; The effect tracking and strategy iteration optimization unit (255) collects the actual execution data of this scheduling task, including the actual response time, the actual path travel time, and the actual reception status of the hospital. The actual execution data is input into the machine learning / reinforcement learning component for iterative optimization of the scheduling strategy, and the internal parameters and weights of the multi-objective optimization model are updated and optimized.

7. A working method for a regional emergency medical command voice question-and-answer interactive system, characterized in that, Includes the following steps: Step 1: Parse the input raw audio waveform and convert it into machine-readable normalized text; Step 2: Perform semantic understanding on the standardized text, identify the user's intent, and generate structured scheduling instructions; Step 3: According to the structured scheduling instructions, search for relevant event information respectively, and obtain a unified, structured data object through multi-source data fusion and encapsulation; Step 4: Perform real-time analysis on the emergency information in the event information to obtain multi-dimensional risk events and their corresponding characteristics. Output a packaged comprehensive risk assessment report through the risk assessment model and push potential risk alarms. Step 5: Make optimal decisions based on the event information and the potential risk alarms to obtain the optimal scheduling scheme, distribute control instructions, and collect real-time data to iteratively optimize the scheduling strategy.

8. The working method according to claim 7, characterized in that, Step 1 includes, The acoustic feature sequence is obtained by extracting features from the input raw audio waveform; During feature extraction, the following preprocessing steps are performed in parallel: loading a language recognition model to predict the language label of the original audio waveform or using existing language category annotations; loading an acoustic event detection model to predict specific event labels present in the original audio waveform; and providing control rules to perform inverse text normalization on the final recognition output text to generate readable normalized text. Receive the specific event labels and control rules, and convert them into task embedding vectors that match the dimensions of the acoustic feature sequence; Receive the joint feature sequence obtained by fusing the acoustic feature sequence and the task embedding vector, and output a deep encoded feature sequence containing context information; Receive the deep encoded feature sequence and generate the main text sequence for speech recognition; The classification loss for language recognition is calculated based on the predicted language labels and the actual language labels, and the connection time-series classification loss is also calculated based on the difference between the main text sequence and the actual transcribed text. The model is trained by minimizing the weighted sum of the language recognition loss and the connection temporal classification loss; The main text sequence is output, and the language label or language category annotation, the specific event label, and the standardized text output by the language recognition unit are output synchronously or associated with it.

9. The working method according to claim 7, characterized in that, Step 2 includes, Based on the conditional random field model, the normalized text is received and segmented into words and basic semantic units to obtain the segmented semantic sequence; The process involves calling a pre-trained language model to identify and extract key entity objects from the semantic sequence; identifying the core actions expressed in the semantic sequence based on the pre-trained language model; analyzing and extracting the semantic relationships or logical connections between the identified entity objects using a graph neural network-based relation extraction model; and using a conditional random field model to identify attributes, states, or modifiers related to the entity objects or core actions and outputting attribute identification results. Based on the logical architecture and data structure, the obtained entity objects, core actions, semantic relationships or logical associations are integrated, aligned and combined to generate the structured scheduling instructions.

10. The working method according to claim 7, characterized in that, Step 3 includes, Upon receiving the structured scheduling instruction, based on a preset database schema, the entity objects, core actions, semantic relationships, or logical associations contained in the structured scheduling instruction are mapped and combined into one or more SQL query statements that conform to the database syntax specifications. Based on the location, status, and resource requirements contained in the SQL query, a set of candidate vehicles is extracted from the real-time updated ambulance status database. A spatial index structure is used to perform fast nearest neighbor search and sorting based on the Euclidean distance or path planning distance between the current vehicle location and the target location to lock the optimal dispatch vehicle. Based on the resource requirements contained in the SQL query, a set of candidate hospitals meeting the criteria is retrieved from the hospital resource database. Using a spatial index structure, spatial proximity calculations are performed by combining the hospital location with the incident location or the patient's current location to filter out candidate hospitals. Based on the location description, distance judgment, or regional inclusion relationship contained in the SQL query, a geocoding service is invoked to convert the text location into latitude and longitude coordinates, calculate the great circle distance between two points on the Earth's surface, or determine the spatial relationship between the text location and a predetermined area. Online aggregation, statistics, and calculations are performed according to the business indicator model to generate dynamic indicator data reflecting the regional emergency response situation. The optimal dispatch vehicle, alternative hospitals, spatial relationships, and dynamic indicator data are aligned, correlated, and integrated, and encapsulated into a unified, structured data object according to the agreed data interface specifications.

Images