An accessible first-aid service auxiliary decision-making method, system, device and medium

Abstract

This application relates to a method, system, device, and medium for assisting decision-making in accessible emergency medical services. The method includes: acquiring multimodal raw data from the person seeking help; performing disability pattern recognition processing on the multimodal raw data to obtain disability pattern recognition results; the multimodal raw data includes video data, audio data, and interactive text data; generating personalized emergency medical strategies and resource scheduling plans based on the disability pattern recognition results; generating and issuing first and second execution instructions respectively to the user's terminal and the service terminal according to the personalized emergency medical strategies and resource scheduling plans; and obtaining and generating adjustment instructions based on the execution feedback from the user's terminal and the service terminal. This method, through in-depth analysis of multimodal data and a closed-loop decision-making process, significantly improves the adaptability, response efficiency, and intelligence level of emergency medical services for people with disabilities, providing them with more efficient and accurate emergency medical support.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a method, system, device and medium for assisting decision-making in barrier-free emergency services. Background Technology

[0002] The core design of existing emergency response systems revolves around voice communication. While their standardized service procedures can meet the emergency needs of the general able-bodied population, they exhibit significant adaptation deficiencies when providing emergency services to people with disabilities. For example, people with hearing and speech impairments find it difficult to effectively communicate their condition and location information to the emergency dispatch center via traditional voice calls; people with physical movement impairments often cannot accurately describe their own condition or complete related operations according to the verbal instructions of emergency responders; and visually impaired people have difficulty accessing the visually presented guidance content found in routine emergency services.

[0003] To improve this situation, existing emergency response technologies have incorporated improvements such as text relay services and simple video calls. However, these solutions merely replace or add communication channels, failing to deeply integrate disability status as a core decision-making variable into the entire emergency service chain. Whether it's information exchange during the emergency initiation phase, the generation of on-site emergency guidance, the allocation and dispatch of emergency resources, or the coordination between pre-hospital and in-hospital emergency services, a systematic solution targeting disability mode adaptation has not been formed. Furthermore, current emergency response technologies generally suffer from the core problems of "perception lag" and "response disconnect." For example, at the perception level, the system cannot automatically and accurately identify the disability category of the person seeking help in the early stages of emergency interaction, often relying on manual inquiry or self-reporting by the person seeking help. This not only wastes crucial emergency time but may also lead to identification failure due to the person seeking help's anxiety or inability to express themselves effectively. At the response level, even if disability status information is obtained, subsequent interaction guidance methods, emergency instruction generation, and emergency resource matching and dispatch remain fragmented, lacking a unified disability mode adaptation target and failing to form a closed-loop decision-making process. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, system, device and medium for assisting decision-making in barrier-free emergency services to address the above-mentioned technical problems, aiming to improve the accuracy and efficiency of disability pattern recognition and enhance the adaptability and pertinence of emergency services.

[0005] Firstly, this application provides a method for assisting decision-making in accessible emergency medical services, including:

[0006] The process involves acquiring the raw multimodal data of the person seeking help, performing disability pattern recognition processing on the raw multimodal data, and obtaining the disability pattern recognition results. The raw multimodal data includes video data, audio data, and interactive text data.

[0007] Based on the disability pattern recognition results, personalized emergency rescue plans are generated and processed to obtain personalized emergency rescue strategies and resource scheduling plans.

[0008] Based on the personalized emergency rescue strategy and resource allocation plan, the system generates and issues a first execution command and a second execution command to the caller's terminal and the service terminal, respectively. It also obtains and generates adjustment commands based on the execution feedback from the caller's terminal and the service terminal.

[0009] In one embodiment, the multimodal raw data is processed for disability pattern recognition to obtain disability pattern recognition results, including:

[0010] The video data is normalized to obtain intermediate video data; illumination equalization is performed on the intermediate video data to obtain illumination-standardized intermediate video data; region of interest extraction is performed on the illumination-standardized intermediate video data to obtain standardized video data containing face region sequences and hand region sequences.

[0011] The audio data in the multimodal raw data is denoised to obtain denoised intermediate audio data; speech activity detection is performed based on the denoised intermediate audio data to obtain standardized audio data composed of effective speech segments and non-speech audio segments.

[0012] The interactive text data in the multimodal raw data is segmented into words to obtain the word segmentation results. Based on the word segmentation results, word vector conversion is performed to obtain text vector data. The input interval, misspelling distribution and repetition pattern of the interactive text data are analyzed in a time series to obtain the input behavior time series features. The input behavior time series features are used as text behavior feature data.

[0013] Physiological micro-motion features are extracted from face region sequences in standardized video data, and motion trajectory features are extracted from hand region sequences in standardized video data. The physiological micro-motion features and motion trajectory features are fused to obtain a visual feature vector.

[0014] Feature extraction is performed on valid speech segments in standardized audio data to obtain speech feature sub-vectors. Environmental sound classification is performed on non-speech audio segments in standardized audio data to obtain environmental sound feature sub-vectors. The speech feature sub-vectors and environmental sound feature sub-vectors are concatenated to obtain auditory feature vectors. Temporal correlation processing is performed on text vector data and text behavior feature data to obtain comprehensive text feature vectors.

[0015] A gated attention fusion network is constructed. Based on the gated attention fusion network, visual feature vectors, auditory feature vectors, and text comprehensive feature vectors are dynamically weighted according to the reliability index of each modality feature and the context priority of the emergency scene, and the corresponding weights are obtained. The modality features are then weighted according to the weights to obtain the weighted modality features. The reliability index includes speech recognition confidence, visual feature matching degree, and text semantic coherence. The context priority of the emergency scene is dynamically adjusted based on the urgency of the emergency.

[0016] A deep semantic fusion process is performed on the weighted modal features through a gated attention fusion network to obtain the user state representation vector.

[0017] A pre-defined disability pattern classifier is used to classify the user state representation vector to obtain disability pattern recognition results containing corresponding classification labels and classification confidence scores. The classification dimensions of the disability pattern classifier include hearing and speech impairment, upper limb motor impairment, lower limb motor impairment, and visual impairment.

[0018] In one embodiment, based on the disability pattern recognition results, a personalized emergency medical plan generation process is performed to obtain a personalized emergency medical strategy and resource scheduling plan, including:

[0019] Based on the classification labels and classification confidence scores in the disability pattern recognition results, and combined with the preset disability pattern-interaction channel mapping rules, the accessibility interaction channel configuration is determined to obtain the accessibility interaction channel configuration. Among them, the disability pattern-interaction channel mapping rules include configuring sign language video channel and text interaction channel for hearing and speech impairment mode, configuring voice enhancement channel and high contrast text channel for visual impairment mode, and configuring simplified touch channel and voice command channel for upper limb movement impairment mode.

[0020] Based on the accessibility interaction channel configuration, a corresponding symptom guidance interface or instruction is generated, which is used to instruct the person seeking help to input symptom description information; the input information of the person seeking help is received through the accessibility interaction channel, and the input information is semantically parsed to obtain the symptom description information of the person seeking help.

[0021] Based on symptom description information, disability pattern recognition results, and user state representation vectors, the system uses symptom description information as the starting point for retrieval and disability pattern recognition results and user state representation vectors as constraints. It then calls a pre-set accessibility first aid knowledge graph for reasoning processing to generate personalized first aid strategies. The accessibility first aid knowledge graph includes disease and injury nodes, first aid operation nodes, disability type nodes, execution ability requirement nodes, and information presentation method nodes. The personalized first aid strategy includes step-by-step first aid actions, operation variations adapted to disability patterns, and key precautions.

[0022] Personalized emergency response strategies are analyzed and processed to extract specific resource requirements, including requirements for accessible transport equipment, special communication skills for medical staff, accessibility of hospitals, and emergency operation assistance tools.

[0023] Based on special resource needs and disability pattern recognition results, a multi-objective scheduling optimization function is constructed by combining response time indicators, illness severity indicators, disability suitability indicators, and hospital acceptance suitability indicators. Among them, the disability suitability indicator is used to quantify the degree of matching between resources and disability patterns, and the hospital acceptance suitability indicator is used to quantify the target hospital's ability to receive and support disabled people seeking help.

[0024] Real-time status data of emergency medical resources is acquired, including ambulance location data, ambulance accessibility equipment configuration data, medical staff skill data, and hospital reception capacity data. Based on the real-time status data of emergency medical resources, a non-dominated sorting genetic algorithm is used to solve the multi-objective scheduling optimization function to obtain a Pareto optimal solution set. The scheme with the highest comprehensive score is selected from the Pareto optimal solution set as the resource scheduling scheme.

[0025] In one embodiment, based on a personalized emergency rescue strategy and resource allocation plan, a first execution instruction and a second execution instruction are generated and issued to the caller's terminal and the service terminal, respectively, including:

[0026] The system obtains the accessibility interaction channel configuration, and based on the information presentation method corresponding to the accessibility interaction channel configuration, performs targeted format conversion processing on the personalized first aid strategy to obtain the first execution instruction for the user's terminal; the first execution instruction includes first aid action guidance content, operation progress prompts, and feedback interaction entry;

[0027] Based on the disability pattern recognition results, key operations and operation variations in personalized emergency rescue strategies, emergency resource allocation information and special resource needs in resource scheduling schemes, a second execution instruction is generated for the service terminal. The second execution instruction includes dispatcher assistance prompts, ambulance terminal task instructions, and hospital emergency department pre-notification instructions. Among them, the dispatcher assistance prompts are used to integrate the core characteristics of disability patterns and communication precautions, the ambulance terminal task instructions are used to clarify the requirements for the use of accessible equipment and the key points of emergency operation coordination, and the hospital emergency department pre-notification instructions include disability adaptation and reception conditions and specialist preparation requirements.

[0028] The compatibility of the first execution instruction is verified based on the device type of the user's terminal, and the first execution instruction is sent to the user's terminal; the corresponding sub-instructions in the second execution instruction are distributed according to the functional attributes of the service terminal.

[0029] In one embodiment, an adjustment instruction is generated based on the execution feedback from the user's terminal and the service terminal, including:

[0030] The system receives emergency operation feedback data returned by the caller's terminal and resource execution feedback data returned by the service terminal. The emergency operation feedback data includes data on the completion rate of the disability mode adaptation operation, data on the execution time of each step, and interactive feedback data. The resource execution feedback data includes data on the activation status of the ambulance's accessibility equipment, data on the execution of special communication skills by medical staff, and data on the progress of the hospital's accessibility reception conditions preparation.

[0031] Based on personalized emergency rescue strategies and resource scheduling schemes, we extract the expected thresholds for emergency rescue steps, the expected thresholds for resource scheduling time, and the expected indicators for disability mode adaptation. We then process the expected execution state to obtain the expected execution state that includes operational expectations, resource expectations, and adaptation expectations.

[0032] The operation completion deviation data is obtained by comparing the emergency response operation feedback data with the expected operation status in the execution expectation state; the resource scheduling progress deviation data is obtained by comparing the resource execution feedback data with the expected resource status in the execution expectation state; the disability mode adaptation operation completion data is obtained by comparing the adaptation expectation in the execution expectation state; the operation completion deviation data, resource scheduling progress deviation data, and disability adaptation deviation data are weighted and fused to obtain the deviation analysis results.

[0033] Based on preset deviation thresholds and deviation priority rules, the deviation analysis results are judged and processed. If any deviation data in the deviation analysis results exceeds the corresponding deviation threshold, the judgment result that adjustment is required is obtained. If none of the deviation data exceeds the corresponding deviation threshold, the judgment result that no adjustment is required is obtained.

[0034] If the judgment result indicates that adjustment is needed, based on the type and degree of deviation in the deviation analysis results, for the user's terminal, generate instruction adjustment content adapted to the disability mode, and for the service terminal, generate resource scheduling optimization instructions. Resource scheduling optimization instructions include instructions for early activation of ambulance accessibility equipment, instructions for optimization of special communication solutions for medical staff, or instructions for expedited preparation for hospital accessibility reception.

[0035] Based on the instructions to adjust the content and optimize resource scheduling, the adjustment instructions are obtained.

[0036] In one embodiment, the mathematical expression of the multi-objective scheduling optimization function is:

[0037]

[0038] in, To gather those seeking help Indicates the first The person who initiated the emergency call, ; A collection of resource units Indicates the first A combined resource unit, and ; For resource allocation matrix, Indicating the person seeking help Assigned to resource units ; In order to respond to the time target, and , For resource arrival time; As a disease-weighted target, and , For the severity of the illness, For resource processing capacity; For disability adaptation goals, and , For resource-disability fit; For the hospital to receive the target, and , For resource units The corresponding hospital provides assistance to the person seeking help. The reception compatibility of the disabled mode; For resource units Maximum load capacity; For resource units Arrival time; For resource units Transit time; This is the maximum total time for emergency rescue; For resource units The accessibility configuration compliance coefficient, with a value range of [0,1]; The minimum threshold for configuring accessibility compliance coefficients.

[0039] Secondly, this application also provides an accessibility emergency medical services decision support system, including:

[0040] The disability pattern recognition module is used to acquire the multimodal raw data of the person seeking help, perform disability pattern recognition processing on the multimodal raw data, and obtain the disability pattern recognition result; the multimodal raw data includes video data, audio data, and interactive text data;

[0041] The personalized emergency rescue plan generation module is used to generate personalized emergency rescue plans based on the disability pattern recognition results, and obtain personalized emergency rescue strategies and resource scheduling plans.

[0042] The instruction generation and feedback adjustment module is used to generate and issue first and second execution instructions to the caller's terminal and the service terminal respectively, based on personalized emergency rescue strategies and resource scheduling plans, and to obtain and generate adjustment instructions based on the execution feedback from the caller's terminal and the service terminal.

[0043] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the first aspect.

[0044] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the first aspect.

[0045] The aforementioned method, system, equipment, and medium for assisting decision-making in accessible emergency medical services first acquire multimodal raw data and perform disability pattern recognition processing, solving the problem of delayed disability status perception in traditional emergency medical systems and improving the accuracy and timeliness of disability pattern recognition. Secondly, based on the disability pattern recognition results, personalized emergency medical strategies and resource scheduling plans are generated, addressing the disconnect between emergency medical services and disability needs, and improving the relevance of emergency medical plans and the rationality of resource scheduling. Furthermore, execution instructions are generated and issued to different terminals based on the personalized plans, achieving precise adaptation between emergency medical guidance and service collaboration, improving the effectiveness of instruction transmission and the efficiency of multi-terminal collaboration. Finally, adjustment instructions are generated based on execution feedback, solving the problem of insufficient dynamic adaptation in the emergency medical process, and improving the closed-loop adaptability and overall rescue inclusiveness of emergency medical services. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of an accessible emergency medical service auxiliary decision-making method is provided as an exemplary embodiment of the present invention;

[0048] Figure 2 A flowchart illustrating a method for generating adjustment instructions, as provided in an exemplary embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of an accessible emergency medical service auxiliary decision-making system provided as an exemplary embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] In one embodiment, such as Figure 1 As shown, a method for assisting decision-making in accessible emergency services is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0052] S101: Obtain the multimodal raw data of the person seeking help, perform disability pattern recognition processing on the multimodal raw data, and obtain the disability pattern recognition result; the multimodal raw data includes video data, audio data, and interactive text data.

[0053] Specifically, when an emergency request is triggered, the user's terminal can automatically activate a multi-data acquisition module to simultaneously acquire multimodal raw data, including video, audio, and interactive text data. Video data captures visual information such as facial expressions and body movements; audio data collects acoustic information such as vocalization and ambient background noise; and interactive text data comes from the user's manual input and shortcut trigger records. All of this data can be uploaded to the emergency command cloud platform in real time via the emergency communication network. Upon receiving the multimodal raw data, the cloud platform preprocesses the data to eliminate interference factors such as environmental noise and data redundancy, standardizing and normalizing the data format. It then extracts the core features of each data type and performs multimodal feature fusion. The high-dimensional data representation resulting from feature fusion is input into a pre-trained disability pattern classification model. The model outputs a disability pattern recognition result containing disability type classification labels and corresponding confidence levels. This process, through complementary analysis of multimodal data, overcomes the limitations of traditional emergency systems that rely on a single voice modality for user status assessment, achieving accurate recognition of disability patterns.

[0054] S102: Based on the disability pattern recognition results, perform personalized emergency rescue plan generation processing to obtain personalized emergency rescue strategies and resource scheduling plans.

[0055] Specifically, using disability pattern recognition results as the core input, and combining them with general emergency medical knowledge, a rule base for adaptability between disability patterns and emergency procedures can be constructed. By matching these rules, emergency procedures suitable for the current disability pattern are selected, eliminating procedures that the person seeking help cannot perform due to their disability type, thus forming a structured, personalized emergency medical strategy. Furthermore, based on the disability pattern recognition results, corresponding special resource needs can be derived. These needs can be collaboratively modeled with the core objectives of conventional emergency medical resource scheduling (such as response time and severity of illness) to construct a multi-objective optimization model. Solving this model yields a resource scheduling scheme that balances disability adaptability and emergency timeliness. This process effectively solves the technical problem of the disconnect between traditional emergency medical strategy resource scheduling and disability needs. The generated personalized emergency medical strategy matches the person seeking help's execution capabilities, and the resource scheduling scheme accurately addresses the person's specific needs, improving the executability of the emergency medical strategy and the rationality of resource scheduling.

[0056] S103: Based on the personalized emergency rescue strategy and resource allocation plan, generate and issue the first execution command and the second execution command respectively to the helper's terminal and the service terminal, obtain and generate adjustment commands based on the execution feedback from the helper's terminal and the service terminal.

[0057] Specifically, different instructions can be distributed based on the functional attributes of different terminals and the needs of different users. For example, a first execution instruction can be generated for the user's terminal based on a personalized emergency rescue strategy. The presentation format and interaction logic of this instruction are adapted to the disability mode identified in step S101 to ensure that the user can efficiently receive and execute the emergency rescue guidance. Furthermore, a second execution instruction can be generated for the service terminal based on the personalized emergency rescue strategy and resource scheduling plan. This instruction includes core content such as emergency resource allocation information and multi-terminal collaborative operation requirements, ensuring that medical personnel and dispatchers at the service terminal can clearly understand the user's disability status and emergency needs. Both types of execution instructions can be pushed to the user's terminal and the service terminal respectively through the terminal adaptation interface of the emergency rescue command cloud platform, achieving accurate instruction delivery. After the instruction is issued, feedback data on the completion status of emergency operations from the caller's terminal and feedback data on the progress of resource scheduling execution from the service terminal can be continuously obtained. By comparing and analyzing the two types of feedback data with the preset expected execution status, the deviation between the actual execution process and the expected goal can be determined. Corresponding adjustment instructions can be generated for the cause of the deviation, and the first or second execution instruction can be dynamically fine-tuned. In this way, the uncertainty problem at the emergency scene can be solved through the feedback adjustment mechanism, and the stability and reliability of emergency services can be further improved.

[0058] The aforementioned method first accurately determines the disability type of the person seeking help through disability pattern recognition based on multimodal data, enhancing the targeted nature of the emergency response. Second, based on the disability pattern recognition results, it utilizes intelligent processing to generate personalized emergency response strategies and resource allocation plans, improving the adaptability and accuracy of the emergency response plan. Finally, by issuing and adjusting execution instructions, it effectively solves the problem of disconnection between different stages of the emergency response process, enhancing the continuity and closed-loop management capabilities of emergency services. Furthermore, this method also improves the overall efficiency and intelligence level of emergency services, providing more efficient and precise emergency support for people with disabilities.

[0059] In one embodiment, disability pattern recognition processing is performed on the multimodal raw data to obtain disability pattern recognition results, including:

[0060] The video data is normalized to obtain intermediate video data; illumination equalization is performed on the intermediate video data to obtain illumination-standardized intermediate video data; region of interest extraction is performed on the illumination-standardized intermediate video data to obtain standardized video data containing face region sequences and hand region sequences.

[0061] The audio data in the multimodal raw data is denoised to obtain denoised intermediate audio data; speech activity detection is performed based on the denoised intermediate audio data to obtain standardized audio data composed of effective speech segments and non-speech audio segments.

[0062] The interactive text data in the multimodal raw data is segmented into words to obtain the word segmentation results. Based on the word segmentation results, word vector conversion is performed to obtain text vector data. The input interval, misspelling distribution and repetition pattern of the interactive text data are analyzed in a time series to obtain the input behavior time series features. The input behavior time series features are used as text behavior feature data.

[0063] Physiological micro-motion features are extracted from face region sequences in standardized video data, and motion trajectory features are extracted from hand region sequences in standardized video data. The physiological micro-motion features and motion trajectory features are fused to obtain a visual feature vector.

[0064] Feature extraction is performed on valid speech segments in standardized audio data to obtain speech feature sub-vectors. Environmental sound classification is performed on non-speech audio segments in standardized audio data to obtain environmental sound feature sub-vectors. The speech feature sub-vectors and environmental sound feature sub-vectors are concatenated to obtain auditory feature vectors. Temporal correlation processing is performed on text vector data and text behavior feature data to obtain comprehensive text feature vectors.

[0065] A gated attention fusion network is constructed. Based on the gated attention fusion network, visual feature vectors, auditory feature vectors, and text comprehensive feature vectors are dynamically weighted according to the reliability index of each modality feature and the context priority of the emergency scene, and the corresponding weights are obtained. The modality features are then weighted according to the weights to obtain the weighted modality features. The reliability index includes speech recognition confidence, visual feature matching degree, and text semantic coherence. The context priority of the emergency scene is dynamically adjusted based on the urgency of the emergency.

[0066] A deep semantic fusion process is performed on the weighted modal features through a gated attention fusion network to obtain the user state representation vector.

[0067] A pre-defined disability pattern classifier is used to classify the user state representation vector to obtain disability pattern recognition results containing corresponding classification labels and classification confidence scores. The classification dimensions of the disability pattern classifier include hearing and speech impairment, upper limb motor impairment, lower limb motor impairment, and visual impairment.

[0068] Specifically, the issue of inconsistent video resolution caused by hardware differences in the terminal acquisition devices of different users can be addressed first. This ensures a uniform input data format for subsequent feature extraction, avoiding feature bias introduced by resolution differences. For example, bilinear interpolation can be used to scale each frame of the original video data. By establishing a mapping relationship between the original image pixels and the target resolution pixels, the grayscale value of the target pixels can be calculated, where the target resolution is set to a preset uniform size (e.g., 640×480 pixels). Furthermore, a histogram equalization algorithm can be used. By statistically analyzing the grayscale histogram of each frame of the intermediate video data, the cumulative distribution function of the grayscale levels can be calculated, mapping the grayscale values ​​of the original image to a uniformly distributed grayscale range. This expands the grayscale range of the image, enhances contrast, and eliminates the interference of uneven illumination on subsequent feature extraction. Finally, a Haar-like feature model combined with an Adaboost classifier can be used to construct an object detection model to extract regions of interest from the illumination-standardized intermediate video data. This yields core visual information reflecting the disability pattern, thereby reducing the interference of background redundancy on feature extraction. For example, the model can be used to detect face regions in each frame of the image after illumination normalization, determine the bounding box coordinates of the face, and crop to obtain a face image sequence. Then, the same detection model is used, combined with prior knowledge of hand contour features, to detect and crop the hand region in the image to obtain a hand image sequence. Finally, the face region sequence and the hand region sequence are integrated into normalized video data.

[0069] Specifically, by denoising the audio data in the multimodal raw data, interference from environmental noise (such as traffic noise and crowd noise) in emergency scenarios can be removed, ensuring the accuracy of subsequent audio feature extraction. For example, a wavelet threshold denoising algorithm can be used. First, the original audio data is decomposed into wavelet coefficients at different scales. Then, an adaptive threshold is set according to the statistical characteristics of the noise, and the decomposed wavelet coefficients are thresholded. Noise coefficients below the threshold are set to zero, while effective signal coefficients above the threshold are retained and corrected. Finally, inverse wavelet transform is performed on the processed wavelet coefficients to reconstruct the denoised intermediate audio data. Furthermore, based on the denoised intermediate audio data, speech activity detection processing can be performed to separate the effective speech containing the caller's vocalizations from the non-speech portion containing only environmental noise. The features of the effective speech can be used to determine the state of hearing and speech impairment, while the environmental noise features of the non-speech audio segments can help determine the background of the emergency scenario. For example, a dual threshold method of energy and zero-crossing rate can be used. First, the short-time energy and zero-crossing rate of the denoised intermediate audio data are calculated, and energy thresholds and zero-crossing rate thresholds are set. By traversing the audio frames, audio frames with short-term energy greater than the energy threshold and zero-crossing rate within a set range can be identified as speech frames. Combining consecutive speech frames can yield valid speech segments, while audio frames that do not meet the above conditions can be identified as non-speech frames. Consecutive non-speech frames form non-speech audio segments, which are then integrated to obtain standardized audio data.

[0070] In a schematic representation, word segmentation of interactive text data in multimodal raw data breaks down continuous text content into the smallest semantic units (words), providing a foundation for subsequent word vector conversion. For example, a word segmentation algorithm combining dictionary matching and statistical language models, such as the jieba algorithm, can be used. A specialized dictionary for emergency medical services (containing emergency terminology, symptom names, etc.) can be loaded, and the interactive text data can be scanned and matched character by character. A statistical model is then used to calculate the probability of word combinations, and the segmentation result with the highest probability is selected as the final text segmentation result. Word vector conversion based on the text segmentation results maps discrete words to a continuous low-dimensional vector space, enabling the vectors to represent the semantic information of words and the relationships between words, facilitating subsequent feature fusion and analysis. For example, the Word2Vec model can be used. This model, pre-trained on a large corpus of emergency medical service text, establishes a mapping relationship between words and low-dimensional vectors. The text segmentation results are input into the pre-trained Word2Vec model, which outputs a low-dimensional vector corresponding to each word. Mean pooling is then applied to all word vectors to obtain text vector data representing the semantic information of the entire interactive text. The input timestamps of the interactive text data can be extracted first, and the input time difference between two adjacent characters or words can be calculated to obtain the input interval sequence. The mean and variance of this sequence are then calculated to obtain the input interval features. Next, the number and types of misspellings in the text (such as errors involving similar-looking characters or similar-sounding characters) are counted, and the misspelling rate and entropy of the misspelling distribution are calculated to obtain the misspelling distribution features. Finally, sequence pattern mining algorithms (such as frequent itemset mining) can be used to identify recurring words or sentence fragments in the text, and the repetition frequency and length are counted to obtain repetition pattern features. By integrating the input interval features, misspelling distribution features, and repetition pattern features, the temporal features of input behavior can be formed, i.e., text behavior feature data.

[0071] Specifically, physiological micro-movements in the facial region (such as slight facial muscle twitching, drooping of the mouth, and inability to close the eyelids properly) can reflect the nervous system or muscle function of the person seeking help, and are important evidence for judging visual impairment and some motor disorders. Therefore, optical flow can be used to extract physiological micro-movement features. Optical flow field calculations are performed on consecutive frames of images in the facial region sequence to obtain the motion vector of each pixel. By setting a motion amplitude threshold, micro-movement pixels with amplitudes less than the threshold can be filtered out (excluding non-physiological micro-movement interference such as large head rotations). Principal component analysis is performed on the filtered micro-movement vectors to extract principal component feature vectors as physiological micro-movement features. Furthermore, the movement trajectory of the hand (such as whether there is tremor, whether the trajectory is smooth, and whether a specific movement trajectory can be completed) directly reflects the upper limb motor ability and is a core feature for judging upper limb motor disorders. Therefore, the Kalman filter algorithm can be used to track the center of the hand contour in the hand region sequence, obtaining the coordinate sequence of the hand center. This coordinate sequence is then smoothed to eliminate tracking noise. By calculating and integrating the first derivative (velocity), second derivative (acceleration), and parameters such as the curvature and number of inflection points of the trajectory, motion trajectory features can be formed. A weighted average fusion strategy can be used to assign weights to the physiological micro-motion features and motion trajectory features based on their importance in disability recognition (weight coefficients are determined through cross-validation). The fusion formula is as follows: ,in For visual feature vectors, The weighting coefficients for physiological micro-movement features (range 0-1) This is a feature vector of physiological micro-movements. This represents the motion trajectory feature vector. This formula allows the fusion of two types of feature vectors into a single-dimensional visual feature vector.

[0072] Furthermore, Mel frequency cepstral coefficients (MFCCs) can be extracted as core speech features. This involves pre-emphasizing, framing, and windowing effective speech segments, followed by a Fast Fourier Transform (FFT) to obtain the spectrum. Passing this spectrum through a Mel filter bank yields the Mel spectrum. Taking the logarithm of the Mel spectrum and performing a Discrete Cosine Transform (DCT) extracts the first 13 coefficients, along with first and second-order differences, forming speech feature sub-vectors. Since the type of ambient sound (e.g., indoor, outdoor, accident scene) can aid in identifying emergency scenarios and provide contextual support for disability pattern recognition, Mel spectrum combined with Support Vector Machines (SVM) can be used for ambient sound classification. This involves extracting Mel spectral features from non-speech audio segments, inputting these features into a pre-trained SVM classifier to obtain ambient sound category labels, and then performing one-hot encoding on these labels to obtain ambient sound feature sub-vectors. Concatenating the speech and ambient sound feature sub-vectors sequentially creates an auditory feature vector with dimensions equal to the sum of the two. Text semantic information (text vectors) and input behavioral information (text behavioral features) exhibit temporal correlation. Temporal correlation processing of text vector data and text behavioral feature data can more comprehensively reflect the text interaction ability and disability-related status of the person seeking help. Illustratively, an attention mechanism can be used for temporal correlation, constructing a temporal attention weight matrix. This matrix assigns attention weights according to the temporal order of text input. The text vector data and text behavioral feature data are then weighted and summed according to these attention weights to obtain a comprehensive text feature vector.

[0073] Specifically, the structure of a gated attention fusion network can include an input layer, a gate layer, a weight calculation layer, and a fusion layer. The input layer receives visual feature vectors, auditory feature vectors, and a combined text feature vector. The gate layer, composed of a fully connected layer and a sigmoid activation function, adaptively adjusts the input intensity of each modality's features. The weight calculation layer calculates dynamic weights based on the reliability index of each modality and the priority of the scene context. The fusion layer can perform deep fusion of the weighted features. Based on the gated attention fusion network, the reliability index of each modality's features and the priority of the emergency rescue scene context can be combined to dynamically assign weights to the three types of feature vectors, resulting in weighted features for each modality. The speech recognition confidence score can be calculated using the decoding score of the speech recognition model, as shown in the formula... ,in For speech recognition confidence, The decoding score for the current speech segment. This represents the maximum decoding score of the model. The visual feature matching degree can be calculated using the cosine similarity between the feature and a preset disability pattern feature template, as shown in the formula: ,in For visual feature matching degree, This is the current visual feature vector. The preset feature template vector is used; text semantic coherence is calculated through the perplexity of the language model, using the formula: ,in For the semantic coherence of the text, The number of words in the text. For the first The conditional probability of each word.

[0074] Indicatively, the contextual priority of an emergency rescue scenario can be dynamically adjusted based on the urgency of the situation. The urgency can be determined by combining key information in the symptom description (such as unconsciousness) with vital sign data (such as heart rate). The higher the urgency, the more weight is allocated to modalities that quickly reflect the disability status (such as visual features). The formula for calculating the dynamic weights can be... ,in For the first one mode ( Dynamic weights (corresponding to visual, auditory, and textual senses respectively) The weighting coefficient for the reliability index (value range 0-1). For the first Reliability indicators for each mode For the first The scene context priority of each modality is determined by this formula. After calculating the dynamic weight of each modality, the feature vector of each modality can be multiplied by the corresponding dynamic weight to obtain the weighted features of each modality.

[0075] Since the weighted modal features highlight effective information and suppress interfering information, deep semantic fusion of these features using a gated attention fusion network can further uncover cross-modal correlations, forming a high-dimensional vector that comprehensively represents the disability status and emergency-related features of the person seeking help. For example, the fusion layer can employ a two-layer fully connected network. The first fully connected layer maps the weighted modal features to the same high-dimensional space, using the ReLU activation function, as shown in the formula. ReLU ,in The first layer output features, This is the weight matrix of the fully connected layer. This is the concatenated result of weighted visual, auditory, and textual feature vectors. The bias vector; the second fully connected layer pair Dimension compression and feature integration are performed to output a user state representation vector with fixed dimensions. This vector contains both independent feature information of each modality and cross-modal semantic information, providing comprehensive feature support for subsequent classification.

[0076] Specifically, the preset disability pattern classifier can employ a structure combining a convolutional neural network (CNN) and a softmax classifier. The CNN part is used to further extract deep features from the user state representation vector, while the softmax classifier maps these deep features to the probability space of various disability patterns. The classification dimension can be set to hearing and speech impairment, upper limb motor impairment, lower limb motor impairment, and visual impairment. This classification dimension can cover the most common disability types in emergency scenarios, thus meeting the adaptation needs of personalized emergency services. The user state representation vector is input into the preset disability pattern classifier. First, the vector is processed through convolutional and pooling layers of the CNN to extract features and reduce dimensionality, resulting in a deeper feature vector with lower dimensionality and stronger discriminative power. The deep feature vector is then input into the softmax classifier to calculate the probability value of the feature vector belonging to each disability pattern, using the formula: ,in For the feature vector to belong to the th Disability-like mode These correspond to the probabilities of hearing and speech impairment, upper limb motor impairment, lower limb motor impairment, and visual impairment, respectively. The input layer of the Softmax classifier corresponds to the 1st The class score is then calculated. Finally, the disability pattern with the highest probability value is determined as the classification label, and this highest probability value is the classification confidence score. By integrating the classification label and the classification confidence score, a disability pattern recognition result can be generated. This classification process can quickly output accurate recognition results, and the classification confidence score can be used to judge the reliability of the recognition results, providing a reference for the generation of subsequent emergency rescue plans.

[0077] In one embodiment, based on the disability pattern recognition results, a personalized emergency medical plan generation process is performed to obtain a personalized emergency medical strategy and resource scheduling plan, including:

[0078] Based on the classification labels and classification confidence scores in the disability pattern recognition results, and combined with the preset disability pattern-interaction channel mapping rules, the accessibility interaction channel configuration is determined to obtain the accessibility interaction channel configuration. Among them, the disability pattern-interaction channel mapping rules include configuring sign language video channel and text interaction channel for hearing and speech impairment mode, configuring voice enhancement channel and high contrast text channel for visual impairment mode, and configuring simplified touch channel and voice command channel for upper limb movement impairment mode.

[0079] Based on the accessibility interaction channel configuration, a corresponding symptom guidance interface or instruction is generated, which is used to instruct the person seeking help to input symptom description information; the input information of the person seeking help is received through the accessibility interaction channel, and the input information is semantically parsed to obtain the symptom description information of the person seeking help.

[0080] Based on symptom description information, disability pattern recognition results, and user state representation vectors, the system uses symptom description information as the starting point for retrieval and disability pattern recognition results and user state representation vectors as constraints. It then calls a pre-set accessibility first aid knowledge graph for reasoning processing to generate personalized first aid strategies. The accessibility first aid knowledge graph includes disease and injury nodes, first aid operation nodes, disability type nodes, execution ability requirement nodes, and information presentation method nodes. The personalized first aid strategy includes step-by-step first aid actions, operation variations adapted to disability patterns, and key precautions.

[0081] Personalized emergency response strategies are analyzed and processed to extract specific resource requirements, including requirements for accessible transport equipment, special communication skills for medical staff, accessibility of hospitals, and emergency operation assistance tools.

[0082] Based on special resource needs and disability pattern recognition results, a multi-objective scheduling optimization function is constructed by combining response time indicators, illness severity indicators, disability suitability indicators, and hospital acceptance suitability indicators. Among them, the disability suitability indicator is used to quantify the degree of matching between resources and disability patterns, and the hospital acceptance suitability indicator is used to quantify the target hospital's ability to receive and support disabled people seeking help.

[0083] Real-time status data of emergency medical resources is acquired, including ambulance location data, ambulance accessibility equipment configuration data, medical staff skill data, and hospital reception capacity data. Based on the real-time status data of emergency medical resources, a non-dominated sorting genetic algorithm is used to solve the multi-objective scheduling optimization function to obtain a Pareto optimal solution set. The scheme with the highest comprehensive score is selected from the Pareto optimal solution set as the resource scheduling scheme.

[0084] Specifically, the classification results of disability modes can be used to match appropriate information interaction methods, ensuring that those seeking help can effectively transmit and receive information. Classification confidence is used to determine the priority of the configuration. If the classification confidence is higher than a preset threshold (e.g., 0.8), the dedicated channel for the corresponding disability mode can be used directly. If the classification confidence is lower than the threshold, two highly relevant channels can be enabled simultaneously to ensure the effectiveness of the interaction. The preset disability mode-interaction channel mapping rules are pre-stored in the system's association table. For example, the sign language video channel for hearing and speech impairment modes calls a pre-trained emergency sign language video library (containing standardized sign language content such as symptom inquiries and operational instructions), while the text interaction channel can be a text input box with large font and low latency. The voice enhancement channel for visual impairment modes can be a voice broadcast module with noise reduction and adaptive volume enhancement, and the high-contrast text channel adjusts the interface color scheme to a highly recognizable combination such as black background and yellow text. The simplified touch channel corresponding to the upper limb movement disorder mode can be to enlarge the size of the interactive button to a preset threshold (such as twice the original size) and reduce the operation level, while the voice command channel can be an input module that supports short voice keywords.

[0085] By matching the aforementioned mapping rules and configuring the channels, the interaction barriers between people with disabilities and the system can be directly eliminated, laying the foundation for subsequent information acquisition. Based on the accessible interaction channel configuration, corresponding symptom guidance interfaces or instructions can also be generated. For example, for the hearing and speech impairment mode, the symptom guidance interface is a sign language video sequence (demonstrating symptom dimensions such as "pain location" and "pain level" in sequence) + a text selection box (preset common symptom options), and the input information is the text option selected by the person seeking help or the recognition result of the sign language action. For the visual impairment mode, the symptom guidance instruction can be a point-by-point voice broadcast, and the input information is the voice response of the person seeking help. For the upper limb movement impairment mode, the symptom guidance interface can be a large symptom button, and the input information is the button content clicked by the person seeking help. By calling the emergency medical terminology dictionary, the input information can be matched with the symptom terms in the dictionary, and after removing redundant information, a structured symptom description information is obtained, ensuring the accuracy of the input for subsequent knowledge graph reasoning.

[0086] Specifically, the pre-built accessible first aid knowledge graph is a pre-constructed directed graph structure. Its nodes include disease / injury nodes, first aid operation nodes, disability type nodes, performance capability requirement nodes, and information presentation method nodes. The edges between nodes represent the relationships (e.g., the edge between "acute chest pain" and "remain resting" and "remain resting" and "upper limb movement disorder" can be labeled "no upper limb operation required"). Therefore, starting from the disease / injury node corresponding to the symptom description information, we can traverse its associated first aid operation nodes. Then, through the constraints of the disability type node and the performance capability requirement node, we can filter out operations that the person seeking help cannot perform (e.g., the "self-pressing acupoints" operation for people with upper limb movement disorders), retain the appropriate first aid operations and associate them with the corresponding information presentation method nodes, and finally generate a personalized first aid strategy that includes step-by-step first aid actions, operation variations adapted to the disability mode, and key precautions. Furthermore, by traversing the operational variations and key considerations within personalized emergency response strategies, corresponding resource requirement types can be matched. For example, the operational variation "assisting in a semi-recumbent position" corresponds to the requirement for accessible transport equipment (such as a lifting stretcher); the key consideration "sign language communication" corresponds to the requirement for special communication skills among medical staff (such as medical staff with sign language abilities); the disease / injury node-related "hospital specialist reception" corresponds to the requirement for accessible hospital reception conditions (such as beds for disabled persons and accessible pathways); and the emergency response operation "using a simple breathing aid" corresponds to the requirement for emergency response assistive tools (such as a breathing mask adapted for people with limb disabilities). Moreover, the extracted special resource requirements can be stored in structured data format, providing core constraints for subsequent resource scheduling.

[0087] Furthermore, based on specific resource requirements and disability pattern recognition results, combined with response time indicators, illness severity indicators, disability fit indicators, and hospital acceptance fit indicators, a multi-objective scheduling optimization function can be constructed. Its mathematical expression can be:

[0088]

[0089] in, To gather those seeking help Indicates the first The person who initiated the emergency call, ; A collection of resource units Indicates the first A combined resource unit, and ; For resource allocation matrix, Indicating the person seeking help Assigned to resource units ; In order to respond to the time target, and , For resource arrival time; As a disease-weighted target, and , For the severity of the illness, For resource processing capacity; For disability adaptation goals, and , For resource-disability fit; For the hospital to receive the target, and , For resource units The corresponding hospital provides assistance to the person seeking help. The reception compatibility of the disabled mode; For resource units Maximum load capacity; For resource units Arrival time; For resource units Transit time; This is the maximum total time for emergency rescue; For resource units The accessibility configuration compliance coefficient, with a value range of [0,1]; The minimum threshold for configuring accessibility compliance coefficients.

[0090] Among the constraints of the above formula, This ensures that each person seeking help is allocated only one resource unit. It can limit the maximum capacity of resource unit j. This ensures that the total emergency response time does not exceed the threshold. , This ensures that resource units have basic accessibility equipment.

[0091] Specifically, based on the aforementioned multi-objective scheduling optimization function, real-time status data can first be obtained through the emergency medical resource management system, where ambulance location data is used for calculation. Accessibility equipment configuration data is used for calculation Medical staff skill data is used for calculation Hospital reception capacity data is used for calculation Subsequently, the Non-Dominated Sorting Genetic Algorithm (NSGA-II) is used to solve the multi-objective function. The population is initialized with randomly generated resource allocation schemes. Fast non-dominated sorting divides the population into different levels (lower levels indicate better schemes). The crowding degree of each individual is calculated to maintain population diversity. The next generation is generated through selection, crossover, and mutation operations. After a preset number of iterations, a Pareto optimal solution set is obtained (schemes within the solution set have no mutual dominance relationships). Finally, the schemes within the solution set are comprehensively scored. The scoring rule assigns weights to the four objectives (with disability adaptation having the highest weight). A weighted score is calculated for each scheme, and the scheme with the highest score is selected as the resource scheduling scheme, thus ensuring that the scheduling result balances timeliness and disability adaptation.

[0092] In one embodiment, based on a personalized emergency rescue strategy and resource allocation plan, a first execution instruction and a second execution instruction are generated and issued to the caller's terminal and the service terminal, respectively, including:

[0093] The system obtains the accessibility interaction channel configuration, and based on the information presentation method corresponding to the accessibility interaction channel configuration, performs targeted format conversion processing on the personalized first aid strategy to obtain the first execution instruction for the user's terminal; the first execution instruction includes first aid action guidance content, operation progress prompts, and feedback interaction entry;

[0094] Based on the disability pattern recognition results, key operations and operation variations in personalized emergency rescue strategies, emergency resource allocation information and special resource needs in resource scheduling schemes, a second execution instruction is generated for the service terminal. The second execution instruction includes dispatcher assistance prompts, ambulance terminal task instructions, and hospital emergency department pre-notification instructions. Among them, the dispatcher assistance prompts are used to integrate the core characteristics of disability patterns and communication precautions, the ambulance terminal task instructions are used to clarify the requirements for the use of accessible equipment and the key points of emergency operation coordination, and the hospital emergency department pre-notification instructions include disability adaptation and reception conditions and specialist preparation requirements.

[0095] The compatibility of the first execution instruction is verified based on the device type of the user's terminal, and the first execution instruction is sent to the user's terminal; the corresponding sub-instructions in the second execution instruction are distributed according to the functional attributes of the service terminal.

[0096] Specifically, based on the information presentation format corresponding to the accessible interaction channel, structured personalized first aid strategies can be converted into perceptible and operable instructions for the person seeking help. For example, for the hearing and speech impairment mode, the information presentation method can be sign language video + text. That is, a pre-built first aid sign language video library can be called to convert the step-by-step first aid actions in the personalized first aid strategy into corresponding first aid sign language video clips. Each step is matched with a corresponding standardized sign language demonstration video. The operation variant adapted to the disability mode is converted into large font text instructions. The operation progress prompt is a text progress bar combined with the sign language "pause / continue" demonstration option. The feedback interaction entry is a text confirmation button and a sign language action recognition trigger entry. For the visual impairment mode, the information presentation method can be voice enhancement + high-contrast text. That is, a slow-speed, high-definition speech synthesis engine can be used to convert the personalized first aid strategy into a step-by-step voice broadcast script. The operation variant is a voice emphasis explanation. The operation progress prompt is a voice broadcast "Currently performing step X, out of Y steps". The feedback interaction entry is a voice command "completed" trigger and a high-contrast confirmation button. For the upper limb movement disorder mode, the information presentation method can be simplified touch + voice command. Therefore, the personalized emergency rescue strategy can be broken down into a single large-size touch button (the button size is more than 80% of the width of the terminal interface). The operation variation is the voice prompt "Please ask the people around to assist in completing this step". The operation progress is indicated by the color switch of the button status (not executed / in execution / completed). The feedback interaction entry is a one-click touch button and voice confirmation command.

[0097] The first execution instruction after the above process is completed contains emergency action guidance content that is adapted to the disability mode, operation progress prompts provide real-time feedback on the current execution stage, and the feedback interaction entry is a confirmation method that conforms to the interaction ability of the person seeking help, thereby ensuring that the person seeking help can clearly understand, execute and provide feedback on the emergency operation.

[0098] Furthermore, based on the disability pattern recognition results, key operations and operation variations in personalized emergency care strategies, emergency resource allocation information in resource scheduling plans, and special resource requirements, a second execution instruction for the service terminal can be generated. This instruction includes dispatcher assistance prompts, ambulance terminal task instructions, and hospital emergency department pre-notification instructions. Specifically, classification tags (such as "hearing and speech impairment") from the disability pattern recognition results and communication-related precautions (such as "avoid voice communication") from personalized emergency care strategies can be extracted and integrated into structured prompt text, such as "The person seeking help has a hearing and speech impairment; only text / sign language interaction is supported. Please do not initiate a voice call," thus obtaining dispatcher assistance prompts. Additionally, ambulance configuration information (such as "equipped with a lifting stretcher") from the resource scheduling plan, equipment usage requirements from special resource requirements (such as "lower limb movement disorders require the use of a lifting stretcher"), and operation variations from personalized emergency care strategies (such as "assist in transferring to the stretcher") can be extracted and integrated into task items, such as "Upon arrival at the scene, prioritize using the lifting stretcher and assist the person seeking help in transferring to the stretcher, avoiding pulling on the lower limbs," thus generating ambulance terminal task instructions. Furthermore, information such as target hospital details (e.g., "Emergency Department of XX Hospital"), hospital acceptance conditions (e.g., "Dedicated beds for disabled persons"), and specialist matching requirements (e.g., "Medical staff with sign language skills") from the resource scheduling plan can be extracted and integrated into pre-notification content such as "Please reserve dedicated beds for disabled persons, ensure unobstructed emergency access, and arrange for medical staff with sign language skills to provide matching services," thus obtaining pre-notification instructions from the hospital's emergency department. By integrating these instructions, it can be ensured that different roles on the server side only receive disability adaptation information relevant to their responsibilities, avoiding information redundancy.

[0099] Specifically, verifying the compatibility of the first execution command based on the device type of the user's terminal allows for adaptation to the hardware capabilities and functionalities of different terminals, ensuring the command can be effectively presented and executed. For example, the device identifier of the user's terminal (such as a smartphone, smartwatch, or IoT emergency alarm) can be obtained first, and then matched against a pre-stored table of device capability parameters (including screen size, interaction method, audio output capabilities, etc.) to adjust the first execution command for compatibility. For instance, if a smartwatch has a small screen, the text description can be simplified to a shorter text, and the touch button size can be further enlarged. If the IoT emergency alarm has no screen, the first execution command can be converted into a pure voice broadcast combined with physical button feedback. After successful verification, the first execution command can be pushed to the user's terminal via the emergency communication network. Furthermore, based on the role identifier of the service terminal, such as the dispatcher workstation, the ambulance tablet, or the hospital emergency department system, the corresponding sub-instructions in the second execution instruction can be distributed to the target terminal. Among them, the dispatcher auxiliary prompt instruction can be pushed to the real-time prompt bar of the dispatcher workstation, the ambulance terminal task instruction can be synchronized to the task list module of the tablet, and the hospital emergency department pre-notification instruction can be pushed to the patient information bar of the emergency department information system, ensuring that each service terminal only receives instruction content that matches its function, thereby improving the accuracy of instruction execution.

[0100] In one embodiment, such as Figure 2 As shown, based on the execution feedback from the user's terminal and the service terminal, adjustment instructions are generated, including:

[0101] S201: Receive emergency operation feedback data returned by the caller's terminal and resource execution feedback data returned by the service terminal. The emergency operation feedback data includes data on the completion of the disability mode adaptation operation, data on the execution time of each step, and interactive feedback data. The resource execution feedback data includes data on the activation status of the ambulance's accessibility equipment, data on the execution of medical and nursing special communication skills, and data on the progress of the hospital's accessibility reception conditions preparation.

[0102] S202: Based on personalized emergency rescue strategies and resource scheduling schemes, extract the expected thresholds for emergency rescue steps, the expected thresholds for resource scheduling time, and the expected indicators for disability mode adaptation, and perform execution expectation state construction processing to obtain the execution expectation state including operation expectation, resource expectation, and adaptation expectation.

[0103] S203: Compare the emergency medical operation feedback data with the operation expectations in the expected execution state to obtain operation completion deviation data; compare the resource execution feedback data with the resource expectations in the expected execution state to obtain resource scheduling progress deviation data; compare the disability mode adaptation operation completion data with the adaptation expectations in the expected execution state to obtain disability adaptation deviation data; perform weighted fusion processing on the operation completion deviation data, resource scheduling progress deviation data, and disability adaptation deviation data to obtain deviation analysis results;

[0104] S204: Based on the preset deviation threshold and deviation priority rules, the deviation analysis results are judged and processed. If any deviation data in the deviation analysis results exceeds the corresponding deviation threshold, the judgment result that needs to be adjusted is obtained. If none of the deviation data exceeds the corresponding deviation threshold, the judgment result that no adjustment is required is obtained.

[0105] S205: If the judgment result indicates that adjustment is required, based on the type and degree of deviation in the deviation analysis results, generate instruction adjustment content adapted to the disability mode for the user's terminal, and generate resource scheduling optimization instructions for the service terminal. Resource scheduling optimization instructions include instructions for early activation of ambulance accessibility equipment, instructions for optimizing special communication solutions for medical staff, or instructions for expedited preparation for hospital accessibility reception; and obtain adjustment instructions based on the instruction adjustment content and resource scheduling optimization instructions.

[0106] Specifically, for the hearing and speech impairment mode, the disability mode adaptation operation completion data can be the action matching degree of sign language commands, that is, the feature matching result of the helper's actions captured by the terminal camera and the sign language video of the command; the step execution time data can be the length of time it takes for the helper to complete the corresponding step; and the interactive feedback data can be the confirmation information of text input. For the visual impairment mode, the disability mode adaptation operation completion data can be the response accuracy of voice commands, that is, the degree of matching between the helper's voice response and the command requirement; the step execution time data can be the time interval from issuing the voice command to receiving feedback; and the interactive feedback data can be the voice confirmation information. For the upper limb movement impairment mode, the disability mode adaptation operation completion data can be the button click completion rate of simplified touch steps; the step execution time data can be the interval time of button clicks; and the interactive feedback data can be the trigger status of the touch button. In the resource execution feedback data of the service terminal, the ambulance accessibility equipment activation status data can be the on / off status and self-test results of the equipment; the medical staff special communication skills execution data can be the communication method record between medical staff and the helper (such as whether sign language is used); and the hospital accessibility reception condition preparation progress data can be the completion rate of the preparation of dedicated beds and passages. The aforementioned resource execution feedback data can be uploaded to the emergency command cloud platform in real time through the information collection modules of each service terminal.

[0107] Furthermore, expected thresholds for the execution of emergency steps can be obtained from the preset time intervals of each step in the personalized emergency strategy. For example, the expected time for the "remain still" step is 1-2 minutes. Expected thresholds for resource dispatch time can also be obtained from parameters such as ambulance arrival time and hospital preparation time in the resource dispatch plan. For example, the expected ambulance arrival time is 5-8 minutes. In addition, expected indicators for disability mode adaptation can be determined based on the operation completion standards corresponding to the disability mode. For example, the expected action matching degree of sign language commands in the hearing and speech impairment mode is ≥90%, and the expected button click completion rate of simplified touch steps in the upper limb movement impairment mode is ≥80%. By integrating the above-extracted thresholds and indicators according to three dimensions—operation, resources, and adaptation—where operation expectations correspond to the time consumption and completion standards of emergency steps, resource expectations correspond to the progress and status standards of resource dispatch, and adaptation expectations correspond to the operation matching degree standards of the disability mode, a structured expectation benchmark can be formed.

[0108] The emergency response feedback data can then be compared with the expected execution status to calculate the deviation. For example, the deviation data can be calculated as follows: Operation completion deviation data = (Actual operation completion rate - Expected operation completion rate) / Expected operation completion rate; Resource scheduling progress deviation data = (Actual scheduling progress - Expected resource progress) / Expected resource progress; Disability adaptation deviation data = (Actual adaptation completion rate - Expected adaptation completion rate) / Expected adaptation completion rate. If the above calculation result is negative, it indicates that the actual situation has not met expectations. Furthermore, the operation completion deviation data, resource scheduling progress deviation data, and disability adaptation deviation data can be weighted and fused, assigning the disability adaptation deviation the highest priority. The calculation formula is as follows:

[0109]

[0110] in, The results of the deviation analysis are as follows. For disability adaptation bias data, Its weighting coefficient, This is data on resource scheduling progress deviation. Its weighting coefficient, This is the data on the deviation of the operation completion rate. This is used as the weighting coefficient. By integrating the absolute values ​​of the three types of biases using this formula, the overall bias result can be obtained.

[0111] Specifically, judgments can be made based on the comprehensive deviation results, preset deviation thresholds, and deviation priority rules. For example, preset deviation thresholds can be critical values ​​set according to the needs of emergency rescue scenarios, such as a disability adaptation deviation threshold of -0.2 (i.e., actual adaptation completion rate is 20% lower than expected), a resource scheduling progress deviation threshold of -0.1 (i.e., actual progress is 10% lower than expected), and an operation completion deviation threshold of -0.15 (i.e., actual completion rate is 15% lower than expected). Deviation priority rules can be: disability adaptation deviation has a higher priority than resource scheduling progress deviation, which in turn has a higher priority than operation completion deviation, and so on. Using this rule, if the disability adaptation deviation data exceeds the threshold, regardless of whether other deviations meet the standards, it can be determined that adjustment is needed. If the disability adaptation deviation meets the standards, then the resource scheduling progress deviation is judged, and so on.

[0112] If the assessment indicates that adjustments are needed, adjustments can be generated based on the type and degree of deviation in the deviation analysis results. For example, for the user's terminal, instructions can be adjusted to adapt to the disability mode. Specifically, for the hearing and speech impairment mode, if the disability adaptation deviation exceeds a threshold, instructions to reduce the playback speed of sign language videos (e.g., from 1.0x to 0.7x) or to repeat the steps can be generated. For the visual impairment mode, instructions to increase the volume of voice broadcasts or to repeat the instructions can be generated. For the upper limb movement impairment mode, instructions to further simplify the emergency procedures can be generated, such as breaking down "adjusting position" into two sub-steps: "informing surrounding personnel for assistance" and "waiting for assistance." Furthermore, for the service terminal, resource scheduling optimization instructions can be generated. For instance, if the resource scheduling progress deviation exceeds a threshold, instructions to activate ambulance accessibility equipment in advance can be generated, such as requiring the ambulance to activate the lifting stretcher 5 minutes before arrival. If the execution data of special medical communication skills does not meet expectations, instructions to optimize special medical communication solutions can be generated (e.g., requiring medical personnel to switch to text-based communication). If the hospital's progress in preparing for accessibility exceeds a threshold, an expedited accessibility preparation order can be generated, such as requiring the hospital to prioritize the preparation of beds specifically for disabled persons.

[0113] Finally, the instruction adjustment content for the user's terminal and the resource scheduling optimization instruction for the service terminal can be integrated into a structured adjustment instruction. This can include the terminal identifier corresponding to the instruction, the specific parameters of the adjustment content (such as playback speed, step splitting method), execution time requirements, and other information to ensure that each terminal can accurately identify and execute the adjustment content, and achieve dynamic adaptation of the emergency rescue execution process.

[0114] Based on the same inventive concept, this application also provides an accessible emergency medical service auxiliary decision-making system for implementing the aforementioned accessible emergency medical service auxiliary decision-making method. The solution provided by this system is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of the accessible emergency medical service auxiliary decision-making system provided below can be found in the limitations of the accessible emergency medical service auxiliary decision-making method described above, and will not be repeated here.

[0115] In one exemplary embodiment, such as Figure 3 As shown, an accessible emergency medical service auxiliary decision-making system 300 is provided, including:

[0116] The disability pattern recognition module 301 is used to acquire the multimodal raw data of the person seeking help, perform disability pattern recognition processing on the multimodal raw data, and obtain the disability pattern recognition result; the multimodal raw data includes video data, audio data, and interactive text data;

[0117] The personalized emergency rescue plan generation module 302 is used to generate personalized emergency rescue plans based on the disability pattern recognition results, and obtain personalized emergency rescue strategies and resource scheduling plans.

[0118] The instruction generation and feedback adjustment module 303 is used to generate and issue a first execution instruction and a second execution instruction to the helper's terminal and the service terminal respectively, based on the personalized emergency rescue strategy and resource scheduling plan, and to obtain and generate adjustment instructions based on the execution feedback from the helper's terminal and the service terminal.

[0119] In one exemplary embodiment, the present invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the accessible first aid service auxiliary decision-making method of this application. A multi-core processor is preferred to improve the system's parallel processing capability. The memory provides sufficient temporary storage space to support program execution and data processing. The memory capacity should be large enough to accommodate large amounts of data and computational tasks.

[0120] In one exemplary embodiment, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the accessible emergency medical service auxiliary decision-making method of this application. The computer-readable storage medium may include: a read-only memory, a random access memory (RAM), a solid-state drive (SSD), or an optical disc, etc.

[0121] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for assisting decision-making in accessible emergency medical services, characterized in that, The method includes: The system acquires the multimodal raw data of the person seeking help, performs disability pattern recognition processing on the multimodal raw data, and obtains the disability pattern recognition result; the multimodal raw data includes video data, audio data, and interactive text data; Based on the disability pattern recognition results, a personalized emergency rescue plan is generated to obtain a personalized emergency rescue strategy and resource scheduling plan. Based on the personalized emergency rescue strategy and the resource scheduling scheme, a first execution instruction and a second execution instruction are generated and issued to the helper's terminal and the service terminal, respectively. Based on the execution feedback from the helper's terminal and the service terminal, an adjustment instruction is generated.

2. The method according to claim 1, characterized in that, The process of performing disability pattern recognition processing on the multimodal raw data to obtain disability pattern recognition results includes: The video data is normalized to obtain intermediate video data; illumination equalization is performed on the intermediate video data to obtain illumination-standardized intermediate video data; region of interest extraction is performed on the illumination-standardized intermediate video data to obtain standardized video data containing face region sequences and hand region sequences. The audio data in the multimodal raw data is denoised to obtain denoised intermediate audio data; speech activity detection is performed based on the denoised intermediate audio data to obtain standardized audio data composed of effective speech segments and non-speech audio segments. The interactive text data in the multimodal raw data is segmented into words to obtain text segmentation results. Based on the text segmentation results, word vector conversion is performed to obtain text vector data. The input interval, misspelling distribution and repetition pattern of the interactive text data are analyzed in a time series to obtain input behavior time series features. The input behavior time series features are used as text behavior feature data. Physiological micro-motion features are extracted from the face region sequence in the standardized video data, and motion trajectory features are extracted from the hand region sequence in the standardized video data. The physiological micro-motion features and the motion trajectory features are fused to obtain a visual feature vector. Feature extraction is performed on valid speech segments in the standardized audio data to obtain speech feature sub-vectors. Environmental sound classification is performed on non-speech audio segments in the standardized audio data to obtain environmental sound feature sub-vectors. The speech feature sub-vectors and environmental sound feature sub-vectors are concatenated to obtain auditory feature vectors. Temporal correlation processing is performed on the text vector data and the text behavior feature data to obtain a comprehensive text feature vector. A gated attention fusion network is constructed. Based on the gated attention fusion network, the visual feature vector, the auditory feature vector, and the text comprehensive feature vector are dynamically weighted according to the reliability index of each modality feature and the context priority of the emergency scene, so as to obtain the corresponding weights. The modality features are then weighted according to the weights to obtain the weighted modality features. The reliability index includes speech recognition confidence, visual feature matching degree, and text semantic coherence. The context priority of the emergency scene is dynamically adjusted based on the urgency of the emergency. The weighted modal features are subjected to deep semantic fusion processing through the gated attention fusion network to obtain the user state representation vector. A preset disability pattern classifier is used to classify the user state representation vector to obtain a disability pattern recognition result containing corresponding classification labels and classification confidence scores; wherein, the classification dimensions of the disability pattern classifier include hearing and speech impairment, upper limb motor impairment, lower limb motor impairment and visual impairment.

3. The method according to claim 2, characterized in that, Based on the disability pattern recognition results, a personalized emergency medical plan generation process is performed to obtain a personalized emergency medical strategy and resource scheduling plan, including: Based on the classification labels and classification confidence scores in the disability pattern recognition results, and combined with the preset disability pattern-interaction channel mapping rules, the accessibility interaction channel configuration is determined to obtain the accessibility interaction channel configuration; wherein, the disability pattern-interaction channel mapping rules include configuring a sign language video channel and a text interaction channel for hearing and speech impairment modes, configuring a speech enhancement channel and a high-contrast text channel for visual impairment modes, and configuring a simplified touch channel and a voice command channel for upper limb movement impairment modes. Based on the accessibility interaction channel configuration, a corresponding symptom guidance interface or instruction is generated. The symptom guidance interface or instruction is used to instruct the person seeking help to input symptom description information. The input information of the person seeking help is received through the accessibility interaction channel, and the input information is semantically parsed to obtain the symptom description information of the person seeking help. Based on the symptom description information, the disability pattern recognition result, and the user state representation vector, using the symptom description information as the retrieval starting point and the disability pattern recognition result and the user state representation vector as constraints, a preset accessible first aid knowledge graph is invoked for reasoning processing to generate the personalized first aid strategy. The accessible first aid knowledge graph includes disease / injury nodes, first aid operation nodes, disability type nodes, execution ability requirement nodes, and information presentation method nodes. The personalized first aid strategy includes step-by-step first aid actions, operation variations adapted to disability patterns, and key precautions. The personalized emergency rescue strategy is analyzed and processed to extract special resource requirements; these special resource requirements include requirements for accessible transport equipment, special communication skills for medical staff, accessibility of hospitals, and emergency operation assistance tools. Based on the specific resource requirements and the disability pattern recognition results, a multi-objective scheduling optimization function is constructed by combining response time indicators, illness severity indicators, disability suitability indicators, and hospital acceptance suitability indicators. Among them, the disability suitability indicator is used to quantify the degree of matching between resources and disability patterns, and the hospital acceptance suitability indicator is used to quantify the target hospital's ability to receive and support disabled people seeking help. Real-time status data of emergency medical resources is acquired, including ambulance location data, ambulance accessibility equipment configuration data, medical staff skill data, and hospital reception capacity data. Based on the real-time status data of emergency medical resources, a non-dominated sorting genetic algorithm is used to solve the multi-objective scheduling optimization function to obtain a Pareto optimal solution set. The scheme with the highest comprehensive score is selected from the Pareto optimal solution set as the resource scheduling scheme.

4. The method according to claim 3, characterized in that, The step of generating and issuing a first execution instruction and a second execution instruction, respectively, to the caller's terminal and the service terminal, based on the personalized emergency rescue strategy and the resource scheduling scheme, includes: The accessibility interaction channel configuration is obtained, and based on the information presentation method corresponding to the accessibility interaction channel configuration, the personalized first aid strategy is subjected to targeted format conversion processing to obtain a first execution instruction for the user's terminal; the first execution instruction includes first aid action guidance content, operation progress prompts, and feedback interaction entry; Based on the disability pattern recognition results, the key operations and operation variations in the personalized emergency rescue strategy, the emergency resource allocation information and special resource requirements in the resource scheduling scheme, a second execution instruction is generated for the service terminal. The second execution instruction includes a dispatcher assistance prompt instruction, an ambulance terminal task instruction, and a hospital emergency department pre-notification instruction. The dispatcher assistance prompt instruction is used to integrate the core features of the disability pattern and communication precautions. The ambulance terminal task instruction is used to clarify the requirements for the use of accessible equipment and the key points of emergency operation coordination. The hospital emergency department pre-notification instruction includes disability adaptation and reception conditions and specialist preparation requirements. The compatibility of the first execution instruction is verified based on the device type of the user's terminal, and the first execution instruction is sent to the user's terminal; the corresponding sub-instructions in the second execution instruction are distributed according to the functional attributes of the service terminal.

5. The method according to claim 1, characterized in that, The step of obtaining and generating adjustment instructions based on the execution feedback from the user's terminal and the service terminal includes: The system receives emergency operation feedback data returned by the user's terminal and resource execution feedback data returned by the service terminal. The emergency operation feedback data includes disability mode adaptation operation completion data, step execution time data, and interaction feedback data. The resource execution feedback data includes ambulance accessibility equipment activation status data, medical staff special communication skills execution data, and hospital accessibility reception condition preparation progress data. Based on the personalized emergency rescue strategy and the resource scheduling scheme, the expected threshold for emergency rescue step execution, the expected threshold for resource scheduling time, and the expected indicators for disability mode adaptation are extracted. The expected execution state is then constructed to obtain the expected execution state that includes operation expectations, resource expectations, and adaptation expectations. The emergency rescue operation feedback data is compared with the operation expectation in the expected execution state to obtain operation completion deviation data; the resource execution feedback data is compared with the resource expectation in the expected execution state to obtain resource scheduling progress deviation data; the disability mode adaptation operation completion data is compared with the adaptation expectation in the expected execution state to obtain disability adaptation deviation data; the operation completion deviation data, resource scheduling progress deviation data, and disability adaptation deviation data are weighted and fused to obtain deviation analysis results; Based on preset deviation thresholds and deviation priority rules, the deviation analysis results are judged and processed. If any deviation data in the deviation analysis results exceeds the corresponding deviation threshold, a judgment result that needs adjustment is obtained. If none of the deviation data exceeds the corresponding deviation threshold, a judgment result that does not need adjustment is obtained. If the judgment result indicates that adjustment is needed, based on the deviation type and degree in the deviation analysis result, an instruction adjustment content adapted to the disability mode is generated for the helper's terminal, and a resource scheduling optimization instruction is generated for the service terminal. The resource scheduling optimization instruction includes an instruction to activate the ambulance's accessibility equipment in advance, an instruction to optimize the special communication plan for medical staff, or an instruction to expedite the hospital's accessibility reception preparation. The adjustment instruction is obtained based on the content of the instruction adjustment and the resource scheduling optimization instruction.

6. The method according to claim 3, characterized in that, The mathematical expression for the multi-objective scheduling optimization function is: in, To gather those seeking help Indicates the first The person who initiated the emergency call, ; A collection of resource units Indicates the first A combined resource unit, and ; For resource allocation matrix, Indicating the person seeking help Assigned to resource units ; In order to respond to the time target, and , For resource arrival time; As a disease-weighted target, and , For the severity of the illness, For resource processing capacity; For disability adaptation goals, and , For resource-disability fit; For the hospital to receive the target, and , For resource units The corresponding hospital provides assistance to the person seeking help. The reception compatibility of the disabled mode; For resource units Maximum load capacity; For resource units Arrival time; For resource units Transit time; This is the maximum total time for emergency rescue; For resource units The accessibility configuration compliance coefficient, with a value range of [0,1]; The minimum threshold for configuring the accessibility compliance coefficient.

7. A barrier-free emergency medical service auxiliary decision-making system, characterized in that, The system includes: The disability pattern recognition module is used to acquire the multimodal raw data of the person seeking help, perform disability pattern recognition processing on the multimodal raw data, and obtain the disability pattern recognition result; the multimodal raw data includes video data, audio data, and interactive text data; The personalized emergency rescue plan generation module is used to generate personalized emergency rescue plans based on the disability pattern recognition results, and obtain personalized emergency rescue strategies and resource scheduling plans. The instruction generation and feedback adjustment module is used to generate and issue a first execution instruction and a second execution instruction to the caller terminal and the service terminal respectively, based on the personalized emergency rescue strategy and the resource scheduling scheme, and to obtain and generate adjustment instructions based on the execution feedback from the caller terminal and the service terminal.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images