Medical monitoring alarm system

Abstract

The invention discloses a medical monitoring alarm system, which comprises a multi-modal data acquisition module, a multi-modal fusion analysis module and a three-level intelligent alarm module, and is characterized in that emotion, facial action, language and physiological data of a patient are acquired through a medical camera, a microphone array and a physiological sensor; and carrying out multi-modal fusion analysis by using an improved deep learning model, and realizing three-level intelligent alarm based on comprehensive scoring. The system integrates a dynamic weighted fusion algorithm and a hierarchical response mechanism, significantly improves the accuracy and timeliness of medical monitoring, and is suitable for various nursing scenes.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular discloses a medical monitoring and alarm system. Background Technology

[0002] Medical monitoring and alarm systems, as core technologies in clinical nursing, directly impact patient treatment efficiency and safety. Currently, related technologies in the field of medical monitoring primarily focus on physiological parameter monitoring, image information recognition, and single-modal data analysis, but significant shortcomings exist in multimodal fusion and intelligent early warning. A search of the image information recognition method proposed in publication number CN119181093A reveals that while it involves medical image analysis, it lacks linkage with non-physiological signals such as voice and movement, making it difficult to comprehensively assess the patient's condition. Such single-modal monitoring technologies suffer from insufficient information dimensions, failing to meet the needs of complex clinical scenarios. A search of the multi-level early warning system disclosed in publication number CN119274297A shows that its grading mechanism is not adapted to medical scenarios, lacking dynamic weight allocation and threshold calibration for multimodal data. Existing technologies generally suffer from the individual problems of multimodal data, failing to leverage the synergistic advantages of data fusion.

[0003] To address the aforementioned issues, existing medical monitoring and alarm technologies have significant shortcomings in terms of monitoring dimensions, data fusion, and early warning mechanisms. This invention constructs a multimodal biosignal fusion analysis model, combined with a three-level dynamic threshold early warning mechanism, to achieve comprehensive monitoring and intelligent hierarchical response of patients' emotions, facial movements, language, and physiological indicators. This effectively solves the pain points of existing technologies and provides more efficient and accurate technical support for clinical nursing. Summary of the Invention

[0004] In view of the shortcomings of the prior art, this application provides a medical monitoring and alarm system.

[0005] This application provides a medical monitoring and alarm system, comprising: Multimodal data acquisition module: This module includes a visual acquisition unit, a speech acquisition unit, and a physiological acquisition unit, and is mainly used to acquire patients' visual, speech, and physiological data; Multimodal fusion analysis module: This module includes an emotion recognition unit, a facial motion analysis unit, a language emotion processing unit, and a data fusion algorithm unit. It mainly analyzes the collected visual, speech, and physiological data and obtains the patient's comprehensive status value through the data fusion algorithm. The three-level intelligent alarm module includes a level one yellow alarm, a level two orange alarm, a level three red alarm, and an alarm verification mechanism. It uses three different levels of alarms to display different degrees of abnormality in patients.

[0006] Preferably, the visual acquisition unit uses an 8-megapixel medical camera equipped with the MTCNN face detection algorithm to capture the patient's facial images in real time and identify 68 facial key points for emotion recognition and facial movement analysis; the voice acquisition unit uses a high-sensitivity microphone array and integrates VAD voice activity detection technology to extract 24-dimensional acoustic features for language emotion analysis; the physiological acquisition unit uses wearable sensors to collect the patient's physiological data in real time.

[0007] Preferably, the emotion recognition unit employs an improved ResNet-50 neural network, taking a facial image as input and outputting an emotion probability distribution, calculated using the following formula: ,in For facial images, For feature extraction function, and These are the parameters for the classification layer.

[0008] Preferably, the facial motion analysis unit processes the facial key point sequence through an LSTM network to calculate the eyelid opening and closing degree and the corner of the mouth offset FACS motion unit parameters. The calculation formula is as follows: , ,in For facial key point vectors, This is the hidden layer state. Output the parameters for the action unit.

[0009] Preferably, the language sentiment processing unit employs a BiLSTM-Transformer model to extract semantic features of urgent words, calculated using the following formula: in For query, key, and value vector, represents the dimension.

[0010] Preferably, the data fusion algorithm unit employs a dynamic weighted fusion algorithm, calculating weights based on the confidence levels of each modality. The calculation formula is as follows: ,in, For modal weights, For modal accuracy, For modal feature vectors, This is the comprehensive state vector.

[0011] Preferably, the Level 1 yellow alert is triggered when a single modality is abnormal, with a response time of ≤5 seconds; the Level 2 orange alert is triggered when two modalities are abnormal in tandem, with a response time of ≤10 seconds; and the Level 3 red alert is triggered when three modalities are abnormal or physiological indicators are critical, with a response time of ≤15 seconds.

[0012] Preferably, the three-level red alert is triggered by a comprehensive score S, calculated as follows: S = 0.4E + 0.3F + 0.2L + 0.1P, where E is the emotion score, F is the facial movement score, L is the language score, and P is the physiological indicator score.

[0013] Preferably, the alarm verification mechanism involves the system performing the following verification steps when a level 3 alarm is triggered: Step 1: Calculate the Pearson correlation coefficient between the emotional fluctuation curve and physiological indicators over the past 30 minutes. When the correlation coefficient is greater than 0.7, automatically increase the alert priority. Step 2: Generate a comparison table of multimodal feature sequences before and after the abnormal event, and mark the timestamps and feature value changes of key abnormal points; Step 3: Call the pre-trained clinical decision support model and output the three most likely clinical diagnoses and their corresponding confidence levels based on the current multimodal data.

[0014] Preferably, the multimodal data acquisition module supports adaptive power consumption adjustment. The specific mechanism is as follows: during non-active periods, the camera automatically switches to 15fps infrared low-power mode, the microphone sampling rate is reduced to 8kHz, and the data acquisition interval of the physiological sensor is extended to 30 seconds; the edge computing node integrates basic feature extraction function, and only abnormal data is uploaded to the cloud, thereby reducing network traffic consumption by ≥40%; when any sensor detects abnormal data, the system wakes up all modules to normal working mode within 3 seconds.

[0015] In summary, this application includes at least one of the following beneficial technical effects: 1. Multimodal Fusion Innovation: This system is the first to integrate a patient's emotions, facial movements, language, and physiological indicators to achieve comprehensive health monitoring. It collects multimodal data through an 8-megapixel medical camera, a high-sensitivity microphone array, and wearable sensors, and analyzes the data using an improved deep learning model, resulting in an accuracy rate increase of over 30%. Compared to traditional single-modal monitoring technologies, it can more comprehensively and accurately reflect the patient's physical and mental state.

[0016] 2. Tiered Alarm Mechanism: The system's three-tiered intelligent alarm module triggers Level 1 (yellow), Level 2 (orange), and Level 3 (red) alarms based on different scenarios, such as single-modal anomaly, two-modal co-occurrence, three-modal anomaly, or critical physiological indicators. Response times are controlled to ≤5 seconds, ≤10 seconds, and ≤15 seconds, respectively. This tiered response mechanism covers different risk levels, enabling medical staff to quickly and accurately assess patient conditions, significantly improving medical efficiency and ensuring timely treatment for patients experiencing health problems of varying degrees.

[0017] 3. Strong individual adaptability: The system adopts a dynamic threshold algorithm, which dynamically adjusts the alarm threshold based on the patient's basic data, reducing the false alarm rate to below 6% and the false alarm rate to ≤3%. Compared with the traditional fixed threshold system, it greatly improves the accuracy of early warning, better adapts to the individual differences of different patients, reduces unnecessary alarm interference, and avoids the omission of important health risks. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 It is a schematic diagram of the system structure, showing the connection relationships and functional divisions between the modules.

[0020] Figure 2 This is a flowchart of the processing logic of the medical monitoring and alarm system. Detailed Implementation

[0021] The following is in conjunction with the appendix Figure 1-2 This application will be described in further detail.

[0022] Reference Figure 1 and Figure 2 Taking a 65-year-old male patient with hypertension and coronary heart disease, currently in the postoperative recovery observation period, as an example in the cardiology ward of a tertiary hospital, this patient faces potential risks such as postoperative pain and anxiety. The medical monitoring and alarm system described in this invention is used to monitor him in real time.

[0023] Step 1: The multimodal data acquisition module is used to collect visual, speech, and physiological data from the male patient.

[0024] Visual data acquisition: Using an 8-megapixel medical camera installed in the ward, equipped with the MTCNN face detection algorithm, the system captures patients' facial images in real time. The system acquires 30 frames per second, identifies 68 facial key points, and achieves a positioning accuracy of ±0.3mm, providing a data foundation for subsequent emotion recognition and facial motion analysis.

[0025] Voice data acquisition: A high-sensitivity microphone array integrates VAD (Voice Activity Detection) technology to acquire patient voice signals in real time. The system processes the voice signals in frames and extracts 24-dimensional acoustic features such as MFCC Mel-frequency cepstral coefficients, fundamental frequency, and energy entropy for language sentiment analysis. Physiological data acquisition: Patients wear wearable sensors with a PPG heart rate module accuracy of ±1 bpm, a blood pressure module conforming to AAMI SP10 standards, and a blood oxygen sensor resolution of 0.1%. The sensors collect physiological data such as heart rate, blood pressure, blood oxygen, and body temperature in real time and upload them to the system every 5 seconds.

[0026] Step 2: The multimodal fusion analysis module analyzes the male patient's emotions, facial movements, and verbal emotions, and performs data fusion and comprehensive score calculation on the obtained data.

[0027] Emotion recognition analysis: The acquired facial images are preprocessed to 128×128 pixels and input into an emotion recognition model based on an improved ResNet-50 neural network. According to the formula: The system calculates and outputs the probability distribution of eight basic emotions. At a certain moment, the system identifies the patient's anxiety probability as 0.85, exceeding the 0.8 threshold set for the first-level alarm.

[0028] Facial motion analysis: An LSTM network was used to process facial keypoint sequences and calculate eight FACS motion unit parameters, including eyelid opening and closing, and mouth corner deviation. These parameters were then analyzed using formulas. , Time series analysis revealed that the patient's eyelid opening and closing value remained below 0.4 for a period of time, and the duration reached 1 minute, meeting the criteria for a Level 1 alarm for abnormal facial movements.

[0029] Language emotion processing: A BiLSTM-Transformer model is constructed to analyze the acquired speech signals. According to the formula... The semantic features were extracted, and the patient's speech was identified to contain emergency words such as pain. The emotion classification was determined to be painful emotion, but the conditions for triggering an alarm were not met at this time.

[0030] Data fusion and comprehensive score calculation: A dynamic weighted fusion strategy is adopted, and the weights are calculated based on the confidence scores of each modality. Given that the accuracy rates for emotion recognition are 92.5%, facial motion analysis is 90%, and language emotion processing is 88%, the calculation is based on the formula... Calculate the modal weights. The results are obtained through calculation. Then according to the formula The comprehensive score S is calculated by integrating emotion probability, facial movement parameters, language emotion score, and physiological indicators. Assuming the normal physiological indicator score is 100, the emotion score is 85, the facial movement score is 80, and the language score is 75, then S = 0.34 × 85 × 0.4 + 0.33 × 80 × 0.3 + 0.33 × 75 × 0.2 + 100 × 0.1 = 51.83. This score does not reach the level two alarm threshold of 80, but the single modal abnormality in emotion and facial movement triggers a level one alarm.

[0031] Step 3: The Level 3 intelligent alarm module triggers alarms and responses based on the abnormal data obtained from the data analysis above. Since the patient's anxiety probability is greater than 0.8 for 1 minute and their eyelid opening / closing degree is less than 0.4 for 1 minute, the trigger conditions for a Level 1 yellow alarm are met. Within 3 seconds, the system triggers a flashing bedside lamp and simultaneously displays a pop-up notification at the nurses' station, showing the patient's number, abnormal modality, and related data. Upon receiving the alarm, the nurse quickly goes to the ward to check on the patient and finds that the patient is indeed experiencing anxiety and mild discomfort. The nurse promptly provides psychological reassurance and pain assessment, and provides appropriate nursing care. The patient's condition gradually stabilizes, and the alarm is deactivated.

[0032] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the concept of the invention, they should all fall within the protection scope of the present invention.

[0033] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0034] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A medical monitoring and alarm system, characterized in that, include: Multimodal data acquisition module: This module includes a visual acquisition unit, a speech acquisition unit, and a physiological acquisition unit, and is mainly used to acquire patients' visual, speech, and physiological data; Multimodal fusion analysis module: This module includes an emotion recognition unit, a facial motion analysis unit, a language emotion processing unit, and a data fusion algorithm unit. It mainly analyzes the collected visual, speech, and physiological data and obtains the patient's comprehensive status value through the data fusion algorithm. The three-level intelligent alarm module includes a level one yellow alarm, a level two orange alarm, a level three red alarm, and an alarm verification mechanism. It uses three different levels of alarms to display different degrees of abnormality in patients.

2. The medical monitoring and alarm system according to claim 1, characterized in that, The visual acquisition unit uses an 8-megapixel medical camera equipped with the MTCNN face detection algorithm to capture patients' facial images in real time and identify 68 key facial points for emotion recognition and facial motion analysis. The voice acquisition unit uses a high-sensitivity microphone array and integrates VAD voice activity detection technology to extract 24-dimensional acoustic features for language sentiment analysis. The physiological data acquisition unit collects patients' physiological data in real time through wearable sensors.

3. The medical monitoring and alarm system according to claim 1, characterized in that, The emotion recognition unit employs an improved ResNet-50 neural network, taking a facial image as input and outputting an emotion probability distribution, calculated using the following formula: ,in For facial images, For feature extraction function, and These are the parameters for the classification layer.

4. A medical monitoring and alarm system according to claim 1, characterized in that, The facial motion analysis unit processes facial key point sequences through an LSTM network to calculate the FACS motion unit parameters, including eyelid opening and closing and mouth corner offset. The calculation formula is as follows: , ,in For facial key point vectors, This is the hidden layer state. Output the parameters for the action unit.

5. A medical monitoring and alarm system according to claim 1, characterized in that, The language sentiment processing unit employs a BiLSTM-Transformer model to extract semantic features of urgent words, calculated using the following formula: ， in For query, key, value vector, For dimensions.

6. A medical monitoring and alarm system according to claim 1, characterized in that, The data fusion algorithm unit employs a dynamic weighted fusion algorithm, calculating weights based on the confidence levels of each modality. The calculation formula is as follows: ,in, For modal weights, For modal accuracy, For modal feature vectors, This is the comprehensive state vector.

7. A medical monitoring and alarm system according to claim 1, characterized in that, The Level 1 Yellow Alert is triggered when a single modality is abnormal, with a response time of ≤5 seconds. The Level 2 Orange Alert is triggered when two modalities are abnormal in tandem, with a response time of ≤10 seconds. The Level 3 Red Alert is triggered when three modalities are abnormal or when physiological indicators are critical, with a response time of ≤15 seconds.

8. A medical monitoring and alarm system according to claim 7, characterized in that, The Level 3 red alert is triggered by a comprehensive score S, which is calculated as follows: S = 0.4E + 0.3F + 0.2L + 0.1P, where E is the emotion score, F is the facial movement score, L is the language score, and P is the physiological indicator score.

9. A medical monitoring and alarm system according to claim 1, characterized in that, The alarm verification mechanism requires the system to perform the following verification steps when a Level 3 alarm is triggered: Step 1: Calculate the Pearson correlation coefficient between the emotional fluctuation curve and physiological indicators over the past 30 minutes. When the correlation coefficient is greater than 0.7, automatically increase the alert priority. Step 2: Generate a comparison table of multimodal feature sequences before and after the abnormal event, and mark the timestamps and feature value changes of key abnormal points; Step 3: Call the pre-trained clinical decision support model and output the three most likely clinical diagnoses and their corresponding confidence levels based on the current multimodal data.

10. A medical monitoring and alarm system according to claim 1, characterized in that, The multimodal data acquisition module supports adaptive power consumption adjustment. The specific mechanism is as follows: during non-active periods, the camera automatically switches to 15fps infrared low-power mode, the microphone sampling rate is reduced to 8kHz, and the data acquisition interval of the physiological sensor is extended to 30 seconds; the edge computing node integrates basic feature extraction function, and only abnormal data is uploaded to the cloud, thereby reducing network traffic consumption by ≥40%; when any sensor detects abnormal data, the system wakes up all modules to normal working mode within 3 seconds.

Images