ICU ward monitoring data processing system

Abstract

The invention discloses an ICU (Intensive Care Unit) ward monitoring data processing system, relates to the technical field of medical information, and solves the problems that a comprehensive vital sign stability degree model is difficult to construct through original vital sign data and various characteristic data obtained through analysis, and data of medical equipment and visitors in a ward are difficult to analyze through camera equipment; and corresponding alarm levels are not triggered for different ward data. Comprising the following modules: a data acquisition and transmission module used for acquiring and transmitting various data; the vital sign data analysis module is used for constructing a comprehensive vital sign stability degree model; the medical equipment state identification module is used for constructing a model for identifying normal operation and abnormal operation working states of medical equipment; and the visitor identity recognition and track abnormity judgment module is used for constructing an identity recognition model, generating a track data sequence and recognizing abnormal behaviors of visitors.

Description

Technical Field

[0001] The present invention relates to the field of medical information technology, and particularly to an ICU ward monitoring data processing system. Background Art

[0002] The ICU ward is a special area in the hospital for monitoring and treating patients with severe conditions or unstable vital signs. Patients in the ICU ward usually need to continuously monitor multiple vital sign parameters, such as heart rate, blood pressure, respiratory rate, etc., and continuously monitor the medical equipment and visiting personnel in the ward through multiple camera devices, obtain corresponding data for analysis, construct multiple models to automatically detect the operating status of medical equipment, and analyze and identify the behavior trajectories of visiting personnel. When the model detects an abnormal situation, the system will promptly trigger the corresponding alarm to notify the medical staff for handling.

[0003] Existing ICU ward monitoring systems usually only provide basic monitoring functions, with limited data processing capabilities, and are unable to effectively integrate and analyze a large amount of monitoring data. Therefore, it is difficult to construct a comprehensive vital sign stability degree model through the original vital sign data and various characteristic data obtained by analyzing it, lacking a comprehensive assessment of the patient's vital sign stability state. It is also difficult to process and analyze the data of medical equipment and visiting personnel in the ward through camera devices respectively, lacking models for identifying the normal and abnormal operating states of medical equipment and an abnormal analysis model for the trajectory of visiting personnel, and also lacking corresponding alarm levels for different ward data. Summary of the Invention

[0004] The present invention provides an ICU ward monitoring data processing system to solve the following technical problems:

[0005] Existing ICU ward monitoring systems usually only provide basic monitoring functions, with limited data processing capabilities, and are unable to effectively integrate and analyze a large amount of monitoring data. Therefore, it is difficult to construct a comprehensive vital sign stability degree model through the original vital sign data and various characteristic data obtained by analyzing it, lacking a comprehensive assessment of the patient's vital sign stability state. It is also difficult to process and analyze the data of medical equipment and visiting personnel in the ward through camera devices respectively, lacking models for identifying the normal and abnormal operating states of medical equipment and an abnormal analysis model for the trajectory of visiting personnel, and also lacking corresponding alarm levels for different ward data.

[0006] To solve the above problems, the present invention provides an ICU ward monitoring data processing system, including the following modules:

[0007] Data Acquisition and Transmission Module: Real-time collect the vital sign data of patients in the ICU ward through a variety of monitoring devices and sensors, and real-time obtain the image data of the ward through camera devices. Transmit the collected data to the Data Processing and Analysis Module through a wireless network, and encrypt the data using encryption technology;

[0008] Vital Sign Data Analysis Module: Analyze the data collected by a variety of medical devices to obtain the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate. Construct a comprehensive vital sign stability degree model, and output the comprehensive vital sign stability degree score of the patient through the trained model;

[0009] Medical Device Status Recognition Module: Use the appearance features and indicator light status features in the images of normal and abnormal medical devices as training data, and use the convolutional neural network framework to construct a model for identifying the normal and abnormal operating states of medical devices, and automatically detect the operating state of medical devices;

[0010] Visitor Identity Recognition and Trajectory Abnormality Judgment Module: Construct an identity recognition model based on visitors. Input the preprocessed video frame images containing visitors into the YOLO model. The model outputs the position information of the visitors, associates it with the timestamp to generate a trajectory data sequence. Use the visitor trajectory data as the input observation sequence of the HMM, and judge whether the trajectory conforms to the normal trajectory pattern according to the set probability threshold. Construct a multi-dimensional ward parameter evaluation value and a multi-dimensional ward parameter threshold, and compare them.

[0011] Further, the real-time collection of the vital sign data of patients in the ICU ward through a variety of monitoring devices and sensors, and the real-time obtaining of the image data of the ward through monitoring devices, include the following steps:

[0012] Real-time collect the vital sign data of patients in the ICU ward through a heart rate monitor, an electronic blood pressure monitor, and a ventilator. The vital sign data includes heart rate, blood pressure, and respiratory rate; Install multiple camera devices at the entrance, internal key areas, and passage positions in the ICU ward, adjust the shooting angle, resolution, and frame rate parameters of the camera devices, and real-time obtain the image data of the ward through the camera devices. The image data includes the operating state data of various medical devices and visitor data in real-time monitoring;

[0013] Among them, the internal key areas include the hospital bed area, medical device storage area, drug storage area, and sterile item storage area.

[0014] Further, the transmission of the collected data to the Data Processing and Analysis Module through a wireless network and the encryption of the data using encryption technology include the following steps:

[0015] Configure the network for the acquisition device. At the acquisition device end, perform preliminary preprocessing operations on the acquired data, turn on the wireless communication module of the acquisition device, search for and connect to the pre-configured wireless network, and send the preprocessed data via the wireless network. Run a data receiving program on the server to enable it to receive wireless data transmissions from the acquisition device. According to the security requirements and performance requirements of the data, use a symmetric encryption algorithm to encrypt the data, store the decrypted data in the database, and regularly back up the stored data.

[0016] Furthermore, analyzing the data collected from multiple medical devices to obtain the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate includes the following steps:

[0017] Determine the starting point of the heart rate data, set the moving window size to 10 minutes, starting from the starting point of the heart rate data, intercept the data for the first 10 minutes, and calculate the maximum and minimum heart rates to obtain the heart rate fluctuation range for this window. Slide the window backward by one data point for interception and calculation, and repeat the above process until the entire heart rate data set is traversed to obtain a continuous sequence of heart rate fluctuation ranges;

[0018] Determine the time period for blood pressure monitoring, measure the patient's blood pressure within the selected time period, and record each measurement result to form a blood pressure data set for this time period. In the blood pressure data for each collected time period, find the highest and lowest values of systolic and diastolic blood pressures respectively, and the difference obtained is the blood pressure fluctuation range for this time period. Calculate the fluctuation ranges of diastolic and systolic blood pressures respectively and record them;

[0019] Use a ventilator to record the number of breaths per time period, record the corresponding respiratory rate data at a determined time interval through data acquisition software, calculate the difference in the corresponding respiratory rates between each pair of adjacent time points in the data set, take the absolute value and then calculate the average value. Calculate the square of the difference between the absolute value of the difference in respiratory rates between each adjacent time point and the average value, add up the squared values and divide by the total number of differences, and then take the square root to obtain the standard deviation, which is used as the respiratory rate change rate.

[0020] Furthermore, constructing a comprehensive vital sign stability degree model and outputting the comprehensive vital sign stability degree score of the patient through the trained model includes the following steps:

[0021] Obtain the vital sign data of the patient's heart rate, blood pressure, and respiratory rate from the medical record, clean and standardize the data. Through the original vital sign data and the calculated heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate, substitute the processed data of the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate into the calculation of the Pearson correlation coefficient, and obtain r HR-BP , r HR-RR and r BP-RR . Organize the calculated Pearson correlation coefficients into a matrix, denoted as [r HR-BP , r HR-RR , r BP-RR , which is used as the feature input of the model. Select a framework based on linear regression to construct a comprehensive vital sign stability model;

[0022] For each sample, calculate r HR-BP , r HR-RR and r BP-RR respectively. The expert gives the comprehensive vital sign stability score corresponding to each sample according to the clinical criteria. Make a data set table with r HR-BP , r HR-RR , r BP-RR and the score of each sample. Use the part of the data set for the training set to train the model, and output the comprehensive vital sign stability score through the trained model.

[0023] Furthermore, use the convolutional neural network framework to construct a model for identifying the normal and abnormal operating states of medical devices, including the following steps:

[0024] Use a camera device to take pictures of the medical device in normal and abnormal states, take pictures from different angles and distances, obtain the full - range appearance features and indicator light state features of the medical device, screen the images, and classify and organize the images according to the normal and abnormal operating states to construct a folder structure;

[0025] Use an image annotation tool to annotate the appearance features and indicator light state features in the normal and abnormal medical device images to generate corresponding annotation files. Among them, the appearance features include the shell, buttons, and interfaces of the medical device, and the indicator light state features include color and shape;

[0026] Use the convolutional neural network framework to construct a model for identifying the normal and abnormal operating states of medical devices. Use the annotated image data as the input for training, deploy the trained model in actual applications, automatically detect the operating state of the medical device, and output the working state of the medical device as normal or abnormal;

[0027] When the model outputs that the working status of the medical device is normal, the system continuously monitors the ward; when the model outputs that the working status of the medical device is abnormal, the system automatically triggers a secondary alarm to notify the medical staff for handling.

[0028] Furthermore, the method for constructing an identity recognition model based on visitors includes inputting the preprocessed video frame images containing visitors into the YOLO model. The model outputs the position information of the visitors, which is associated with the timestamp to generate a trajectory data sequence, and includes the following steps:

[0029] Collect the face images of known visitors from the entrance monitoring video of the ICU ward and the photos taken at the visitor registration office. Annotate the collected face images, and the annotation information includes the name, ID number, image acquisition time and location of the visitors. Organize the annotated images according to the rules, perform preprocessing operations on the organized image data. After initializing the identity recognition model using the deep learning framework, input the organized annotated image dataset for training into the initialized model. According to the predicted probability distribution output by the model, determine the known visitor with the highest probability value as the matching result;

[0030] Perform preprocessing operations on the video surveillance images. Use the YOLO object detection algorithm in image recognition technology to train the video images containing visitors as training data. Input the preprocessed video frame images into the trained YOLO model. The model outputs the position information of the visitors detected in each frame image. In the video sequence, by associating the detection positions of the visitors between consecutive frames, assign a unique identification ID to each tracked visitor. Associate the position coordinates of the visitors extracted from the video with the timestamp to generate a trajectory data sequence for each visitor.

[0031] Furthermore, the method for using the visitor trajectory data as the input observation sequence of the HMM, judging whether the trajectory conforms to the normal trajectory pattern according to the set probability threshold, constructing a multi-dimensional ward parameter evaluation value and a multi-dimensional ward parameter threshold, and comparing them, includes the following steps:

[0032] Collect the trajectory data of the visitors, perform cleaning operations on the trajectory data, label the trajectory data of the visitors as normal or abnormal, extract features from the original trajectory data, including position coordinates, speed, acceleration, and stay time features. Define the state set of the model according to the behavior patterns of the visitors and the actual situation of the ICU ward. Among them, the states are divided into inside the ward, in the corridor, and in the rest area. Determine the observation set of the model, set the probabilities of the model being in each state at the initial moment, the probabilities of transitioning from one state to another state, and the probabilities of generating each observation value in each state. Estimate the HMM parameters through the labeled trajectory training set data, including the initial state probability distribution, state transition probability distribution, and observation probability distribution. Take the new visitor trajectory data as the input observation sequence of the HMM, and use the forward algorithm of the trained HMM to calculate the probability that the trajectory belongs to the normal trajectory pattern. Judge whether the trajectory conforms to the normal trajectory pattern according to the set probability threshold;

[0033] If the output probability is lower than or equal to the set probability threshold, it is considered that the visitor has an abnormal trajectory behavior; otherwise, it is considered a normal trajectory behavior;

[0034] Weight and sum the comprehensive vital sign stability degree score of the patient, the corresponding numerical values of the medical device working state rules, and the output probability value of the visitor behavior model to obtain the multi-dimensional ward parameter evaluation value;

[0035]

[0036] Among them, D represents the multi-dimensional ward parameter evaluation value, S represents the comprehensive vital sign stability degree score of the patient, E total represents the total number of medical devices, E abnormal represents the total number of medical devices with abnormal working states, P represents the output probability value of the visitor behavior model, ω 1 、ω 2 and ω 3 represent the corresponding weight coefficients;

[0037] Determine the stability threshold, device threshold, and probability threshold according to the score output by the comprehensive vital sign stability degree model, the distribution of the medical device state recognition model, and clinical experience, and weight and sum them to obtain the multi-dimensional ward parameter threshold;

[0038] When the multi-dimensional ward parameter evaluation value > the multi-dimensional ward parameter threshold, it indicates that the patient and environmental states in the ward are normal; otherwise, it indicates that the patient and environmental states in the ward are abnormal, and the system triggers a first-level alarm.

[0039] The beneficial effects of the present invention:

[0040] The present invention comprehensively uses the Pearson correlation coefficient to relate the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate, and constructs a comprehensive vital sign stability degree model with the original multiple vital sign data, heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate as features, automatically scoring the health stability degree of new patients, which can more comprehensively reflect the vital sign stability degree of patients, realizing the early detection of potential health problems and providing a basis for further examinations and early interventions;

[0041] The present invention uses the YOLO object detection algorithm combined with the HMM model to detect and track the visitors entering the ICU ward, obtains the position information of the visitors in each frame image, correlates it between consecutive frames, records the movement path of each visitor, and when the trajectory of the visitor does not conform to the normal mode, the system can detect it in time and mark it as an abnormal trajectory, and trigger a first-level alarm. The entire trajectory monitoring and abnormal judgment process is automatically completed by the trained model, realizing 24-hour uninterrupted real-time monitoring, greatly saving labor costs, and allowing medical staff to concentrate more on the care of patients without continuously staring at the monitoring screen to identify the abnormal behaviors of visitors. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of the implementation module of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0044] Please refer to Figure 1 As shown, the present invention is an ICU ward monitoring data processing system, including the following modules:

[0045] Data acquisition and transmission module: Real-time collects the vital sign data of patients in the ICU ward through a variety of monitoring devices and sensors, real-time obtains the image data of the ward through a camera device, transmits the collected data to the data processing and analysis module through a wireless network, and encrypts the data using encryption technology;

[0046] Vital sign data analysis module: Analyzes the data collected by a variety of medical devices, obtains the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate, constructs a comprehensive vital sign stability degree model, and outputs the comprehensive vital sign stability degree score of the patient through the trained model;

[0047] Medical device status recognition module: Using the appearance features and indicator light status features in the images of normal and abnormal medical devices as training data, a model for identifying the normal and abnormal operating states of medical devices is constructed using a convolutional neural network framework to automatically detect the operating state of medical devices.

[0048] Visitor identity recognition and abnormal trajectory judgment module: Construct an identity recognition model based on visitors. Input the preprocessed video frame images containing visitors into the YOLO model. The model outputs the position information of the visitors, which is associated with the time stamp to generate a trajectory data sequence. The visitor trajectory data is used as the input observation sequence of the HMM, and whether the trajectory conforms to the normal trajectory pattern is judged according to the set probability threshold.

[0049] Specifically, attach the electrode pads of the heart rate monitor to the corresponding parts of the patient's chest in the correct position, wrap the cuff of the electronic sphygmomanometer around the patient's upper arm, and the ventilator monitors the patient's breathing rate in real time through the built-in sensor. Install a camera device to capture the physical condition of the patient in the hospital bed and the images of the medical devices operating around the bed and the visitors. Substitute the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate into the calculation of the Pearson correlation coefficient to obtain the Pearson correlation coefficients between each pair respectively. Organize the calculated Pearson correlation coefficients into a matrix. At the same time, experts give the comprehensive vital sign stability degree score corresponding to each sample according to clinical standards, and select a framework based on linear regression to construct a comprehensive vital sign stability degree model. Use the selected convolutional neural network framework to construct a model for identifying the normal and abnormal operating states of medical devices. After initializing the model using the deep learning framework, input the sorted labeled image data set for training into the initialized identity recognition model. According to the predicted probability distribution output by the model, determine the known visitor with the highest probability value as the matching result. Use the YOLO object detection algorithm in image recognition technology to train the video images containing visitors as training data. Input the preprocessed video frame images into the trained YOLO model. The model outputs the position information of the visitors detected in each frame image, which is combined with the time stamp data to form a trajectory sequence. Use the forward algorithm of the trained HMM to calculate the probability that the trajectory belongs to the normal trajectory pattern. Set a probability threshold according to the statistics of the normal trajectory data to judge whether the trajectory conforms to the normal trajectory pattern. If the probability is lower than or equal to the threshold, it is considered that the visitor has an abnormal trajectory behavior; otherwise, it is considered a normal trajectory behavior.

[0050] In one embodiment of the present invention, the method for collecting the vital sign data of patients in the ICU ward in real time through a variety of monitoring devices and sensors and obtaining the image data of the ward in real time through monitoring devices includes the following steps:

[0051] Collect the vital sign data of patients in the ICU ward in real time through a heart rate monitor, an electronic blood pressure monitor, and a ventilator. The vital sign data includes heart rate, blood pressure, and respiratory rate. Install multiple camera devices at the entrance, internal key areas, and passage positions in the ICU ward, adjust the shooting angles, resolutions, and frame rate parameters of the camera devices, and obtain the image data of the ward in real time through the camera devices. The image data includes the operation status data of various medical devices and the visitor data monitored in real time;

[0052] Among them, the internal key areas include the hospital bed area, the medical device storage area, the medicine storage area, and the sterile item storage area.

[0053] Specifically, attach the electrode patches of the heart rate monitor to the corresponding parts of the patient's chest in the correct position, start the corresponding function to collect the electrocardiogram signal of the patient in real time, and convert it into heart rate data through the built-in sensors and algorithms, and automatically record the heart rate value per minute. Ensure that the electronic blood pressure monitor is in a normal working state, wrap its cuff around the upper arm of the patient, measure and record the patient's blood pressure. For patients who need long-term blood pressure monitoring, set the timing measurement function to automatically measure the blood pressure once every hour. Set the parameters of the ventilator according to the patient's condition and doctor's diagnosis. The ventilator monitors the patient's respiratory rate in real time through the built-in sensors and displays the results on the display screen. According to the layout and monitoring requirements of the ICU ward entrance, the height and angle that can clearly capture the front features of the personnel entering and leaving the ward, such as about 2.5 - 3 meters above the ground, and the angle is slightly tilted downward to ensure that the faces and body contours of the personnel are captured. Install camera devices in the key areas inside the ward to ensure that the physical condition of the patient on the hospital bed and the operation of the medical devices around the hospital bed can be completely captured. Synchronize the data collected from different devices according to a unified time standard.

[0054] In one embodiment of the present invention, the method for transmitting the collected data to the data processing and analysis module through a wireless network and encrypting the data using encryption technology includes the following steps:

[0055] Perform network configuration on the acquisition device. At the acquisition device end, perform preliminary preprocessing operations on the collected data, turn on the wireless communication module of the acquisition device, search for and connect to a pre-configured wireless network, and send the preprocessed data through the wireless network. Run a data reception program on the server so that it can receive wireless data transmissions from the acquisition device. According to the security requirements and performance requirements of the data, use a symmetric encryption algorithm to encrypt the data, store the decrypted data in the database, and regularly back up the stored data.

[0056] Specifically, enter the system settings interface of the acquisition device, find the network configuration option, turn on the wireless communication module of the acquisition device, search for and connect to a pre-configured wireless network, perform corresponding data preprocessing procedures according to the data types collected by the acquisition device. The preprocessed data is encapsulated in a predetermined data format. For example, vital sign data such as heart rate, blood pressure, and respiratory rate is encapsulated into a data packet, including data type, collection time, and numerical information. Use a symmetric encryption algorithm to encrypt the encapsulated data. For example, use the AES algorithm to generate a key on the acquisition device and use this key to encrypt the data. The encrypted data will be converted into ciphertext form to ensure the security of the data during transmission. Through the wireless communication module on the acquisition device, send the encrypted data. Configure the network parameters of the server, open the firewall on the server, set the port forwarding rules, and run a data reception program on the server. When the acquisition device sends data, the data reception program on the server side can accept the connection request and data packet from the acquisition device, use the same symmetric encryption algorithm and key as the acquisition device to decrypt the received encrypted data, convert the ciphertext into plaintext data, store the decrypted data in the database, and establish a corresponding database table structure according to the data type. For example, create a table for vital sign data, including fields such as medical record number, collection time, heart rate, blood pressure, and respiratory rate.

[0057] In one embodiment of the present invention, the analysis of the data collected by multiple medical devices to obtain the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate includes the following steps:

[0058] Determine the starting point of the heart rate data, set the moving window size to 10 minutes, start from the starting point of the heart rate data, intercept the data for the first 10 minutes, and calculate the maximum and minimum heart rates to obtain the heart rate fluctuation range of this window. Slide the window backward by one data point for interception and calculation, and repeat the above process until the entire heart rate data set is traversed to obtain a continuous sequence of heart rate fluctuation ranges;

[0059] Determine the time period for blood pressure monitoring. Measure the patient's blood pressure within the selected time period, and record each measurement result to form a blood pressure dataset for this time period. Among the blood pressure data for each collected time period, find the maximum and minimum values of the systolic blood pressure and diastolic blood pressure respectively. The difference obtained is the blood pressure fluctuation range for this time period. Calculate the fluctuation ranges of the diastolic blood pressure and systolic blood pressure respectively and record them;

[0060] Use a ventilator to record the breathing rate for each time period. Record the corresponding respiratory frequency data at the determined time intervals through data acquisition software. Calculate the difference in the corresponding respiratory frequencies between each pair of adjacent time points in the dataset, take the absolute value, and then calculate the average value. Calculate the square of the difference between the absolute value of the difference in the respiratory frequencies between each adjacent time point and the average value. Add up the squared values and divide by the total number of differences, and then take the square root to obtain the standard deviation, which is used as the respiratory frequency change rate.

[0061] Specifically, according to the research purpose and data characteristics, set a moving window size of 10 minutes. Starting from the starting point of the heart rate data, intercept the data according to the set window size. Calculate the maximum and minimum values of the heart rate within each window to obtain the heart rate fluctuation range for this window. Then, slide the window backward by one data point and repeat the above calculation process until the entire heart rate dataset is traversed to obtain continuous heart rate fluctuation ranges; Determine the time period for blood pressure monitoring. For example, from 8 to 10 am, from 2 to 4 pm, etc. Within each selected time period, use a calibrated electronic blood pressure monitor to measure the blood pressure and record the measurement results. Among the collected data, find the maximum and minimum values of the systolic blood pressure and diastolic blood pressure within each time period respectively. Subtract the minimum value from the maximum value in each time period, and the obtained difference is used as the blood pressure fluctuation range for this time period. Calculate the fluctuation ranges of the systolic blood pressure and diastolic blood pressure respectively. Use a ventilator to record the breathing rate for each time period. According to the determined time intervals, record the corresponding respiratory frequency data through data acquisition software. Arrange the collected respiratory frequency data in chronological order to form a complete dataset. For each pair of adjacent time points in the dataset, calculate the corresponding difference in respiratory frequencies. Take the absolute value of each calculated difference. Add up the absolute values of the differences in respiratory frequencies for all adjacent time points and divide by the total number of differences to obtain the average value. Calculate the square of the difference between the absolute value of the difference in the respiratory frequencies between each adjacent time point and the average value. Add up the squared values and divide by the total number of differences, and then take the square root to obtain the standard deviation as the respiratory frequency change rate.

[0062] In one embodiment of the present invention, for constructing the comprehensive vital sign stability degree model and outputting the comprehensive vital sign stability degree score of the patient through the trained model, the following steps are included:

[0063] Obtain the vital sign data of the patient's heart rate, blood pressure, and respiratory rate from the medical record, clean and standardize the data. Through the original vital sign data and the calculated heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate, substitute the processed data of the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate into the calculation of the Pearson correlation coefficient, and obtain r HR-BP , r HR-RR and r BP-RR . Organize the calculated Pearson correlation coefficients into a matrix, denoted as [r HR-BP , r HR-RR , r BP-RR , which is used as the feature input of the model. Select a framework based on linear regression to construct a comprehensive vital sign stability model;

[0064] For each sample, calculate r HR-BP , r HR-RR and r BP-RR respectively. Let the expert give the comprehensive vital sign stability score corresponding to each sample according to the clinical criteria. Make a data set table with r HR-BP , r HR-RR , r BP-RR and the score of each sample. Use the part of the data set for the training set to train the model, and output the comprehensive vital sign stability score through the trained model.

[0065] Specifically, obtain the vital sign data of the patient's heart rate, blood pressure, and respiratory rate from the medical record, and remove the data points with obvious errors or abnormalities, such as negative heart rate or blood pressure values, respiratory rates that do not conform to the physiological range, etc. Process the outliers and standardize the data such as heart rate, blood pressure, and respiratory comments. Obtain the continuous heart rate fluctuation range sequence, blood pressure fluctuation range, and respiratory rate change rate respectively according to the previous calculation method. Substitute the processed heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate into the calculation of the Pearson correlation coefficient to obtain the Pearson correlation coefficients between each pair. Organize the calculated Pearson correlation coefficients into a matrix. At the same time, the expert gives the comprehensive vital sign stability degree score corresponding to each sample according to the clinical standard. For each sample, calculate the Pearson correlation coefficient between heart rate and blood pressure, the Pearson correlation coefficient between heart rate and respiratory rate, and the Pearson correlation coefficient between blood pressure and respiratory rate respectively, and make them into a data set table with the comprehensive vital sign stability degree score of the sample. Each row represents a sample, such as a patient or a time period, and each column represents a feature. Select a framework based on linear regression to construct a comprehensive vital sign stability degree model, train the model with the data set part used for model training, and output the comprehensive vital sign stability degree score through the trained model. For the new patient data, first calculate the heart rate fluctuation range, blood pressure fluctuation range, and respiratory rate change rate according to the above steps, and then substitute these data into the trained model to obtain the comprehensive vital sign stability degree score.

[0066] In one embodiment of the present invention, the steps of constructing a model for identifying the normal and abnormal operating states of a medical device using a convolutional neural network framework include the following:

[0067] Use a camera device to take images of the medical device in normal and abnormal states, take the images from different angles and distances to obtain the omnidirectional appearance features and indicator light state features of the medical device, screen the images, and classify and organize the images according to the normal and abnormal operating states to construct a folder structure;

[0068] Use an image annotation tool to annotate the appearance features and indicator light state features in the normal and abnormal medical device images to generate corresponding annotation files. Among them, the appearance features include the shell, buttons, and interfaces of the medical device, and the indicator light state features include color and shape;

[0069] Use a convolutional neural network framework to construct a model for identifying the normal and abnormal operating states of a medical device, use the annotated image data as input for training, deploy the trained model in actual use, automatically detect the operating state of the medical device, and output the working state of the medical device as normal or abnormal;

[0070] When the model outputs that the working state of the medical device is normal, the system continuously monitors the ward; when the model outputs that the working state of the medical device is abnormal, the system automatically triggers a secondary alarm to notify the medical staff to handle it.

[0071] Specifically, use a camera device to capture images of the medical device in normal and abnormal operating states from different angles and distances, screen the captured images, remove images with poor quality, duplicates or irrelevancies, classify and organize the images according to normal and abnormal operating states, and construct a folder structure. For example, create two main folders, normal and abnormal, and create sub-folders under each main folder according to the type of medical device. Select a suitable image annotation tool and annotate the appearance features in the normal and abnormal medical device images, including the housing, buttons and interfaces of the medical device. After assigning a unique identifier to each annotation, annotate the status features of the indicator lights, including color and shape. Use different colors and shapes to distinguish different indicator light states. For example, green indicates normal, red indicates abnormal, a circle indicates a power indicator light, and a square indicates a fault indicator light. Similarly, assign a unique identifier to each annotation. Use the selected convolutional neural network framework to construct a model for identifying the normal and abnormal operating states of the medical device. The input of the model is the annotated image data, and the output is whether the working state of the medical device is normal or abnormal. Input the training set part of the annotated image data into the model for training, deploy the trained model in actual use, and input the medical device images collected by the camera device into the deployed model for automatic detection. The model outputs the working state of the medical device. If the model outputs that the working state of the medical device is normal, the system continuously monitors the ward, continues to collect images and input them into the model for detection. If the model outputs that the working state of the medical device is abnormal, the system automatically triggers a secondary alarm and notifies the medical staff to handle it through sound.

[0072] In one embodiment of the present invention, the construction of an identity recognition model based on visitors, input the preprocessed video frame images containing visitors into the YOLO model, the model outputs the position information of the visitors, and associates it with the time stamp to generate a trajectory data sequence, including the following steps:

[0073] Collect face images of known visitors from the entrance surveillance video of the ICU ward and the photos taken at the visitor registration desk. Label the collected face images, and the labeling information includes the name, ID number, image acquisition time and location of the visitors. Organize the labeled images according to the rules, perform preprocessing operations on the organized image data, initialize the identity recognition model using the deep learning framework, and then input the organized labeled image dataset for training into the initialized model. According to the predicted probability distribution output by the model, determine the known visitor with the highest probability value as the matching result;

[0074] Perform preprocessing operations on the video surveillance images. Use the YOLO object detection algorithm in image recognition technology to train the video images containing visitors as training data. Input the preprocessed video frame images into the trained YOLO model. The model outputs the position information of the visitors detected in each frame image. In the video sequence, by associating the detection positions of the visitors between consecutive frames, assign a unique identification ID to each tracked visitor, and associate the position coordinates of the visitors extracted from the video with the timestamps to generate a trajectory data sequence for each visitor.

[0075] Specifically, face images are obtained from the entrance surveillance video of the ICU ward and the visitor registration office at different times, under different lighting conditions, and with different expressions. The collected face images are labeled, and the labeling information includes the name, ID number, image acquisition time, and location of the visitor. The labeled images are sorted according to rules, for example, classified and stored by the name of the visitor, and all images of the visitor are stored in each folder. Preprocessing operations are performed on the image data, including unifying the image format, adjusting the size, normalizing, and data augmentation. After initializing the model using a deep learning framework, the sorted labeled image dataset for training is input into the initialized model. According to the predicted probability distribution output by the model, the known visitor with the highest probability value is determined as the matching result; the YOLO object detection algorithm in image recognition technology is used to train the video images containing visitors as training data. The preprocessed video frame images are input into the trained YOLO model, and the model outputs the position information of the visitors detected in each frame image, usually represented in the form of a bounding box, including the upper left coordinates (xmin, ymin) and the lower right coordinates (xmax, ymax) of the bounding box, as well as the confidence score. A confidence threshold is set, for example, 0.5. When the confidence score of the detection box is higher than this threshold, the box contains a visitor. In the video sequence, tracking is achieved by associating the detection positions of visitors between consecutive frames. A unique identification ID is assigned to each tracked visitor to distinguish different individuals in the entire video sequence. For example, when a visitor is detected in the current frame and a certain tracked target in the previous frame is matched within a reasonable range according to the prediction of the Kalman filter, the ID of the target remains unchanged; if it is a newly emerged target, a new ID is assigned. The position coordinates of the visitors extracted from the video are associated with the timestamps, and the timestamps can be calculated based on the frame rate and frame number of the video. For example, if the video frame rate is 30 frames per second, the timestamp of the first frame is 0, and the timestamp of the second frame is 1 / 30 second. A trajectory data sequence is generated for each visitor. For example, for visitor A, its trajectory data may be [(0, (x1, y1)), (1 / 30, (x2, y2)), (2 / 30, (x3, y3)),...].

[0076] In one embodiment of the present invention, the trajectory data of the visitor is used as the input observation sequence of the HMM, and whether the trajectory conforms to the normal trajectory pattern is judged according to the set probability threshold. A multi-dimensional ward parameter evaluation value and a multi-dimensional ward parameter threshold are constructed and compared, including the following steps:

[0077] Collect the trajectory data of the visitors, perform cleaning operations on the trajectory data, label the trajectory data of the visitors as normal or abnormal, extract features from the original trajectory data, including position coordinates, speed, acceleration, and stay time features. Define the state set of the model according to the behavior patterns of the visitors and the actual situation of the ICU ward. Among them, the states are divided into inside the ward, in the corridor, and in the rest area. Determine the observation set of the model, set the probabilities of the model being in each state at the initial moment, the probabilities of transitioning from one state to another state, and the probabilities of generating each observation value in each state. Estimate the HMM parameters through the labeled trajectory training set data, including the initial state probability distribution, state transition probability distribution, and observation probability distribution. Take the new visitor trajectory data as the input observation sequence of the HMM, and use the forward algorithm of the trained HMM to calculate the probability that the trajectory belongs to the normal trajectory pattern. Judge whether the trajectory conforms to the normal trajectory pattern according to the set probability threshold;

[0078] If the output probability is lower than or equal to the set probability threshold, it is considered that the visitor has abnormal trajectory behavior; otherwise, it is considered normal trajectory behavior;

[0079] Weight and sum the comprehensive vital sign stability degree score of the patient, the corresponding numerical value of the medical device working state rule, and the output probability value of the visitor behavior model to obtain the multi-dimensional ward parameter evaluation value;

[0080]

[0081] Among them, D represents the multi-dimensional ward parameter evaluation value, S represents the comprehensive vital sign stability degree score of the patient, E total represents the total number of medical devices, E abnormal represents the total number of medical devices with abnormal working states, P represents the output probability value of the visitor behavior model, ω 1 、ω 2 and ω 3 represent the corresponding weight coefficients;

[0082] Determine the stability threshold, device threshold, and probability threshold according to the score output by the comprehensive vital sign stability degree model, the distribution of the medical device state recognition model, and clinical experience, and weight and sum them to obtain the multi-dimensional ward parameter threshold;

[0083] When the multi-dimensional ward parameter evaluation value > the multi-dimensional ward parameter threshold, it means that the patient and environmental states in the ward are normal; otherwise, it means that the patient and environmental states in the ward are abnormal, and the system triggers a first-level alarm.

[0084] Specifically, according to the behavior patterns of the visitors and the actual situation of the ICU ward, define the state set of the model. Among them, the states are divided into inside the ward, in the corridor, and in the rest area. Determine the observation set of the model, that is, the possible observed values of the visitors in each state. For example, the observed values can be the specific position coordinates, residence time, etc. of the visitors. Collect a large amount of historical data about the system. The data contains the frequency information in different states. Calculate the number of times each state appears in the historical data. Divide the number of times each state appears by the total number of data to obtain the initial probability of this state. For example, if state A appears 200 times in 1000 historical observations, then the initial probability of state A is 200 / 1000 = 0.2. Observe the state changes of the system at different time points, record the number of times transferred from one state to another state. For each pair of states i and j, calculate the ratio of the number of times transferred from state i to state j to the total number of transfers starting from state i, which is the transition probability Pij from state i to state j. Divide the statistical data into different categories, count the number of data points in each category, and calculate the ratio of the number of data points in each category to the total number of data points as the observation probability on this category. For example, for continuous observed data, it can be divided into several small intervals, and count the proportion of data points falling in each small interval. Estimate the HMM parameters through the labeled trajectory data for training, including the initial state probability distribution, state transition probability distribution, and observation probability distribution. Take the new detected personnel trajectory data as the input observation sequence of the HMM, and use the forward algorithm of the trained HMM to calculate the probability that this trajectory belongs to the normal trajectory pattern. According to the statistics of the normal trajectory data, such as the mean and standard deviation, set a preliminary probability threshold to judge whether this trajectory conforms to the normal trajectory pattern. For example, the threshold can be set as the mean minus several times the standard deviation. If the probability is lower than or equal to the threshold, it is considered that the visitor has an abnormal trajectory behavior; otherwise, it is considered a normal trajectory behavior. For the corresponding medical device working state rules, determine that the total number of medical devices corresponding to the patient is n, the number of devices in the normal working state, and the number of devices in the abnormal working state. If all the medical device working states are normal, assign a value of 10. If there is 1 medical device with an abnormal working state, multiply 10 by (n - 1) / n to get the corresponding assignment, and so on. If there are 2 medical devices with abnormal working states, multiply 10 by (n - 2) / n to get the corresponding assignment;

[0085] Among them, the value of ω 1 is 0.4, and the values of ω 2 and ω 3 are both 0.3.

[0086] The above embodiments are only used to illustrate the technical method of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An ICU ward monitoring data processing system, characterized in that: Includes the following modules: Data collection and transmission module: collects the vital signs data of patients in the ICU ward in real time through a variety of monitoring equipment and sensors, obtains the image data of the ward in real time through the camera equipment, transmits the collected data to the data processing and analysis module through the wireless network, and encrypts the data using encryption technology; Vital sign data analysis module: Analyze the data collected by various medical devices, obtain the fluctuation range of heart rate, blood pressure and respiratory rate, build a comprehensive vital sign stability model, and output the patient's comprehensive vital sign stability score through the trained model; Medical equipment status recognition module: The appearance features and indicator light status features in normal and abnormal medical equipment images are used as training data, and a convolutional neural network framework is used to build a model to identify the normal and abnormal operating states of medical equipment, and automatically detect the operating state of medical equipment; Visitor identity recognition and trajectory abnormality judgment module: Build an identity recognition model based on visitors, input the preprocessed video frame images containing visitors into the YOLO model, the model outputs the location information of the visitors, associates it with the timestamp to generate a trajectory data sequence, and uses the visitor trajectory data as the input observation sequence of HMM. According to the set probability threshold, it is judged whether the trajectory conforms to the normal trajectory pattern, and the multidimensional ward parameter evaluation value and multidimensional ward parameter threshold are constructed and compared.

2. An ICU ward monitoring data processing system according to claim 1, characterized in that: The method of collecting the vital sign data of patients in the ICU ward in real time through a variety of monitoring devices and sensors, and obtaining the image data of the ward in real time through the monitoring device, includes the following steps: The vital signs data of patients in the ICU ward are collected in real time through heart rate monitors, electronic blood pressure monitors and ventilators. The vital signs data include heart rate, blood pressure and respiratory rate. Multiple cameras are installed at the entrance, key internal areas and passageways of the ICU ward. The shooting angle, resolution and frame rate parameters of the cameras are adjusted to obtain the image data of the ward in real time through the cameras. The image data includes real-time monitoring of the operating status data of various medical equipment and visitor data. Among them, the key internal areas include the bed area, medical equipment storage area, medicine storage area and sterile item storage area.

3. An ICU ward monitoring data processing system according to claim 1, characterized in that: The collected data is transmitted to the data processing and analysis module via a wireless network, and encryption technology is used to encrypt the data, including the following steps: Configure the network for the collection device, perform preliminary preprocessing on the collected data at the collection device end, turn on the wireless communication module of the collection device, search and connect to the pre-configured wireless network, send the preprocessed data through the wireless network, run the data receiving program on the server so that it can receive wireless data transmission from the collection device, encrypt the data using a symmetric encryption algorithm based on the data security and performance requirements, store the decrypted data in the database, and back up the stored data regularly.

4. The ICU ward monitoring data processing system according to claim 1, characterized in that: The analysis of data collected by various medical devices to obtain the heart rate fluctuation range, blood pressure fluctuation range and respiratory rate change rate includes the following steps: Determine the starting point of the heart rate data, set the moving window size to 10 minutes, intercept the first 10 minutes of data from the starting point of the heart rate data, and calculate the maximum and minimum heart rate values ​​to obtain the heart rate fluctuation range of the window, slide the window backward by one data point to intercept and calculate, and repeat the above process until the entire heart rate data set is traversed to obtain a continuous heart rate fluctuation range sequence; Determine the time period for blood pressure monitoring, measure the patient's blood pressure within the selected time period, and record each measurement result to form a blood pressure data set within the time period. In the collected blood pressure data of each time period, find the highest and lowest values ​​of systolic and diastolic pressures, respectively. The difference is the fluctuation range of blood pressure within the time period. Calculate and record the fluctuation range of diastolic and systolic pressures respectively. A ventilator was used to record the number of breaths in each time period, and the corresponding respiratory frequency data were recorded at a certain time interval through the data acquisition software. The corresponding respiratory frequency difference of each pair of adjacent time points in the data set was calculated, and the average value was calculated after taking the absolute value. The square of the difference between the absolute value of the difference in respiratory frequency at each adjacent time point and the average value was calculated, and the square values ​​were added and divided by the total number of differences, and the standard deviation obtained by square root was used as the respiratory frequency change rate.

5. The ICU ward monitoring data processing system according to claim 1, characterized in that: The method of constructing a comprehensive vital sign stability model and outputting a comprehensive vital sign stability score of a patient through a trained model includes the following steps: The patient's heart rate, blood pressure and respiratory rate vital signs data were obtained from the medical records, and the data were cleaned and standardized. The processed heart rate fluctuation range, blood pressure fluctuation range and respiratory rate change rate data were substituted into the calculation of the Pearson correlation coefficient through the original vital signs data and the calculated heart rate fluctuation range, blood pressure fluctuation range and respiratory rate change rate, and the r and y values ​​were obtained respectively. HR-BP 、r HR-RR and r BP-RR , the calculated Pearson correlation coefficient is organized into a matrix, expressed as [r HR-BP , r HR-RR , r BP-RR ], as the feature input of the model, a linear regression-based framework was selected to construct a comprehensive vital signs stability model; For each sample, calculate r HR-BP 、r HR-RR and r BP-RR Experts give each sample a comprehensive vital sign stability score based on clinical standards, and the r HR-BP 、r HR-RR 、r BP-RR The data and scores are made into a data set table, the part of the data set used for the training set is used to train the model, and the trained model outputs the comprehensive vital sign stability score.

6. The ICU ward monitoring data processing system according to claim 1, characterized in that: The method of using a convolutional neural network framework to build a model for identifying normal operation and abnormal operation of medical equipment includes the following steps: Use video equipment to capture images of medical equipment in normal and abnormal states, capture images from different angles and distances, obtain all-round appearance features and indicator light status features of medical equipment, filter images, and classify and organize images according to normal and abnormal working states, and build a folder structure; Use image annotation tools to annotate the appearance features and indicator light status features in normal and abnormal medical device images, and generate corresponding annotation files, where the appearance features include the housing, buttons, and interfaces of the medical device, and the indicator light status features include color and shape; Use the convolutional neural network framework to build a model to identify the normal and abnormal working status of medical equipment. Use the labeled image data as input for training. Deploy the trained model in actual use to automatically detect the operating status of medical equipment and output whether the working status of the medical equipment is normal or abnormal. If the model outputs that the working status of the medical equipment is normal, the system will continue to monitor the ward; if the model outputs that the working status of the medical equipment is abnormal, the system will automatically trigger a secondary alarm and notify medical staff to handle it.

7. The ICU ward monitoring data processing system according to claim 1, characterized in that: The construction of the visitor-based identity recognition model includes the following steps: inputting the preprocessed video frame image containing the visitor into the YOLO model, and the model outputs the location information of the visitor, which is associated with the timestamp to generate a trajectory data sequence: Collect facial images of known visitors from the entrance surveillance video of the ICU ward and the photos taken at the visitor registration office, annotate the collected facial images, and the annotated information includes the visitor's name, ID number, image acquisition time and location. Arrange the annotated images according to the rules, perform preprocessing operations on the organized image data, initialize the identity recognition model using a deep learning framework, input the organized annotated image data set for training into the initialized model, and determine the known visitor with the highest probability value as the matching result based on the predicted probability distribution output by the model; The video surveillance images are preprocessed, and the YOLO target detection algorithm in image recognition technology is used to train the video images containing visitors as training data. The preprocessed video frame images are input into the trained YOLO model. The model outputs the location information of the visitors detected in each frame image. In the video sequence, a unique identification ID is assigned to each tracked visitor by associating the detected positions of the visitors between consecutive frames. The visitor position coordinates extracted from the video are associated with the timestamp to generate a trajectory data sequence for each visitor.

8. An ICU ward monitoring data processing system according to claim 1, characterized in that: The method uses the visitor trajectory data as the input observation sequence of the HMM, judges whether the trajectory conforms to the normal trajectory mode according to the set probability threshold, constructs a multidimensional ward parameter evaluation value and a multidimensional ward parameter threshold, and compares them, including the following steps: Collect the trajectory data of visitors, clean the trajectory data, mark the trajectory data of visitors as normal or abnormal, extract features from the original trajectory data, including position coordinates, speed, acceleration and residence time features, define the state set of the model according to the behavior pattern of visitors and the actual situation of the ICU ward, where the state is divided into in the ward, in the corridor and in the rest area, determine the observation set of the model, set the probability of the model being in each state at the initial moment, the probability of transitioning from one state to another, and the probability of generating each observation value in each state, estimate the HMM parameters through the labeled trajectory training set data, including the initial state probability distribution, state transition probability distribution and observation probability distribution, use the new visitor trajectory data as the input observation sequence of HMM, use the forward algorithm of the trained HMM to calculate the probability that the trajectory belongs to the normal trajectory mode, and judge whether the trajectory conforms to the normal trajectory mode according to the set probability threshold; If the output probability is lower than or equal to the set probability threshold, the visitor is considered to have abnormal trajectory behavior; otherwise, it is considered to have normal trajectory behavior; The patient's comprehensive vital sign stability score, the corresponding value of the medical equipment working status rule and the probability value output by the visitor behavior model are weighted and added to obtain a multi-dimensional ward parameter evaluation value; Among them, D represents the multidimensional ward parameter evaluation value, S represents the patient's comprehensive vital sign stability score, and E total represents the total number of medical devices, E abnormal represents the total number of abnormal working states of medical equipment, P represents the output probability value of the visitor behavior model, ω1, ω2 and ω3 represent the corresponding weight coefficients; According to the score output by the comprehensive vital sign stability model and the distribution of the medical equipment status recognition model and clinical experience, the stability threshold, equipment threshold and probability threshold are weighted and added to obtain the multi-dimensional ward parameter threshold; When the multi-dimensional ward parameter evaluation value is greater than the multi-dimensional ward parameter threshold, it indicates that the patient and environmental status in the ward are normal; otherwise, it indicates that the patient and environmental status in the ward are abnormal, and the system triggers a level one alarm.

Images