Patient monitoring system, monitoring method, and storage medium

Abstract

A patient monitoring system and method, using an image acquisition device to acquire video data containing a patient, analyzing the video data to obtain activity information of the patient, and displaying on a display unit the monitoring result and the activity information corresponding to the same patient and generated in the same preset time interval, or calculating the activity correlation result of the monitoring result and the patient motion amplitude, and displaying on the display unit the monitoring result and the activity correlation result corresponding to the same patient and in time, which can effectively assist medical staff in objectively analyzing and evaluating the activity of the patient during diagnosis and treatment.

Description

TECHNICAL FIELD

[0001] The present application relates to the medical field, in particular to a patient monitoring system, a monitoring method and a storage medium. BACKGROUND

[0002] At present, patient monitoring systems have been widely applied in medical and health care places such as hospitals, nursing homes and clinics. Generally, the patient monitoring system is used for detecting and monitoring the physiological parameters of patients, and is also used for observing and analyzing the health status or state of patients.

[0003] For most patients, if appropriate exercise can be carried out when their physical condition permits, it will help them recover as soon as possible. However, there are two problems in the current rehabilitation activities of patients: first, the assessment of patient activity and physiological condition. At present, medical staff usually assess the activity of patients on site or in videos, and give guidance on whether the activity is suitable and whether the activity amount should be increased or decreased. However, this assessment is based on subjectivity and experience, which requires high professional level of medical staff and lacks objective data support. Second, the patient's activity may cause false alarms. False alarms not only bring invalid work to medical staff, but also may cause medical staff to make wrong assessment of the patient's condition. SUMMARY

[0004] The technical problem solved by the present application is to provide a patient monitoring system, a monitoring method and a storage medium, which objectively and visually assess the activity of patients.

[0005] According to a first aspect, in an embodiment, a patient monitoring system is provided, comprising:

[0006] an image processing unit, configured to acquire video data containing a patient output by an image acquisition device, and analyze the video data to obtain activity information of the patient;

[0007] a display unit;

[0008] a data processing unit, configured to receive physiological parameter signals collected from at least one patient and obtain monitoring results of each patient, the monitoring results comprising parameter values of at least one physiological parameter and / or alarm events, and the data processing unit is further configured to receive the activity information of the patient, control the display unit to display the monitoring results and the activity information corresponding to the same patient and generated in the same preset time interval on the same screen.

[0009] According to a second aspect, in an embodiment, a patient monitoring system is provided, comprising:

[0010] an image analysis unit configured to obtain video data containing a patient output by an image acquisition device and analyze the video data to obtain activity information of the patient, the activity information comprising a motion amplitude of the patient;

[0011] a display unit;

[0012] a data processing unit configured to receive physiological parameter signals collected from at least one patient and obtain a monitoring result of each patient, the monitoring result comprising a parameter value of at least one physiological parameter and / or an alarm event, the data processing unit being configured to calculate an activity correlation result of the patient based on the monitoring result and the motion amplitude corresponding thereto in time, and control the display unit to display the monitoring result and the activity correlation result corresponding to the same patient and generated in the same preset time interval on the same screen, the activity correlation result being a real-time correlation coefficient or an activity correlation curve obtained according to the correlation coefficient changing over time.

[0013] According to a third aspect, an embodiment provides a patient monitoring method, comprising:

[0014] receiving video data containing a patient output by an image acquisition device;

[0015] analyzing the video data to obtain activity information of the patient;

[0016] receiving physiological parameter signals collected from at least one patient and obtaining a monitoring result of each patient, the monitoring result comprising a parameter value of at least one physiological parameter and / or an alarm event;

[0017] displaying the monitoring result and the activity information corresponding to the same patient and generated in the same preset time interval on the same screen on a display unit.

[0018] According to a fourth aspect, an embodiment provides a patient monitoring method, comprising:

[0019] receiving video data containing a patient output by an image acquisition device;

[0020] analyzing the video data to obtain activity information of the patient;

[0021] receiving physiological parameter signals collected from at least one patient and obtaining a monitoring result of each patient, the monitoring result comprising a parameter value of at least one physiological parameter and / or an alarm event;

[0022] Based on the monitoring result, an activity correlation result of the monitoring result and a motion amplitude corresponding to the monitoring result in time is calculated, and the monitoring result corresponding to the same patient and in time and the activity correlation result are displayed on the display unit at the same screen. The activity correlation result is a real-time correlation coefficient or an activity correlation curve obtained according to the correlation coefficient changing over time.

[0023] According to a fifth aspect, a computer readable storage medium is provided in an embodiment, and the medium stores a program executable by a processor to implement the method described above.

[0024] According to the patient monitoring system and method of the above embodiment, the video-related application and display function are integrated in the patient monitoring system, the activity information of the patient and the monitoring result of the physiological parameter are fused and displayed, or the correlation analysis is performed based on the activity trend change of the patient and the monitoring result of the physiological parameter, and the monitoring result of the physiological parameter and the correlation analysis result are fused and displayed, so that the medical staff can be provided with intuitive and objective data support. The medical staff can effectively assist in objectively analyzing and evaluating the activity of the patient in the diagnosis and treatment process, and the subjective evaluation of the activity of the patient by relying on the memory or on-site observation of the medical staff is no longer needed, so that more comprehensive and intelligent life monitoring can be provided for the patient. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A schematic diagram of patient activity;

[0026] Figure 2 A schematic diagram of a patient monitoring system in an embodiment;

[0027] Figure 3 A flowchart of calculating a motion amplitude in an embodiment;

[0028] Figure 4 A schematic diagram of a display interface of a display unit;

[0029] Figure 5 A schematic diagram of an embodiment in which patient activity information is fused;

[0030] Figure 6 A schematic diagram of an embodiment in which a trend graph fused with patient activity information and a physiological parameter trend graph are displayed on a time axis;

[0031] Figure 7 A scatter plot of patient motion amplitude and parameter change in another embodiment. DETAILED DESCRIPTION

[0032] The application will be described in further detail below with specific reference to the drawings. Like elements are referenced with like numerals throughout the various figures and embodiments. In the following description, numerous specific details are described to provide a thorough understanding of the application. However, it will be apparent to one skilled in the art that the application can be practiced without some or all of these details. In other instances, well known process steps have not been described in detail in order not to unnecessarily obscure the application. Unless otherwise specifically defined herein, all terms are to be given their broadest possible interpretation including mod, equivalents, and approximations thereof. It must also be noted that the application can be practiced without being limited to the specific sequences, conditions, material, or parameters set forth herein, and that parts and features can be interchanged for one another.

[0033] In addition, the features, operations, or steps described in the specification can be combined in any suitable manner without departing from the scope of the application. Furthermore, the various embodiments of the methods described herein can be implemented in any suitable order without departing from the scope of the application. Thus, the various steps of the methods described herein are not necessarily in the same order as they are described.

[0034] The terms "first", "second", and the like, as used in this description do not necessarily have any temporal meanings. The terms "connect", "couple", and the like, unless otherwise specified, include both direct and indirect connections (couplings).

[0035] In recent years, with the popularization of video acquisition devices and the development of video analysis technology, video monitoring systems have been gradually introduced into isolation wards, ICUs, and general wards. Through video analysis, the bedside activity behavior of patients can be automatically identified. The monitoring devices for monitoring the vital signs of patients are mainly used for the detection, analysis, and display of physiological signals. The present application envisages the fusion analysis of the detection results of the monitoring device and the video image data of the patient output by the video acquisition device, and proposes a series of comprehensive analysis tools based on the activity trend changes of the patient and the physiological parameters and their alarms, which can effectively assist medical staff in analyzing the activity of patients during diagnosis and treatment, and provide more comprehensive and intelligent life monitoring for patients.

[0036] In an embodiment, an analysis function of video data of a patient in a patient monitoring system can be added to an existing patient monitoring system, activity information of the patient is calculated by analysis, the activity information of the patient is analyzed in combination with common patient monitoring results, and correlation analysis is performed on the activity information of the patient and physiological state information of a selected physiological parameter of the patient according to a physiological parameter selected by medical staff, to obtain a correlation result of the two, which can directly and objectively show the influence of the activity of the patient on the physiological parameter, to assist the medical staff in evaluating whether the activity of the patient is appropriate, and on the other hand, the current activity of the patient can be combined when the medical staff judges whether the physiological parameter of the patient is normal.

[0037] Please refer to Figure 1 , a patient 100 performs rehabilitation activities on a bed, the patient 100 wears at least one sensor 140, the sensor 140 is used to detect physiological parameter signals of the patient, and the physiological parameter signals are transmitted to a bedside monitor 130 in a wired or wireless manner, the sensor 140 can be, for example, an electrocardiogram sensor, a blood pressure sensor, a temperature sensor, a blood oxygen sensor, etc., which respectively detect electrocardiogram signals, blood pressure signals, temperature signals and blood oxygen signals of the patient, the monitor 130 is used to process the received physiological parameter signals, and parameter values of each physiological parameter are calculated through a specific algorithm, such as electrocardiogram, heart rate, respiratory rate, blood pressure and blood oxygen, etc., the monitor 130 displays the calculated physiological parameters on a display unit in the form of real-time numerical values and / or time-related trend graphs. The monitor 130 can further monitor the physiological parameters, determine abnormal physiological parameters, and generate an alarm event.

[0038] In a medical place with a central station, the monitor 130 can also send the obtained parameter values of the physiological parameters of the patient to the central station in a wired or wireless manner, or send the physiological parameter signals output by the sensor to the central station 150, some sensors 140 can also directly send the detected physiological parameter signals to the central station 150 in a wireless manner, the central station 150 processes the received physiological parameter signals, and calculates the parameter values of each physiological parameter through a specific algorithm. The central station 150 can further monitor the physiological parameters, determine abnormal physiological parameters, and generate an alarm event.

[0039] The central station 150 is configured to receive physiological parameters of multiple patients and monitor and / or manage the multiple patients, thus the display interface of the display unit of the central station 150 is usually divided into multiple display areas, each patient has a corresponding display area, in the corresponding display area of the patient, the physiological parameters of the patient are displayed in the form of real-time values and / or time-related trend graphs. Usually, the central station 150 includes a communication module configured to communicate with the monitor or sensor through wired or wireless manner, receive data transmitted by the monitor or sensor, or transmit instructions, configuration information or alarm information to the monitor or sensor, and in some embodiments, the central station further communicates with the portable PAD or wearable device (such as a monitoring device worn on the wrist) through the communication module, so as to send the monitoring result to the PAD or wearable device carried by the medical staff.

[0040] The image acquisition device 120 is installed beside the bed and configured to capture images of the patient on the bed. In actual application, the image acquisition device 120 is associated with the patient, and the image data captured by the image acquisition device 120 is attributed to the patient. The image acquisition device 120 can be an RGB camera, an infrared camera or a depth camera. The image acquisition device 120 captures image data (such as a photo or a continuous frame of video data) containing the patient and then outputs the image data in the form of image frames to the bedside monitor or other electronic devices specially configured to process image or video data, such as the computer of the central station or a specially configured image processor. The image data captured by the image acquisition device 120 has a time mark, i.e. the real-time time when the image data is captured, or the electronic device receiving the image data adds a time mark to the image data subsequently, so as to analyze the image data in the time function.

[0041] In one embodiment, based on the patient monitoring system captured by the image acquisition device 120, the central station 150 is configured to analyze the image data in the time function, and determine the physiological parameters of the patient in the image data, and then display the physiological parameters of the patient in the corresponding display area of the patient in the display interface of the display unit of the central station 150. Figure 2As shown, the system includes a display unit 200, an image processing unit 210 and a data processing unit 220. The image processing unit 210 is communicatively connected with the image acquisition device 120 to receive image data outputted by the image acquisition device 120. The image processing unit 210 is also communicatively connected with the data processing unit 220 to send analysis results of the image to the data processing unit 220. The data processing unit 220 is connected with a physiological parameter source 240, which can be a sensor to detect physiological parameter information, or a bedside monitor when the data processing unit 220 is a processor of a central station. The data processing unit 220 is also connected with the display unit 200 to send data to be presented in a visualized manner to the display unit 200 for display. The data processing unit 220 can also be communicatively connected with the image acquisition device 120 to receive image data outputted by the image acquisition device 120 and save the image data in a memory 230. The data processing unit 220 can also save monitoring results obtained in the memory. The data can be saved in the memory in a one-to-one correspondence with the time when the data is generated, so that the data becomes a function of time, to generate a trend chart and / or to play back the corresponding time subsequently. In another embodiment, the image processing unit 210 receives image data outputted by the image acquisition device 120 and then transmits the image data to the data processing unit 220.

[0042] The image processing unit 210 obtains image data containing a patient outputted by the image acquisition device 120 and analyzes the image data to obtain activity information of the patient. The image processing unit 210 can be a chip or composed of multiple chips to jointly complete the analysis function of the image data. The image processing unit 210 receives image data in the form of image frames. If the image acquisition device 120 sends a photo, one photo can be one image frame. If the image acquisition device 120 needs to send a continuous video, the image acquisition device 120 needs to divide the continuous video into multiple continuous image frames in time sequence. The encoding method of the transmitted image frames can be to encode and transmit each frame, or to encode and transmit the initial image frame and the inter-frame difference to reduce the amount of data transmitted. After receiving the image frames, the image processing unit 210 decodes the image frames in time sequence to obtain the image frames and combines the image frames in time sequence into continuous video data. The image processing unit 210 can analyze the video data by a specific algorithm or artificial intelligence to output analysis results. The analysis results include activity information of the patient and / or other information. The activity information of the patient can be information representing whether the patient is moving, such as an analysis result of 1 indicating that the patient has activity and an analysis result of 0 indicating that the patient has no activity. The activity information of the patient can also be the movement amplitude value of the patient.

[0043] In other embodiments, in addition to the patient activity information described above, the image processing unit 210 can also output information on whether the patient has medical care at the bedside to the data processing unit 220. For example, in some embodiments, the image processing unit includes in the analysis result of the analysis based on the video data whether the patient is on the bed and whether there is other personnel at the bedside, and outputs the analysis result to the data processing unit. Based on the alarm event in the monitoring result, when the analysis result of the patient corresponding to the alarm event is that the patient is on the bed and there is no other personnel at the bedside, the data processing unit displays the current video data of the patient on the display unit to remind the medical staff to pay attention.

[0044] As shown in Figure 3 a specific embodiment for determining whether the patient is moving and calculating the movement amplitude value, the processing process includes the following steps:

[0045] Step 300, input the image frame. The image processing unit 210 inputs the image frame of the video data in time sequence frame by frame.

[0046] Step 310, determine the position of the human body in the image frame. First, human body detection needs to be performed in the image frame of the video data. Since it is a video taken for the patient on the bed, there should be at least one image frame in which the human body and the bed exist in the image frame to be analyzed, otherwise the image frame will continue to be detected until the human body is detected. The human body in the image may be the patient or other personnel such as medical staff. The human body detection method can use artificial intelligence method, input the image frame into the pre-trained model for comparison, and the detection model can use the existing yolo-x model, which is trained through appropriate samples, so that the image frame data including the patient and the bed can be detected to the human body and the bed. For the detected "human" and "bed", the position and area of "human" and "bed" are labeled, for example, the position and area of "human" and "bed" are labeled by rectangular frames.

[0047] Step 320, determine whether the detected "person" is a patient. Based on the detection result of step 310, take the rectangular frame as an example to illustrate how to determine the identity of the detected "person". First, calculate the overlapping part of the rectangular frame of the "person" and the rectangular frame of the "bed", calculate the proportion of the overlapping part in the rectangular frame of the "bed", and compare the proportion with a preset threshold value; if the rectangular frame of the "person" is mostly within the rectangular frame of the "bed", the detected "person" is considered to be a patient; if the rectangular frame of the "person" is mostly outside the rectangular frame of the "bed", the detected "person" is considered to be a medical staff beside the bed. When the rectangular frame is changed to other region marking methods, the same way can be used to identify the identity of the detected "person". When the detected "person" is considered to be a patient, step 330 is executed to perform subsequent patient activity monitoring and evaluation. If the detected "person" is not considered to be a patient, steps 300-320 can be executed in a loop until a patient is detected.

[0048] Step 330, determine the region of interest including the patient. Still take the rectangular frame as an example to illustrate. Take the rectangular frame of the patient as the target frame, and expand the target frame in a predetermined manner to form a new region, for example, expand the target frame by a certain proportion with the center of the target frame as the center to form a new enclosed region, and take the new region as the region of interest for subsequent patient activity monitoring.

[0049] Step 340, calculate the inter-frame difference. Collect image frames sorted by time from the video data, compare the regions with differences between each two adjacent image frames, calculate the difference value between each two adjacent image frames as the inter-frame difference. The difference between adjacent image frames may be caused by the activity of one or more parts of the body (such as the bending of limbs, posture change, displacement) or the displacement of the whole body. These differences will be reflected as pixel differences, and the number of pixels with differences can be counted as the difference value between each two adjacent image frames.

[0050] Step 350, mark the region with differences in the region of interest. The region of interest is composed of a plurality of pixels arranged in rows and columns, and the pixels in the region of interest are binarized, for example, the pixel values of the region with differences calculated in step 340 are set to 255, and the pixel values of other regions are set to 0.

[0051] Step 360, accumulate the calculated inter-frame difference in real time to obtain the accumulated absolute difference value. For example, the inter-frame difference between the nth-1 frame and the nth-2 frame is A, and the inter-frame difference between the nth-1 frame and the nth frame is B, then the accumulated absolute difference value after two operations is A+B. In order to make the accumulated absolute difference value more accurate, the inter-frame difference can be filtered (such as median filtering) after accumulation.

[0052] In the accumulation of the inter-frame difference, the binarization of the different pixels in the region of interest is also performed, i.e. as more inter-frame differences are detected, the pixels in the region of interest that are set to 255 also increase.

[0053] In step 370, the accumulated absolute difference is compared with a predetermined threshold value, when the accumulated absolute difference is less than the predetermined threshold value, it is considered that the patient has no body movement, and step 340 is continued, otherwise, it is considered that the patient has movement, and step 380 is executed.

[0054] In step 380, the motion region is obtained based on the marked different regions in the region of interest. First, the regions Ra' with difference marked in the current region of interest, such as the regions set to 255, are identified, and these regions are regionally connected, such as through image erosion, expansion, and / or noise reduction, to obtain a new region, and the new region is taken as the motion region Ra, so as to realize the detection of the motion region.

[0055] In step 390, the proportion of the motion region in the region of interest is calculated to obtain the real-time motion amplitude. It can be seen that the motion amplitude is a parameter for measuring the stretching degree of each part of the patient's body, the size of the posture change, the size of the displacement, and the size of the overall body displacement.

[0056] After step 390, it is turned to step 340 to compare the adjacent two frames of video data in the next cycle, so as to continuously output the real-time motion amplitude.

[0057] In step 370, when it is judged whether the patient has movement, the image processing unit 210 can output the motion analysis result representing that the patient is in the motion state to the data processing unit 220, but when the motion amplitude needs to be further output, the above steps 380-390 are continued to output the motion amplitude to the data processing unit 220.

[0058] Please continue to refer to Figure 2The data processing unit 220 receives physiological parameter signals collected from at least one patient and obtains monitoring results for each patient, which include parameter values of at least one physiological parameter and / or alarm events. The data processing unit 220 calculates the required parameter values of various physiological parameters, such as electrocardiogram, heart rate, respiratory rate, blood pressure, blood oxygen, etc., based on specific algorithms according to the physiological parameter signals. The data processing unit 220 displays the calculated physiological parameters in the form of real-time numerical values and / or time-dependent trend graphs on the display unit. The data processing unit 220 can further compare the parameter values of the physiological parameters with preset upper and lower threshold values. When the parameter value of a certain physiological parameter exceeds the upper and lower threshold value range, it is considered that the physiological parameter is abnormal, and thus an alarm event is generated. The data processing unit 220 can output an alarm prompt in the form of sound, light and / or tactile sensation, such as highlighting or flashing display of the abnormal physiological parameter on the display interface of the display unit, playing a sound through a sound player, and vibrating a wearable device worn by a medical staff.

[0059] In a specific embodiment, the data processing unit 220 can be a processor of a bedside monitor.

[0060] In another specific embodiment, the data processing unit 220 can be a processor of a central station. When the data processing unit 220 is a processor of a central station, the data processing unit 220 configures the display interface of the display unit of the central station 150 into multiple display regions, each patient having a corresponding display region. In the display region corresponding to the patient, the physiological parameters of the patient are displayed in the form of real-time numerical values and / or time-dependent trend graphs (e.g., real-time heart rate waveform graphs), as shown in Figure 4

[0061] The data processing unit 220 is also used to receive the analysis results of the video data output by the image processing unit 210 and the video data including the patient. The data processing unit 220 fuses and displays the activity information of the patient and the monitoring results output by the image processing unit 210, so that the monitoring results and the activity information corresponding to the same patient and generated in the same preset time interval are displayed on the same screen of the display unit. The same preset time interval is the corresponding monitoring results and activity information in time, i.e., the monitoring results and the activity information are generated at the same time, or the generation time of the two has a relatively short time interval, but in actual situations, the two are considered to be generated at the same time.

[0062] ​When the data processing unit 220 receives activity information including motion analysis results representing the patient's motion state, the data processing unit 220 displays the motion analysis results on the display unit next to the monitoring results for the same patient. The motion analysis results include at least one of a string, a diagram, and a thumbnail. For example, when the data processing unit 220 is a processor in a central station, the data processing unit 220 displays the string "patient movement" representing the patient's motion in the display area corresponding to that patient. Figure 5 The diagram shown integrates patient activity information. After analyzing the video data captured on patient 1, the data processing unit 220 displays the string "Patient Movement" representing the patient's movement in the display area corresponding to patient 1. Therefore, when medical staff observe changes in the patient's physiological parameters or experience alarm events, they can refer to the "Patient Movement" notification to make a more objective assessment of these changes. Furthermore, they can click the "Patient Movement" icon to display the patient's current video on the screen (e.g., in the lower right corner of the display unit).

[0063] In the improved embodiment, the motion analysis results can also be associated with the current video data. For example, when medical staff click on the string "patient movement", the data processing unit 220 can display the patient's current motion video in a predetermined area based on the trigger operation command. This allows medical staff to simultaneously observe the patient's activity video and the real-time waveform or trend graph of its physiological parameters, providing them with more detailed information and assistance, and enabling them to make a more objective assessment of changes in physiological parameters or alarm events.

[0064] When the data processing unit 220 receives activity information including the patient's motion amplitude, it displays the patient's motion amplitude value corresponding to the monitoring result in time, or an amplitude graph representing the magnitude of the motion amplitude, next to the patient's monitoring result on the display unit. The amplitude graph representing the magnitude of the motion amplitude can be a trend graph as a function of time, such as a waveform or bar chart, or a bubble chart that changes in real time with the output motion amplitude value. In one embodiment, the amplitude graph representing the magnitude of the motion amplitude is a motion amplitude trend graph over time, displayed on the same time axis as the patient's physiological parameter trend graph. Figure 6 As shown, the physiological parameter trend graphs of the same patient (e.g.) Figure 6The trend graph of the heart rate parameter and the trend graph of the motion amplitude are displayed in the corresponding display area of the patient. Thus, the medical staff can simultaneously observe the changes of the physiological parameter and the changes of the motion amplitude, and analyze the correlation, more objectively evaluate the influence of the patient's motion on the physiological parameter, and thus judge whether the current motion of the patient is appropriate, improve the monitoring quality of the patient, and make the patient speed up the recovery process through appropriate motion.

[0065] In the improved embodiment, the data processing unit displays the details of the physiological parameter in a predetermined time period before and after the selected time point on the display unit based on the selected time point of the user on the trend graph of the physiological parameter or the trend graph of the motion amplitude, and / or plays back the video data in the predetermined time period before and after the selected time point. As shown in the figure, the medical staff can review the previous situation, compare and analyze the current situation with the previous situation, and more objectively evaluate the physical condition of the patient and the influence of the motion on the physiological parameter. Figure 6

[0066] When the data processing unit 220 receives the motion amplitude of the patient in the activity information, the data processing unit can also calculate the activity correlation result of the selected physiological parameter and the motion amplitude corresponding to the selected physiological parameter in time (for example, generated at the same time or within the allowed error range), which can be a default physiological parameter from at least one physiological parameter, for example, a physiological parameter presented in the form of a waveform on the display interface, or a physiological parameter selected by the user (for example, the medical staff) from at least one physiological parameter. The activity correlation result can be a real-time correlation coefficient, or can be a physiological parameter-motion amplitude correlation curve obtained according to the correlation coefficient changing over time.

[0067] When calculating the correlation, the trend graph of the physiological parameter and the trend graph of the motion amplitude are slid according to the preset time window length and step size, the samples of the physiological parameter and the samples of the motion amplitude are collected in the sliding window, and the correlation coefficient of the physiological parameter and the motion amplitude in the same window is calculated, and the calculation formula can use various existing algorithms for calculating the correlation between two variables. For example, in the calculation window, one algorithm for the correlation coefficient of two variables is as follows:

[0068]

[0069] Wherein, x is the motion amplitude of the patient, y is the parameter value of the physiological parameter of the patient, X is the average motion amplitude in the calculation window, and y is the average physiological parameter in the calculation window.

[0070] ​After the correlation coefficient or the correlation curve is obtained by the above algorithm, the data processing unit 220 displays the real-time correlation coefficient or the correlation curve in the display area corresponding to the patient on the display unit. For example, the data processing unit displays the physiological parameter-movement amplitude correlation curve and the trend graph of the selected physiological parameter on the time axis on the display unit, as shown in Figure 6 .

[0071] In addition, the correlation coefficient between the alarm event of the physiological parameter selected by default or selected by the user and the movement amplitude corresponding thereto in time can also be calculated. When calculating the correlation, the physiological parameter trend graph and the movement amplitude trend graph on the same time axis are slid according to the preset time window length and step size, and the correlation coefficient between the alarm frequency and the movement amplitude in the same window is calculated. The calculation formula can use the existing algorithm for calculating the correlation between a binary variable and a continuous variable, for example, the Point-biserial correlation coefficient calculation method. The formula for calculating the correlation coefficient is as follows:

[0072]

[0073] wherein,

[0074]

[0075] the mean of the movement amplitude when there is no physiological parameter alarm;

[0076] the mean of the movement amplitude when there is a physiological parameter alarm;

[0077] n1: the number of samples in the sliding window in which there is a physiological alarm;

[0078] n0: the number of samples in the sliding window in which there is no physiological alarm;

[0079] n is the total number of samples for one calculation, which is the number of samples covered by the sliding window here;

[0080] Y is the movement amplitude currently collected in the sliding window.

[0081] After the correlation coefficient is obtained, the alarm-movement amplitude correlation curve is further obtained according to the correlation coefficient changing over time, and the alarm-movement amplitude correlation curve and the trend graph of the physiological parameter that has an alarm are displayed on the time axis on the display unit, as shown in Figure 6 .

[0082] In the improved embodiment, as shown in Figure 6As shown, the data processing unit can also display details of the monitoring results of a predetermined time period before and after the time point selected by the user on the trend graph of the physiological parameter or on the correlation curve of the physiological parameter-movement amplitude on the display unit, and / or play back the video data of a predetermined time period before and after the selected time point. The medical staff can click "parameter details" to view the details of the physiological parameter or click "video playback" to play back the movement video of the patient as needed.

[0083] In the improved embodiment, the data processing unit can also display details of the monitoring results of a predetermined time period before and after the time point selected by the user on the trend graph of the physiological parameter or on the correlation curve of the physiological parameter-movement amplitude on the display unit, and / or play back the video data of a predetermined time period before and after the selected time point. The medical staff can compare and analyze the trend graph of the physiological parameter and the correlation curve of the physiological parameter-movement amplitude, closely and intuitively observe the physiological parameters such as heart rate, blood oxygen, and blood pressure of the patient in the movement state, and the correlation between the physiological parameters and the movement amplitude, the correlation between the alarm event and the movement amplitude, objectively evaluate the physiological indicators of the patient and the influence of movement on the physiological parameters, and accordingly evaluate the physical condition of the patient, guide the patient to carry out appropriate movement when the physical condition of the patient allows, and facilitate the early recovery of the patient. Or when it is found that the physiological indicators of the patient exceed the allowed range, the activity level of the patient is adjusted in a timely manner.

[0084] For example, Figure 6 In the example shown in FIG. 6, as the activity of the patient changes, the heart rate obviously rises 61a, falls 62a, and rises again 63a; the rise of the heart rate at 61a triggers an upper limit alarm, and the fall of the heart rate at 62a triggers a lower limit alarm. According to the trend graph of the heart rate parameter and the trend graph of the movement amplitude displayed on the same time axis, at 61b on the trend graph of the movement amplitude corresponding to 61a in time, the movement amplitude increases, and at 62b on the trend graph of the movement amplitude corresponding to 62a in time, the movement amplitude decreases, so the medical staff can determine that the rise of the heart rate indicated by 61a is related to the increase of the movement amplitude, and the fall of the heart rate indicated by 62a is related to the decrease of the movement amplitude. In addition, the correlation between the physiological parameter change and the movement amplitude, and / or the correlation between the alarm and the movement amplitude can also be used for judgment. For example, at 61c on the correlation curve of the physiological parameter-movement amplitude corresponding to 61a in time, the correlation coefficient positively increases, and at 61d on the correlation curve of the alarm-movement amplitude corresponding to 61a in time, the correlation coefficient also positively increases. At 62c on the correlation curve of the physiological parameter-movement amplitude corresponding to 62a in time, the correlation coefficient negatively increases, showing a positive correlation, and at 62d on the correlation curve of the alarm-movement amplitude corresponding to 62a in time, the correlation coefficient also negatively increases, showing a negative correlation.

[0085] From the above observations and analysis, it can be realized that the patient's movement will affect the patient's physiological parameters, for example Figure 6 As shown, the first heart rate rise and the first heart rate drop, although both triggered the heart rate overrun alarm, since both changes are positively correlated with activity, showing the characteristics of normal physiological changes, it can be judged that the alarms generated at 61a and 62a are likely not real physiological alarms, nor technical alarms caused by sensor damage or poor contact, but alarms caused by patient movement. When the medical staff cannot determine the physiological abnormalities and alarms, they can also click on the time point of interest on the curve to display a detailed viewing window on the display unit for the medical staff to choose to view the parameter details and patient video within a preset time period before and after the selected time point.

[0086] In addition, the medical staff can discover potential abnormalities by observing the movement amplitude trend graph, the physiological parameter-movement amplitude correlation curve, and / or the alarm-movement amplitude correlation curve, for example, the second heart rate rise at 63a occurs during a decrease in activity, showing a small parameter increase on the curve, but no alarm is triggered, but from the movement amplitude trend graph, 63b corresponding to 63a in time shows a decrease in movement amplitude, the physiological parameter-movement amplitude correlation curve shows a negative correlation at 63c corresponding to 63a in time, and the correlation coefficient exceeds the lower threshold of negative correlation, and the alarm-movement amplitude correlation curve shows no correlation at 63d corresponding to 63a in time, which does not conform to the characteristics of physiological changes, and therefore requires attention. It is possible that the patient's physical condition is abnormal, or there are technical problems such as sensor aging, damage, or poor contact, at which time the medical staff can click on the time point of interest on the curve to display a detailed viewing window on the display unit for the medical staff to choose to view the parameter details and patient video within a preset time period before and after the selected time point, in order to further analyze and evaluate.

[0087] The trend graph of the physiological parameter is combined with the trend graph of the movement amplitude, the physiological parameter-movement amplitude correlation curve, and the alarm-movement amplitude correlation curve in the time axis display in an exemplary manner, and those skilled in the art should understand that the trend graph of the physiological parameter and the trend graph of the movement amplitude, the physiological parameter-movement amplitude correlation curve, and the alarm-movement amplitude correlation curve can be combined in other ways and displayed with the time axis according to needs.

[0088] For the case that the correlation curve is calculated, the data processing unit can also calculate the time period in which the correlation coefficient exceeds a preset threshold range, and indicate on the physiological parameter-movement amplitude correlation curve diagram, for example, in the form of bold display, different color display or flashing display, so as to attract more attention of medical staff.

[0089] For the case that the correlation curve is calculated, the data processing unit can extract the overall metric value of the correlation coefficient in the time period specified by the user, and display the overall metric value on the display unit, the overall metric value including at least one of the maximum value, the minimum value and the average value. In this way, the medical staff can evaluate the overall situation of the patient movement correlation.

[0090] When the data processing unit is the processor of the central station, the data processing unit can also obtain the overall metric value of the correlation of all patients in the same department monitored and managed by the user based on the user's instruction, and highlight the patients with high and / or low overall metric value on the display unit, so as to attract more attention of medical staff.

[0091] In addition, the activity correlation result is a scatter plot of the selected physiological parameter and the movement amplitude corresponding to the time, as shown in Figure 7 The scatter plot of heart rate and movement amplitude is shown, and it can also be seen from the figure that the movement amplitude and the change of the physiological parameter present a positive correlation relationship as a whole, and when there is an abnormal point in the figure, it also means that the condition of the patient at the time point may be abnormal.

[0092] In summary, the patient activity information can improve the quality of patient care in many ways:

[0093] Early activity state assessment: In the case of stable patient physical condition, early activity needs to be carried out as early as possible. Medical staff need to closely monitor the heart rate, blood pressure and other physiological indicators of the patient during activity, and when the physiological indicators exceed the allowed range, the activity level of the patient needs to be adjusted in time.

[0094] Early detection of changes in patient condition: When the patient is engaged in the same intensity of activity, different changes in physiological parameters reflect changes in the patient's condition.

[0095] Detecting device connection abnormalities: Patient activity information can help confirm the movement interference of physiological parameters. Under the same amplitude of limb movement, the increase in the movement interference alarm of some physiological parameters of the patient often means that the connection of the device accessories is abnormal (such as poor adhesion of ECG electrode, displacement of blood oxygen probe, IBP bending, etc.), which needs to be handled and corrected in time.

[0096] Those skilled in the art can understand that all or part of the functions of various methods in the above embodiments can be realized by hardware or by a computer program. When all or part of the functions in the above embodiments are realized by a computer program, the program can be stored in a computer readable storage medium, which can include read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions are realized by executing the program by a computer. For example, the program is stored in the memory of the device, and the above functions are realized by executing the program in the memory by the processor. In addition, when all or part of the functions in the above embodiments are realized by a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash disk or a mobile hard disk, and is saved in the memory of the local device by downloading or copying, or the system of the local device is updated, and the above functions are realized by executing the program in the memory by the processor. For example, the image processing unit can be one chip or composed of multiple chips, and the data processing unit can be one chip or composed of multiple chips. When the data processing unit is composed of multiple chips, each chip can execute one or more functions, for example, some chips are used to process physiological parameters, some chips are used to process received activity information of the patient and calculate the correlation coefficient, and some chips are used to process visual data. Of course, those skilled in the art can understand that the functions of the image processing unit and the data processing unit can also be realized by one chip.

[0097] The above application of specific examples to the present application is described, which is only used to help understand the present application and does not limit the present application. For those skilled in the art, according to the idea of the present application, several simple deductions, deformations or substitutions can be made.

Claims

1. A patient monitoring system, characterized by, The application comprises: an image processing unit configured to acquire video data containing a patient output by an image acquisition device and analyze the video data to obtain activity information of the patient; a display unit; a data processing unit configured to receive physiological parameter signals of at least one patient and obtain monitoring results of each patient, the monitoring results comprising parameter values of at least one physiological parameter and / or alarm events, and receive the activity information of the patient, and control the display unit to display the monitoring results and the activity information corresponding to the same patient and generated in the same preset time interval on the same screen; the data processing unit controls the display unit to present the monitoring results in the form of real-time parameter values and / or at least one physiological parameter trend graph advancing with time, and the activity information comprises the motion amplitude of the patient; the data processing unit controls the display unit to display the monitoring results of the same patient and the motion amplitude value corresponding to the monitoring results in time or an amplitude graph representing the motion amplitude in the display area corresponding to the patient; the data processing unit further calculates a correlation coefficient between the alarm event and the motion amplitude corresponding to the alarm event in time based on the alarm event in the monitoring results, and further obtains an alarm-motion amplitude correlation curve according to the correlation coefficient changing with time, and displays the alarm-motion amplitude correlation curve and the trend graph of the physiological parameter generating the alarm on the time axis on the display unit.

2. The patient monitoring system of claim 1, wherein, The activity information further comprises a motion analysis result representing the motion state of the patient, and the data processing unit controls the display unit to display the motion analysis result and the monitoring results of the same patient in the display area corresponding to the patient.

3. The patient monitoring system of claim 2, wherein, The data processing unit displays the motion video of the patient in a specified period or at present based on a triggering operation instruction on the motion analysis result in a predetermined area of the display unit.

4. The patient monitoring system of claim 1, wherein, The amplitude graph representing the motion amplitude is a motion amplitude trend graph advancing with time.

5. The patient monitoring system of claim 4, wherein, The motion amplitude trend graph and the physiological parameter trend graph are displayed on the time axis.

6. The patient monitoring system of claim 4, wherein, The data processing unit displays the details of the monitoring results in a predetermined time period before and after a selected time point on the display unit based on the selected time point on the physiological parameter trend graph or the motion amplitude trend graph by a user, and / or plays back the video data in a predetermined time period before and after the selected time point.

7. The patient monitoring system of claim 1, wherein, The data processing unit calculates an activity correlation result between the selected physiological parameter and the motion amplitude corresponding to the selected physiological parameter in time based on the selected physiological parameter from the at least one physiological parameter, and displays the activity correlation result on the display unit.

8. The patient monitoring system of claim 7, wherein, The activity correlation result is a real-time correlation coefficient; or the activity correlation result is a physiological parameter-motion amplitude correlation curve obtained according to the correlation coefficient changing with time, and the data processing unit displays the physiological parameter-motion amplitude correlation curve and the trend graph of the selected physiological parameter on the time axis on the display unit.

9. The patient monitoring system of claim 7, wherein, The activity correlation result is a scatter plot of the selected physiological parameter and the motion amplitude corresponding to the time.

10. A patient monitoring system, characterized by The method comprises: an image processing unit configured to acquire video data containing a patient output by an image acquisition device, and analyze the video data to obtain activity information of the patient, the activity information comprising a motion amplitude of the patient; a display unit; a data processing unit configured to receive physiological parameter signals collected from at least one patient and obtain monitoring results of each patient, the monitoring results comprising parameter values of at least one physiological parameter and / or alarm events, the data processing unit being configured to calculate activity correlation results of the patients based on the monitoring results and motion amplitudes corresponding to the time, and control the display unit to display the monitoring results and the activity correlation results corresponding to the same patient and the time on the same screen, the activity correlation result being a correlation coefficient or an activity correlation curve obtained according to the correlation coefficient changing over time; the activity correlation curve comprises a physiological parameter-motion amplitude correlation curve obtained based on a selected physiological parameter from at least one physiological parameter and a motion amplitude corresponding to the time, and / or an alarm-motion amplitude correlation curve obtained based on an alarm event in the monitoring results and a motion amplitude corresponding to the time.

11. The patient monitoring system of claim 1, 8 or 10, wherein, The data processing unit displays details of the monitoring results of a predetermined time period before and after a time point selected by a user on a trend graph of a physiological parameter or on a correlation curve on the display unit, and / or plays back the video data of the predetermined time period before and after the selected time point.

12. The patient monitoring system of claim 8 or 10, wherein, The data processing unit extracts an overall metric value of the correlation coefficient in a time period specified by a user, and displays the overall metric value on the display unit, the overall metric value comprising at least one of a maximum value, a minimum value and an average value; and / or the data processing unit calculates a time period in which the correlation coefficient exceeds a preset threshold range, and indicates the time period on the physiological parameter-motion amplitude correlation curve.

13. The patient monitoring system of claim 8 or 10, wherein, The data processing unit is a processor of a central station for monitoring and managing at least one patient, the data processing unit being configured to obtain overall metric values of correlations of all patients in the same department monitored and managed by the user based on instructions of the user, and highlight patients with high and / or low rankings in the overall metric values on the display unit.

14. The patient monitoring system of claim 1 or 10, wherein, The image processing unit is further configured to analyze whether the patient is in bed and whether there is another person beside the patient based on the video data, and output the analysis results to the data processing unit, the data processing unit being configured to display current video data of a patient corresponding to an alarm event on the display unit when the analysis result of the patient is that the patient is in bed and there is no other person beside the patient.

15. The patient monitoring system of claim 1 or 10, wherein, The data processing unit is a processor of a bedside monitor.

16. A method of patient monitoring, characterized by, The method comprises: receiving video data containing a patient output by an image acquisition device; analyzing the video data to obtain activity information of the patient; Receiving physiological parameter signals collected from at least one patient and obtaining monitoring results of each patient, the monitoring results comprising parameter values of at least one physiological parameter and / or alarm events; Displaying the monitoring results and activity information corresponding to the same patient and generated in the same preset time interval on the display unit in the same screen; The activity information comprises a motion analysis result representing that the patient is in a motion state, and / or the activity information comprises a motion amplitude of the patient, and the displaying the monitoring results and activity information corresponding to the same patient and generated in the same preset time interval on the display unit in the same screen comprises: displaying the motion amplitude value corresponding to the monitoring results in time or an amplitude graph representing the motion amplitude size of the monitoring results in the same patient on the display unit. Calculating a correlation coefficient of the alarm event and the motion amplitude corresponding to the alarm event in time based on the alarm event in the monitoring results, and further obtaining an alarm-motion amplitude correlation curve according to the correlation coefficient changing over time, and displaying the alarm-motion amplitude correlation curve and a trend graph of the physiological parameter generating the alarm on the time axis on the display unit.

17. The method of claim 16, wherein the patient monitoring method further comprises: The motion amplitude is calculated by: Detecting the patient in the image based on the image frames of the video data, and determining a region of interest including the patient based on the detected patient; Comparing the regions with differences between each two adjacent image frames based on the time-ordered image frames sampled from the video data, calculating the difference value between each two adjacent image frames as an inter-frame difference; Marking the regions with differences in the region of interest; Real-time accumulating the calculated inter-frame difference to obtain an accumulated absolute difference value; Comparing the accumulated absolute difference value with a predetermined threshold value, when the accumulated absolute difference value is less than the predetermined threshold value, considering that the patient has no body movement, otherwise, considering that the patient has activity, then performing region connection on the marked regions with differences in the current region of interest to obtain a motion region; Calculating the proportion of the motion region in the region of interest to obtain a real-time motion amplitude.

18. The patient care method of claim 16, wherein, The amplitude graph representing the motion amplitude size is a motion amplitude trend graph advancing over time, the monitoring results are presented on the display unit in the form of real-time parameter values and / or at least one in the form of a physiological parameter trend graph advancing over time, and the motion amplitude trend graph and the physiological parameter trend graph are displayed on the time axis.

19. The patient care method of claim 16, wherein Further comprising: calculating an activity correlation result of a selected physiological parameter and the motion amplitude corresponding to the selected physiological parameter in time from the selected physiological parameter, and displaying the activity correlation result on the display unit.

20. A method of patient monitoring, comprising: Comprising: Receiving video data containing a patient output by an image acquisition device; Analyzing the video data to obtain activity information of the patient; Receiving physiological parameter signals collected from at least one patient and obtaining monitoring results of each patient, the monitoring results comprising parameter values of at least one physiological parameter and / or alarm events; calculating, based on the monitoring result, an activity correlation result of the monitoring result and a motion amplitude corresponding to the monitoring result in time, and displaying, on the display unit, the monitoring result and the activity correlation result corresponding to the same patient and corresponding in time on the same screen, the activity correlation result being a real-time correlation coefficient or an activity correlation curve obtained according to the correlation coefficient changing over time; the activity correlation curve comprises a physiological parameter-motion amplitude correlation curve obtained based on a physiological parameter selected from at least one physiological parameter and a motion amplitude corresponding to the physiological parameter in time, and / or an alarm-motion amplitude correlation curve obtained based on an alarm event in the monitoring result and a motion amplitude corresponding to the alarm event in time.

21. A computer-readable storage medium, characterized in that, The medium has a program stored thereon, and the program can be executed by the processor to implement the method according to any one of claims 16-20.

Images