Patient monitoring system, monitoring method and storage medium

Abstract

The invention discloses a patient monitoring system and method, and the method comprises the steps: collecting video data including a patient through an image collection device, analyzing the video data to obtain the activity information of the patient, and displaying a monitoring result and the activity information corresponding to the same patient and generated in the same preset time interval on a display unit on the same screen, or an activity correlation result of the monitoring result and the movement amplitude of the patient is calculated, and the monitoring result and the activity correlation result which correspond to the same patient and correspond to the same patient in time are displayed on the same screen of the display unit, so that the medical care can be effectively assisted to objectively analyze and evaluate the activity of the patient in the diagnosis and treatment process.

Description

Technical Field

[0001] The present invention relates to the medical field, and in particular to a patient monitoring system, a monitoring method and a storage medium. Background Art

[0002] Currently, patient monitoring systems have been widely used in healthcare facilities such as hospitals, nursing homes and clinics. Typically, patient monitoring systems are used to detect and monitor patients' physiological parameters, and are also used to observe and analyze patients' health conditions or states.

[0003] For most patients, if they can do proper exercise if their physical condition allows, it will help them get better as soon as possible. However, there are currently two problems facing patients in their rehabilitation activities: the first is the assessment of the patient's activity and physiological condition. Currently, medical staff usually assess the patient's activity based on the scene or video, and give guidance on whether the activity is suitable and whether to increase or decrease the amount of activity. However, this assessment is based on subjectivity and experience, which requires a high level of professionalism from medical staff and lacks objective data support. The second is that the patient's activity may cause false alarms, which not only cause ineffective work for medical staff, but may also cause medical staff to make incorrect assessments of the patient's condition. Summary of the invention

[0004] The main technical problem solved by the present invention is to provide a patient monitoring system, a monitoring method and a storage medium, which can evaluate the activity of a patient in an objective and visual manner.

[0005] According to a first aspect, an embodiment provides a patient monitoring system, comprising:

[0006] An image processing unit, used to obtain video data including the patient output by the image acquisition device, and analyze the video data to obtain activity information of the patient;

[0007] Display unit;

[0008] A data processing unit is used to receive physiological parameter signals collected from at least one patient and obtain monitoring results for each patient, wherein the monitoring results include parameter values ​​and / or alarm events of at least one physiological parameter. The data processing unit is also used to receive activity information of the patient and control the display unit to display the monitoring results and activity information corresponding to the same patient and generated in the same preset time interval on the same screen.

[0009] According to a second aspect, an embodiment provides a patient monitoring system, comprising:

[0010] An image analysis unit, used to obtain video data including a patient output by an image acquisition device, and analyze the video data to obtain activity information of the patient, wherein the activity information includes a movement amplitude of the patient;

[0011] Display unit;

[0012] A data processing unit is used to receive physiological parameter signals collected from at least one patient and obtain monitoring results for each patient, wherein the monitoring results include parameter values ​​and / or alarm events of at least one physiological parameter, and the data processing unit calculates the activity correlation results of the patient based on the monitoring results and the movement amplitude corresponding thereto in time, and controls the display unit to display the monitoring results and the activity correlation results corresponding to the same patient and corresponding in time on the same screen, wherein the activity correlation results are real-time correlation coefficients or activity correlation curves obtained based on correlation coefficients that change over time.

[0013] According to a third aspect, an embodiment provides a patient monitoring method, comprising:

[0014] Receiving video data including the patient output by the image acquisition device;

[0015] Analyzing the video data to obtain activity information of the patient;

[0016] Receiving a physiological parameter signal collected from at least one patient and obtaining a monitoring result for each patient, wherein the monitoring result includes a parameter value and / or an alarm event of at least one physiological parameter;

[0017] The monitoring results and activity information corresponding to the same patient and generated in the same preset time interval are displayed on the same screen on the display unit.

[0018] According to a fourth aspect, an embodiment provides a patient monitoring method, comprising:

[0019] Receiving video data including the patient output by the image acquisition device;

[0020] Analyzing the video data to obtain activity information of the patient;

[0021] Receiving a physiological parameter signal collected from at least one patient and obtaining a monitoring result for each patient, wherein the monitoring result includes a parameter value and / or an alarm event of at least one physiological parameter;

[0022] The activity correlation results of the monitoring results and the movement amplitude corresponding thereto in time are calculated based on the monitoring results, and the monitoring results and the activity correlation results corresponding to the same patient and corresponding in time are displayed on the same screen on the display unit, and the activity correlation results are real-time correlation coefficients or activity correlation curves obtained according to the correlation coefficients that change over time.

[0023] According to the fifth aspect, an embodiment provides a computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to implement the above method.

[0024] According to the patient monitoring system and method of the above-mentioned embodiment, video-related applications and display functions are integrated into the patient monitoring system, and the patient's activity information and physiological parameter monitoring results are fused and displayed, or correlation analysis is performed based on the patient's activity trend changes and the physiological parameter monitoring results, and the physiological parameter monitoring results and the correlation analysis results are fused and displayed, so as to provide intuitive and objective data support for medical staff, and effectively assist medical staff in objectively analyzing and evaluating the patient's activity during the diagnosis and treatment process, and no longer need to rely on their own memory or on-site observation to make a subjective evaluation of the patient's activity, thereby providing patients with more comprehensive and intelligent life monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of the patient's activities;

[0026] Figure 2 A schematic diagram of a patient monitoring system in an embodiment;

[0027] Figure 3 A flowchart of calculating the motion amplitude in one embodiment;

[0028] Figure 4 is a schematic diagram of a display interface of a display unit;

[0029] Figure 5 is a schematic diagram of an embodiment in which patient activity information is integrated;

[0030] Figure 6 A schematic diagram of displaying a trend graph fused with patient activity information and a trend graph of physiological parameters on the same time axis in an embodiment;

[0031] Figure 7 It is a scatter plot of the patient's motion amplitude and parameter changes in another embodiment. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0033] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0034] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0035] In recent years, with the popularization of video acquisition equipment and the development of video analysis technology, video monitoring systems have gradually been introduced into isolation wards, ICUs, and general wards. Through video analysis, the patient's bedside activities can be automatically identified. The monitoring equipment that monitors the patient's vital signs is mostly used for the detection, analysis and display of physiological signals. The present invention envisions a fusion analysis of the detection results of the monitoring equipment and the patient's video image data output by the video acquisition equipment, and proposes a series of comprehensive analysis tools based on the patient's activity trend changes and physiological parameters and their alarms, which can effectively assist medical care in analyzing the patient's activity during the diagnosis and treatment process, and provide patients with more comprehensive and intelligent life monitoring.

[0036] In one embodiment, an analysis function for video data including patients can be added to the existing monitoring system. The patient's activity information is analyzed and calculated, and the patient's activity information and common patient monitoring results are integrated and analyzed. For the physiological parameters selected by the medical staff, the patient's activity information and the physiological state information of the selected physiological parameters are subjected to correlation analysis to obtain the correlation results of the two. The correlation results can intuitively and objectively show the impact of the patient's activities on his physiological parameters, and assist the medical staff in evaluating whether the patient's activities are appropriate. On the other hand, when the medical staff judges whether the patient's physiological parameters are normal, they can also make judgments based on the patient's current activity.

[0037] Please refer to Figure 1 , the patient 100 performs rehabilitation activities on the hospital bed, and the patient 100 wears at least one sensor 140, which is used to detect the patient's physiological parameter signals and transmit the physiological parameter signals to the bedside monitor 130 by wire or wireless means. The sensor 140 can be, for example, an electrocardiogram sensor, a blood pressure sensor, a temperature sensor, a blood oxygen sensor, etc., which respectively detect the patient's electrocardiogram signal, blood pressure signal, temperature signal and blood oxygen signal. The monitor 130 is used to process the received physiological parameter signals, and calculate the parameter values ​​of each physiological parameter, such as electrocardiogram, heart rate, respiratory rate, blood pressure and blood oxygen, etc. through a specific algorithm. The monitor 130 displays the calculated physiological parameters on the display unit in the form of real-time values ​​and / or time-related trend graphs. The monitor 130 can also further monitor the physiological parameters, determine the abnormal physiological parameters, and generate alarm events.

[0038] In a medical place with a central station, the monitor 130 can also send the obtained parameter values ​​of the patient's physiological parameters to the central station by wired or wireless means, or send the physiological parameter signals output by the sensor to the central station 150. Some sensors 140 can also directly send the detected physiological parameter signals to the central station 150 by wireless means, and the central station 150 processes the received physiological parameter signals and calculates the parameter values ​​of each physiological parameter through a specific algorithm. The central station 150 can also further monitor the physiological parameters, determine the abnormal physiological parameters, and generate alarm events.

[0039] The central station 150 is used to receive the physiological parameters of multiple patients and monitor and / or manage the multiple patients. Therefore, the display interface of the display unit set in the central station 150 is usually divided into multiple display areas, and each patient has its corresponding display area. In the display area corresponding to the patient, the physiological parameters of the patient are displayed in the form of real-time numerical values ​​and / or time-related trend graphs. Usually, the central station 150 includes a communication module for communicating with a monitor or sensor in a wired or wireless manner, receiving data transmitted by the monitor or sensor, or transmitting instructions, configuration information or alarm information to the monitor or sensor. In some embodiments, the central station also communicates with a portable PAD or wearable device (such as a monitoring device worn on the wrist) through the communication module to send the monitoring results to the PAD or wearable device carried by the medical staff.

[0040] The image acquisition device 120 is installed next to the bed and is used to take pictures of the patient on the bed. In actual applications, the image acquisition device 120 is associated with the patient, and the image data collected by it is attributed to the patient. The image acquisition device 120 can be an RGB camera, an infrared camera, or a depth camera. The image acquisition device 120 captures image data (such as photos or continuous frames of video data) containing the patient, and then outputs the image data in the form of image frames to a bedside monitor or other electronic device dedicated to processing image or video data, such as a computer at a central station, or a dedicated image processor. The image data collected by the image acquisition device 120 has a time stamp, that is, the real-time time of the image data collection, or the electronic device receiving the image data subsequently adds a time stamp to the image data so as to analyze the image data as a time function.

[0041] In one embodiment, the patient monitoring system based on the image acquisition device 120 acquires Figure 2As shown, it includes a display unit 200, an image processing unit 210 and a data processing unit 220. The image processing unit 210 is connected to the image acquisition device 120 for communication and receives the image data output by the image acquisition device 120. The image processing unit 210 is also connected to the data processing unit 220 for communication and sends the analysis result of the image to the data processing unit 220. The data processing unit 220 is connected to the physiological parameter source 240. The physiological parameter source 240 can be a sensor for detecting physiological parameter information. When the data processing unit 220 is a processor of the central station, the physiological parameter source 240 can also be a bedside monitor. The data processing unit 220 is also connected to the display unit 200 and sends the data that needs to be presented in a visual manner to the display unit 200 for display. The data processing unit 220 can also be connected to the image acquisition device 120 for communication and receives the image data output by the image acquisition device 120, and stores the image data in the memory 230. At the same time, the data processing unit 220 can also store the obtained monitoring result in the memory. When the data is stored in the memory, the data and the time when it is generated can be stored in a one-to-one correspondence, so that the data becomes a function of time, so as to generate a trend graph and / or play back the corresponding time later. In another embodiment, the image processing unit 210 receives the image data output by the image acquisition device 120, and then transmits the image data to the data processing unit 220.

[0042] The image processing unit 210 obtains the image data including the patient output by the image acquisition device 120, and analyzes the image data to obtain the patient's activity information. It can be a chip or a plurality of chips, which together complete the image data analysis function. The image processing unit 210 receives the image data in the form of image frames. If the image acquisition device 120 sends a photo, a photo can be an image frame. If the image acquisition device 120 needs to send a continuous video, the image acquisition device 120 needs to divide the continuous video into a plurality of continuous image frames in time sequence. The encoding method for the image frames to be sent can be to encode and send each frame, or to encode and send the initial image frame plus the frame difference, so as to reduce the amount of data transmitted. After receiving the image frame, the image processing unit 210 decodes the image frame accordingly in time sequence to obtain the image frame, and combines the image frame into continuous video data in time sequence. The image processing unit 210 can analyze the video data by a specific algorithm or artificial intelligence method, and output an analysis result, wherein the analysis result includes the patient's activity information and / or other information. The patient's activity information can be information indicating whether the patient is moving, for example, the analysis result is 1, indicating that the patient is moving, and the analysis result is 0, indicating that the patient's body is not moving. The patient's activity information can also be the patient's movement amplitude information, such as the patient's movement amplitude value.

[0043] In other embodiments, in addition to the above-mentioned patient activity information, the image processing unit 210 may also output information on whether there is medical care at the patient's bedside to the data processing unit 220. For example, in some embodiments, the image processing unit may include in the analysis result based on the video data whether the patient is in bed and whether there are other people at the bedside, and output the analysis result to the data processing unit. The data processing unit may display the current video data of the patient on the display unit based on the alarm event in the monitoring result, when the analysis result of the patient corresponding to the alarm event is that the patient is in bed and there are no other people at the bedside, so as to alert the medical staff.

[0044] like Figure 3 As shown, a specific embodiment of determining whether a patient is exercising and calculating the exercise amplitude value is provided, and the processing process includes the following steps:

[0045] Step 300: input image frames. The image processing unit 210 inputs image frames of video data frame by frame in time sequence.

[0046] Step 310, determine the position of the human body in the image frame. First, it is necessary to detect the human body in the image frame of the video data. Since the video is shot for the patient on the hospital bed, there should be at least one frame of the image frame to be analyzed in which the human body and the hospital bed exist, otherwise the image frame will continue to be detected until the human body is detected. The human body in the image may be a patient or other personnel, such as medical staff. The human body detection method can adopt an artificial intelligence method, and the image frame is input into a pre-trained model for comparison. The detection model can adopt the existing method yolo-x model, which is obtained by training with suitable samples, so that the human body and the hospital bed can be detected after the image frame data including the patient and the hospital bed are input. For the detected "person" and "bed", the position and area of ​​the "person" and "bed" are marked, for example, the position and area of ​​the "person" and "bed" are marked respectively by using rectangular frames.

[0047] Step 320, determine whether the detected "person" is a patient. Based on the detection result of step 310, take the rectangular frame as an example to illustrate how to determine the identity of the detected "person". First, calculate the overlapping part of the rectangular frame of the "person" and the rectangular frame of the "bed", calculate the proportion of the overlapping part in the rectangular frame of the "bed", and compare the proportion with the preset threshold; if the rectangular frame of the "person" is mostly within the rectangular frame of the "bed", the detected "person" is considered to be a patient; if the rectangular frame of the "person" is mostly outside the rectangular frame of the "bed", the detected "person" is considered to be a bedside medical staff. When the rectangular frame is changed to other area marking methods, the same method can also be used to identify the identity of the detected "person". When the detected "person" is considered to be a patient, step 330 is executed to perform subsequent patient activity monitoring and evaluation. If the detected "person" is considered not to be a patient, steps 300-320 can be executed repeatedly until a patient is detected.

[0048] Step 330, determine the region of interest including the patient. Still using a rectangular frame as a region marker as an example for explanation. Taking the patient's rectangular frame as the target frame, the target frame is enlarged in a predetermined manner to form a new region. For example, taking the center of the target frame as the center, the target frame is enlarged proportionally to form a new enclosed region after enlargement, and the new region is used as the region of interest for subsequent patient activity monitoring.

[0049] Step 340, calculate the frame difference. Collect image frames sorted in time from the video data, compare the areas where there are differences between every two adjacent image frames, and calculate the difference between every two adjacent image frames as the frame difference. The difference between adjacent image frames may be caused by the activities of one or more parts of the body (such as flexion and extension of limbs, posture changes, displacement), etc., or it may be caused by the displacement of the whole body. These differences will be reflected as pixel differences, and the number of pixels with differences can be counted as the difference between every two adjacent image frames.

[0050] Step 350: Mark the area with differences in the region of interest. The region of interest is composed of a plurality of pixels arranged in rows and columns. The pixels in the region of interest are binarized, for example, the pixel value of the area with differences calculated in step 340 is set to 255, and the pixel values ​​of other areas are set to 0.

[0051] Step 360, the calculated inter-frame differences are accumulated in real time to obtain a cumulative absolute difference. For example, if the inter-frame difference between the n-1th frame and the n-2th frame is A, and the inter-frame difference between the n-1th frame and the nth frame is B, then the cumulative absolute difference after two operations is A+B. In order to make the cumulative absolute difference more accurate, the inter-frame differences can be filtered (e.g., median filtering) after accumulation.

[0052] When accumulating the inter-frame differences, the difference pixels in the region of interest are also binarized, that is, as more inter-frame differences are detected, the number of pixels set to 255 in the region of interest increases accordingly.

[0053] In step 370 , the accumulated absolute difference is compared with a predetermined threshold value. When the accumulated absolute difference is less than the predetermined threshold value, it is considered that the patient has no body movement, and step 340 is continued. Otherwise, it is considered that the patient has movement, and step 380 is executed.

[0054] Step 380, based on the marked regions with differences in the region of interest, a motion region is obtained. First, the marked regions Ra' with differences in the current region of interest are identified, for example, regions set to 255, and these regions are connected, for example, after image erosion, expansion, and / or noise reduction, a new region is obtained, and the new region is used as the motion region Ra, thereby realizing the detection of the motion region.

[0055] Step 390, calculate the proportion of the motion area in the region of interest to obtain the real-time motion amplitude. It can be seen that the motion amplitude is a parameter to measure the degree of extension and contraction of each part of the patient's body, the magnitude of posture change, the magnitude of displacement, and the magnitude of overall body displacement.

[0056] After step 390 is completed, the process turns to step 340 to compare two adjacent frames of video data in the next cycle, thereby continuously outputting the real-time motion amplitude.

[0057] In step 370, when it is determined whether the patient is moving, the image processing unit 210 can output the motion analysis results representing that the patient is in a motion state to the data processing unit 220, but when it is necessary to further output the motion amplitude, continue to execute the above steps 380-390 and output the motion amplitude to the data processing unit 220.

[0058] Please continue to refer to Figure 2, the data processing unit 220 receives the physiological parameter signal collected from at least one patient, and obtains the monitoring result of each patient, and the monitoring result includes the parameter value of at least one physiological parameter and / or an alarm event. The data processing unit 220 calculates the parameter values ​​of each required physiological parameter, such as electrocardiogram, heart rate, respiratory rate, blood pressure and blood oxygen, etc., based on the physiological parameter signal based on a specific algorithm. The data processing unit 220 displays the calculated physiological parameters on the display unit in the form of real-time numerical values ​​and / or time-related trend graphs. The data processing unit 220 can also further compare the parameter value of the physiological parameter with the preset upper and lower limit thresholds. When the parameter value of a physiological parameter exceeds the upper and lower limit threshold range, it is considered that the physiological parameter is abnormal, so an alarm event is generated. The data processing unit 220 can output an alarm prompt in the form of sound, light and / or touch, for example, by highlighting or flashing the abnormal physiological parameter on the display interface through the display unit, for example, by playing a sound through a sound player, and for example, by the vibration of the wearable device worn by the medical staff.

[0059] In a specific embodiment, the data processing unit 220 may be a processor of a bedside monitor.

[0060] In another specific embodiment, the data processing unit 220 may be a processor of the central station. When the data processing unit 220 is the processor of the central station, the data processing unit 220 configures the display interface of the display unit of the central station 150 into a plurality of display areas, each patient having a corresponding display area, and in the display area corresponding to the patient, the physiological parameters of the patient are displayed in the form of real-time numerical values ​​and / or time-related trend graphs (e.g., real-time heart rate waveform graphs), such as Figure 4 shown.

[0061] The data processing unit 220 is also used to receive the analysis results of the video data and the video data including the patient output by the image processing unit 210. The data processing unit 220 performs fusion display processing on the activity information and monitoring results of the patient received from the image processing unit 210, so that the monitoring results and activity information corresponding to the same patient and generated in the same preset time interval are displayed on the display unit on the same screen. The same preset time interval refers to the monitoring results and activity information that correspond in time, that is, the monitoring results and activity information are generated at the same time, or there is a shorter time interval between the generation time of the two, but in actual situations, it is allowed to regard the two as being generated at the same time.

[0062] When the data processing unit 220 receives the motion analysis result including the motion analysis result indicating that the patient is in motion, the data processing unit 220 displays the motion analysis result on the display unit next to the monitoring result corresponding to the same patient, and the motion analysis result includes at least one of a character string, a schematic diagram, and a thumbnail. For example, when the data processing unit 220 is a processor of the central station, the data processing unit 220 displays the character string "patient motion" indicating that the patient is in motion in the display area corresponding to the patient, such as Figure 5 As shown in the schematic diagram with patient activity information, the analysis result of the video data taken by patient 1 shows that the patient is moving, and the data processing unit 220 displays the character string "patient movement" representing the patient's movement in the display area corresponding to the patient 1. Therefore, when the medical staff observes the changes in the patient's physiological parameters or the occurrence of an alarm event, they can refer to the prompt information of the patient's movement and make a more objective assessment of the changes in the physiological parameters or the alarm events that have occurred. And they can click the "patient movement" icon as needed to display the patient's current video on the same screen on the display interface (for example, in the lower right corner of the display unit).

[0063] In an improved embodiment, the motion analysis results can also be associated with current video data. For example, when the medical staff clicks on the character string "patient movement", the data processing unit 220 can display the patient's current motion video in a predetermined area based on the trigger operation instruction, so that the medical staff can simultaneously observe the patient's activity video and the real-time waveform or trend graph of his physiological parameters, providing the medical staff with more detailed information and assistance, so that the medical staff can make a more objective assessment of the changes in physiological parameters or the occurrence of alarm events.

[0064] When the data processing unit 220 receives activity information including the patient's movement amplitude, the data processing unit displays the patient's movement amplitude value corresponding to the monitoring result in time or an amplitude graph used to characterize the movement amplitude size on the display unit next to the monitoring result of the same patient. The amplitude graph characterizing the movement amplitude size can be a trend graph as a function of time, such as a waveform graph or a bar graph, or a bubble graph that changes in real time with the output movement amplitude value. In one embodiment, the amplitude graph used to characterize the movement amplitude size is a movement amplitude trend graph that progresses over time, and is displayed on the same time axis as the patient's physiological parameter trend graph. Figure 6 As shown, the trend graph of physiological parameters of the same patient (such as Figure 6The heart rate parameter trend graph (as shown in the figure) and the movement amplitude trend graph are displayed in the display area corresponding to the patient on the same time axis. In this way, medical staff can observe the changes in physiological parameters and movement amplitude at the same time, and conduct correlation analysis to more objectively evaluate the impact of patient exercise on physiological parameters, thereby judging whether the patient's current exercise volume is appropriate, improving the quality of patient monitoring, and enabling patients to accelerate their recovery process through appropriate exercise.

[0065] In an improved embodiment, the data processing unit displays the details of the physiological parameters for a predetermined period of time before and after the selected time point on the display unit based on the time point selected by the user on the trend graph of the physiological parameters or the trend graph of the movement amplitude, and / or plays back the video data for a predetermined period of time before and after the selected time point. Figure 6 As shown. This allows medical staff to review previous situations, compare and analyze the current situation with the previous situation, and more objectively evaluate the patient's physical condition and the impact of exercise on physiological parameters.

[0066] When the data processing unit 220 receives the patient's movement amplitude in the activity information, the data processing unit can also calculate the activity correlation result of the selected physiological parameter and the movement amplitude corresponding to it in time (for example, generated at the same time, or the generation time difference is within the allowable error range) through the movement amplitude. The selected physiological parameter can be a default physiological parameter from at least one physiological parameter, such as a physiological parameter presented in the form of a waveform on the display interface, or a physiological parameter selected by the user (for example, a medical staff) from at least one physiological parameter. The activity correlation result can be a real-time correlation coefficient, or can be a physiological parameter-movement amplitude correlation curve obtained according to the correlation coefficient that changes over time.

[0067] When calculating the correlation, slide on the trend graph of the physiological parameter and the trend graph of the movement amplitude on the same time axis according to the preset time window length and step length, collect samples of the physiological parameter and the movement amplitude in the sliding window, and calculate the correlation coefficient of the parameter value of the physiological parameter and the movement amplitude in the same window. The calculation formula can adopt various existing algorithms for calculating the correlation between two variables. For example, an algorithm for calculating the correlation coefficient of two variables in the calculation window is as follows:

[0068]

[0069] Wherein, x is the patient's motion amplitude, y is the parameter value of the patient's physiological parameter, X is the mean value of the motion amplitude within the calculation time window, and y is the mean value of the physiological parameter within the calculation time window.

[0070] After obtaining the correlation coefficient or correlation curve through the above algorithm, the data processing unit 220 displays the real-time correlation coefficient or correlation curve in the display area corresponding to the patient on the display unit. For example, the data processing unit displays the physiological parameter-motion amplitude correlation curve and the trend graph of the selected physiological parameter on the same time axis on the display unit, such as Figure 6 shown.

[0071] In addition, the correlation coefficient between the alarm event of the physiological parameter and the corresponding movement amplitude in time can also be calculated according to the default setting or the physiological parameter selected by the user. When calculating the correlation, according to the preset time window length and step length, the trend graph of the physiological parameter and the trend graph of the movement amplitude on the same time axis are slid to calculate the correlation coefficient between the alarm frequency and the movement amplitude in the same window. The calculation formula can adopt the existing algorithm for calculating the correlation between binary variables and continuous variables, such as the Point-biserial correlation coefficient calculation method. The formula for calculating the correlation coefficient is as follows:

[0072]

[0073] in,

[0074]

[0075] The average value of the movement amplitude when there is no physiological parameter alarm;

[0076] The mean value of the movement amplitude when there is a physiological parameter alarm;

[0077] n1: the number of samples in the sliding window where physiological alarms exist;

[0078] n0: number of samples in the sliding window without physiological alarm;

[0079] n is the total number of samples used for one calculation, which is the number of samples covered by the sliding window here;

[0080] Y is the current motion amplitude collected within the sliding window.

[0081] After the correlation coefficient is obtained, the alarm-exercise amplitude correlation curve is further obtained according to the correlation coefficient that changes with time, and the alarm-exercise amplitude correlation curve and the trend graph of the physiological parameter causing the alarm are displayed on the same time axis on the display unit, such as Figure 6 shown.

[0082] In an improved embodiment, Figure 6As shown, the data processing unit can also display the details of the monitoring results for a predetermined time period before and after the selected time point on the display unit based on the time point selected by the user on the trend graph of the physiological parameters or the physiological parameter-motion amplitude correlation curve, and / or play back the video data of the predetermined time period before and after the selected time point. Medical staff can click on "Parameter Details" to view the details of the physiological parameters or click on "Video Playback" to play back the patient's motion video as needed.

[0083] In an improved embodiment, the data processing unit can also display the details of the monitoring results of the predetermined time period before and after the selected time point on the display unit based on the time point selected by the user on the trend graph of the physiological parameters or the alarm-exercise amplitude correlation curve, and / or replay the video data of the predetermined time period before and after the selected time point. Medical staff can compare and analyze the trend graph of physiological parameters and the physiological parameter-exercise amplitude correlation curve, closely and intuitively observe the patient's heart rate, blood oxygen, blood pressure and other physiological parameters in the exercise state, as well as the correlation between the physiological parameters and the exercise amplitude, and the correlation between the alarm event and the exercise amplitude, objectively evaluate the patient's physiological indicators and the impact of exercise on the physiological parameters, evaluate the patient's physical condition based on this, and guide the patient to carry out appropriate exercise if the patient's physical condition allows, which is conducive to the patient's early recovery. Or when it is found that the patient's physiological indicators exceed the allowable range, the patient's activity level is adjusted in time.

[0084] For example, Figure 6 With the change of the patient's activity, the heart rate has a significant rise 61a, a fall 62a and a second rise 63a; the heart rate rise at 61a triggers the upper limit alarm, and the heart rate drop at 62a triggers the lower limit alarm. According to the patient's heart rate parameter trend graph and the movement amplitude trend graph displayed on the same time axis, at 61b corresponding to 61a in time on the movement amplitude trend graph, the movement amplitude shows an increase, while at 62b corresponding to 62a in time, the movement amplitude shows a decrease, so the medical staff can judge that the increase in heart rate indicated by 61a should be related to the increase in movement amplitude, and the decrease in heart rate indicated by 62a should be related to the decrease in movement amplitude. In addition, it can also be judged by the correlation between physiological parameter changes and movement amplitude, and / or the correlation between alarm and movement amplitude. For example, at 61c corresponding to 61a in time on the physiological parameter-movement amplitude correlation curve, the correlation coefficient increases positively, and at 61d corresponding to 61a in time on the alarm-movement amplitude correlation curve, the correlation coefficient also increases positively. On the physiological parameter-motion amplitude correlation curve, at point 62c corresponding in time to point 62a, the correlation coefficient increases negatively, showing a positive correlation. On the alarm-motion amplitude correlation curve, at point 62d corresponding in time to point 62a, the correlation coefficient also increases negatively, showing a negative correlation.

[0085] Through the above observations and analysis, it can be realized that the patient's movement will affect the patient's physiological parameters, such as Figure 6 As shown, although the first heart rate rise and heart rate drop both triggered the heart rate over-limit alarm, since both changes were positively correlated with activity, they showed normal physiological change characteristics, so it can be judged that the alarms generated at 61a and 62a may not be true physiological alarms, nor technical alarms caused by sensor damage or poor contact, but alarms caused by patient movement. When medical staff still cannot accurately judge physiological abnormalities and alarms, they can also click on the time point of interest on the curve, so that a window for viewing details is displayed on the display unit, allowing medical staff to choose to view parameter details and patient videos for a preset time period before and after the selected time point.

[0086] In addition, medical staff can find potential abnormalities by observing the movement amplitude trend graph, the physiological parameter-movement amplitude correlation curve and / or the alarm-movement amplitude correlation curve. For example, the second heart rate rise at 63a occurs during the period of decreased activity, and a small parameter increase is shown on the curve, but no alarm is triggered. However, the movement amplitude decreases at 63b corresponding to 63a in time on the movement amplitude trend graph, and a negative correlation is shown at 63c corresponding to 63a in time on the physiological parameter-movement amplitude correlation curve, and the correlation coefficient exceeds the lower limit threshold of the negative correlation. However, there is no correlation at 63d corresponding to 63a in time on the alarm-movement amplitude correlation curve, which is inconsistent with the characteristics of physiological changes and therefore needs to be paid attention to. It is possible that the patient's physical condition is abnormal, or the sensor has technical problems such as aging, damage or poor contact. At this time, the medical staff can click on the time point of interest on the curve, so that a window for viewing details is displayed on the display unit, so that the medical staff can choose to view the parameter details and the patient's video of the preset time period before and after the selected time point for further analysis and evaluation.

[0087] The above describes in an exemplary manner the situation where the trend graph of physiological parameters and the trend graph of motion amplitude, the physiological parameter-motion amplitude correlation curve and the alarm-motion amplitude correlation curve are displayed on the same time axis. Those skilled in the art should understand that, as needed, the trend graph of physiological parameters and the trend graph of motion amplitude, the physiological parameter-motion amplitude correlation curve and the alarm-motion amplitude correlation curve may be combined in other ways and displayed on the same time axis.

[0088] In the case of calculating the correlation curve, the data processing unit can also calculate the time period when the correlation coefficient exceeds the preset threshold range, and indicate it on the physiological parameter-motion amplitude correlation curve graph, for example, by bolding, displaying in different colors or flashing, so as to attract more attention from medical staff.

[0089] In the case where the correlation curve is calculated, the data processing unit can extract the overall measurement value of the correlation coefficient within the time period specified by the user, and display it on the display unit, where the overall measurement value includes at least one of the maximum value, the minimum value and the average value. Thus, the medical staff can evaluate the overall situation of the patient's movement correlation.

[0090] When the data processing unit is the processor of the central station, the data processing unit can also obtain the overall measurement value of the correlation of all patients in the same department monitored and / or managed by it based on the user's instructions, and highlight the patients with high and / or low overall measurement value rankings on the display unit to attract more attention from medical staff.

[0091] In addition, the activity correlation result is a scatter plot of the selected physiological parameter and the movement amplitude at the corresponding time, such as Figure 7 Shown is a scatter plot of heart rate and exercise amplitude. It can be seen from the figure that the changes in exercise amplitude and physiological parameters are generally positively correlated. When there are abnormal points in the graph, it also means that the patient's condition at that time point may be abnormal.

[0092] In summary, patient activity information can improve the quality of patient monitoring in many ways:

[0093] Early activity status assessment: When the patient's physical condition is relatively stable, early activity should be carried out as early as possible. Medical staff need to pay close attention to the patient's heart rate, blood pressure and other physiological indicators during activities. When the physiological indicators exceed the allowable range, the patient's activity level needs to be adjusted in time.

[0094] Early detection of changes in patients' condition: When patients engage in activities of the same intensity, different changes in their physiological parameters reflect changes in their condition.

[0095] Detecting abnormal device connection: Patient activity information can help confirm the motion interference of physiological parameters. Under the same amplitude of limb movement, the increase in the patient's certain physiological parameter motion interference alarms often means abnormalities in the device accessory connection (such as poor ECG electrode fit, blood oxygen sensor displacement, IBP bending, etc.), which need to be handled and corrected in time.

[0096] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented. For example, the image processing unit can be a chip or a plurality of chips, the data processing unit can be a chip or a plurality of chips, and when the data processing unit is composed of a plurality of chips, each chip can perform one or more functions set, for example, some chips are used to process physiological parameters, some chips are used to process the received patient activity information and calculate the correlation coefficient, some chips are used to process the visual data, or two functions are implemented by one chip. Of course, those skilled in the art can understand that the functions of the image processing unit and the data processing unit can also be implemented by one chip.

[0097] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A patient monitoring system, characterized in that: include: An image processing unit, used to obtain video data including the patient output by the image acquisition device, and analyze the video data to obtain activity information of the patient; Display unit; A data processing unit is used to receive physiological parameter signals collected from at least one patient and obtain monitoring results for each patient, wherein the monitoring results include parameter values ​​and / or alarm events of at least one physiological parameter. The data processing unit is also used to receive activity information of the patient and control the display unit to display the monitoring results and activity information corresponding to the same patient and generated in the same preset time interval on the same screen.

2. The patient monitoring system according to claim 1, characterized in that: The activity information includes a motion analysis result indicating that the patient is in a motion state, and the data processing unit controls the display unit to display the motion analysis result and the monitoring result of the same patient in a display area corresponding to the patient.

3. The patient monitoring system according to claim 2, characterized in that: The data processing unit displays the patient's designated period or current motion video in a predetermined area of ​​a display unit based on a triggering operation instruction for the motion analysis result.

4. The patient monitoring system according to claim 1, wherein: The data processing unit controls the display unit to present the monitoring results in the form of real-time parameter values ​​and / or in the form of at least one physiological parameter trend graph advancing over time, and the activity information includes the patient's movement amplitude.

5. The patient monitoring system according to claim 4, characterized in that: The data processing unit controls the display unit to display the monitoring result of the same patient and the motion amplitude value corresponding to the monitoring result in time or the amplitude diagram used to characterize the size of the motion amplitude in the display area corresponding to the patient.

6. The patient monitoring system according to claim 5, characterized in that: The amplitude graph used to characterize the magnitude of the motion amplitude is a motion amplitude trend graph advancing over time.

7. The patient monitoring system according to claim 6, characterized in that: The movement amplitude trend graph and the physiological parameter trend graph are displayed on the same time axis.

8. The patient monitoring system according to claim 6, wherein: Based on the time point selected by the user on the trend graph of the physiological parameters or the trend graph of the motion amplitude, the data processing unit displays the details of the monitoring results for a predetermined time period before and after the selected time point on the display unit, and / or plays back the video data for a predetermined time period before and after the selected time point.

9. The patient monitoring system according to claim 4, characterized in that: The data processing unit calculates an activity correlation result of the selected physiological parameter and the movement amplitude corresponding thereto in time based on the physiological parameter selected from the at least one physiological parameter, and displays the activity correlation result on the display unit.

10. The patient monitoring system according to claim 9, characterized in that: The activity correlation result is a real-time correlation coefficient; or the activity correlation result is a physiological parameter-motion amplitude correlation curve obtained according to the correlation coefficient that changes over time, and the data processing unit displays the physiological parameter-motion amplitude correlation curve and the trend graph of the selected physiological parameter on the same time axis on the display unit.

11. The patient monitoring system according to claim 4, characterized in that: The data processing unit also calculates the correlation coefficient between the alarm event and the corresponding movement amplitude in time based on the alarm event in the monitoring result, and further obtains the alarm-movement amplitude correlation curve according to the correlation coefficient changing with time, and displays the alarm-movement amplitude correlation curve and the trend graph of the physiological parameters causing the alarm on the same time axis on the display unit.

12. The patient monitoring system according to claim 9, wherein: The activity correlation result is a scatter plot of the selected physiological parameter and the movement amplitude at the corresponding time.

13. A patient monitoring system, characterized in that: include: An image analysis unit, used to obtain video data including a patient output by an image acquisition device, and analyze the video data to obtain activity information of the patient, wherein the activity information includes a movement amplitude of the patient; Display unit; A data processing unit is used to receive physiological parameter signals collected from at least one patient and obtain monitoring results for each patient, wherein the monitoring results include parameter values ​​and / or alarm events of at least one physiological parameter, and the data processing unit calculates the activity correlation results of the patient based on the monitoring results and the movement amplitude corresponding thereto in time, and controls the display unit to display the monitoring results and the activity correlation results corresponding to the same patient and corresponding in time on the same screen, wherein the activity correlation result is a correlation coefficient or an activity correlation curve obtained according to the correlation coefficient that changes over time.

14. The patient monitoring system according to claim 13, wherein: The activity correlation curve includes a physiological parameter-motion amplitude correlation curve obtained based on a physiological parameter selected from at least one physiological parameter and the corresponding motion amplitude in time, and / or an alarm-motion amplitude correlation curve obtained based on an alarm event in the monitoring result and the corresponding motion amplitude in time.

15. The patient monitoring system according to claim 10, 11 or 13, characterized in that: Described: The data processing unit displays the details of the monitoring results of a predetermined time period before and after the selected time point on the display unit based on the time point selected by the user on the trend graph or correlation curve of the physiological parameters, and / or plays back the video data of the predetermined time period before and after the selected time point.

16. The patient monitoring system according to claim 10 or 13, characterized in that: The data processing unit extracts the overall measurement value of the correlation coefficient within the time period specified by the user, and displays it on the display unit, wherein the overall measurement value includes at least one of a maximum value, a minimum value and an average value; And / or the data processing unit calculates the time period during which the correlation coefficient exceeds a preset threshold range, and indicates it on the physiological parameter-motion amplitude correlation curve graph.

17. The patient monitoring system according to claim 10 or 13, characterized in that: The data processing unit is a processor of a central station for monitoring and / or managing at least one patient. Based on the user's instructions, the data processing unit obtains an overall measurement value of the correlation of all patients in the same department that it monitors and / or manages, and highlights the patients who rank high and / or low in the overall measurement value on a display unit.

18. The patient monitoring system according to claim 1 or 13, characterized in that: The image processing unit also analyzes whether the patient is in bed and whether there are other people beside the bed based on the video data, and outputs the analysis result to the data processing unit. The data processing unit displays the current video data of the patient on the display unit based on the alarm event in the monitoring result, when the analysis result of the patient corresponding to the alarm event is that the patient is in bed and there are no other people beside the bed.

19. The patient monitoring system according to claim 1 or 13, characterized in that: The data processing unit is a processor of a patient bedside monitor.

20. A patient monitoring method, characterized in that: include: Receiving video data including the patient output by the image acquisition device; Analyzing the video data to obtain activity information of the patient; Receiving a physiological parameter signal collected from at least one patient and obtaining a monitoring result for each patient, wherein the monitoring result includes a parameter value and / or an alarm event of at least one physiological parameter; The monitoring results and activity information corresponding to the same patient and generated in the same preset time interval are displayed on the same screen on the display unit.

21. The patient monitoring method according to claim 20, characterized in that: The activity information includes motion analysis results characterizing that the patient is in a state of motion, and / or the activity information includes the patient's motion amplitude. The monitoring results and activity information corresponding to the same patient and generated in the same preset time interval are displayed on the same screen on the display unit, including: displaying the motion amplitude value corresponding to the monitoring result in time or an amplitude diagram used to characterize the size of the motion amplitude next to the monitoring result of the same patient on the display unit.

22. The patient monitoring method according to claim 21, characterized in that: The motion amplitude is calculated by the following steps: detecting a patient in the image based on the image frame of the video data, and determining a region of interest including the patient based on the detected patient; Based on the time-ordered image frames sampled from the video data, comparing the regions where there are differences between every two adjacent image frames, and calculating the difference between every two adjacent image frames as the inter-frame difference; Mark the areas of difference in the region of interest; The calculated frame differences are accumulated in real time to obtain the accumulated absolute difference value; The accumulated absolute difference is compared with a predetermined threshold. When the accumulated absolute difference is less than the predetermined threshold, it is considered that the patient has no body movement. Otherwise, it is considered that the patient has movement. Then, the marked regions with differences in the current region of interest are connected to obtain the movement region. Calculate the proportion of the motion area in the region of interest and obtain the real-time motion amplitude.

23. The patient monitoring method according to claim 21, characterized in that: The amplitude graph used to characterize the size of the movement amplitude is a movement amplitude trend graph advancing over time. The monitoring results are presented on the display unit in the form of real-time parameter values ​​and / or in the form of at least one physiological parameter trend graph advancing over time. The movement amplitude trend graph and the physiological parameter trend graph are displayed on the same time axis.

24. The patient monitoring method according to claim 21, characterized in that The method further comprises: calculating an activity correlation result between the selected physiological parameter and the movement amplitude corresponding thereto in time from the physiological parameter selected from the at least one physiological parameter, and displaying the activity correlation result on a display unit.

25. The patient monitoring method according to claim 21, characterized in that It also includes calculating the correlation coefficient between the alarm event and the movement amplitude corresponding to it in time based on the alarm event in the monitoring result, and further obtaining an alarm-movement amplitude correlation curve according to the correlation coefficient that changes with time, and displaying the alarm-movement amplitude correlation curve and the trend graph of the physiological parameters that cause the alarm on the same time axis on the display unit.

26. A patient monitoring method, characterized in that: include: Receiving video data including the patient output by the image acquisition device; Analyzing the video data to obtain activity information of the patient; Receiving a physiological parameter signal collected from at least one patient and obtaining a monitoring result for each patient, wherein the monitoring result includes a parameter value and / or an alarm event of at least one physiological parameter; The activity correlation results of the monitoring results and the movement amplitude corresponding thereto in time are calculated based on the monitoring results, and the monitoring results and the activity correlation results corresponding to the same patient and corresponding in time are displayed on the same screen on the display unit, and the activity correlation results are real-time correlation coefficients or activity correlation curves obtained according to the correlation coefficients that change over time.

27. The patient monitoring method according to claim 26, characterized in that: The activity correlation curve includes a physiological parameter-motion amplitude correlation curve obtained based on a physiological parameter selected from at least one physiological parameter and the corresponding motion amplitude in time, and / or an alarm-motion amplitude correlation curve obtained based on an alarm event in the monitoring result and the corresponding motion amplitude in time.

28. A computer-readable storage medium, characterized in that: The medium stores a program, which can be executed by a processor to implement the method according to any one of claims 20 to 27.

Images