ICU ward intelligent sensing control system based on vision

Abstract

The invention discloses an ICU ward intelligent sensing control system based on vision. A 2D visual sensor is arranged above an entrance area, and non-contact and non-inductive monitoring is carried out on behavior specifications of medical staff at the entrance area. A 3D visual sensor is installed above a sickbed area, infrared laser point cloud is adopted to obtain three-dimensional information of a target in a field of view, and touch recognition is carried out through distance features. When a non-compliance phenomenon is found, a prompt can be given, occurrence and diffusion of a drug-resistant bacterium price difference infection phenomenon can be avoided, a touched part can be prompted, and timely disinfection treatment can be prompted. The infrared light source is adopted to obtain the information of the target in the view field, non-contact and non-inductive operation is achieved, and the conflict psychology of the patient is greatly eliminated. A centralized monitoring system is adopted, and all monitoring information data are summarized and automatically stored, so that a digital twin system can be generated, and statistical analysis, quantitative evaluation on ward sensing control and quantitative evaluation on compliance of medical staff can be realized.

Description

Technical Field

[0001] The present invention relates to a ward infection control system, and in particular to a vision-based ICU ward intelligent infection control system. Background Art

[0002] Hospital-acquired infections are a prominent public health issue worldwide. The infection rate of drug-resistant bacteria in critically ill patients in ICU is 33.9%-56.4%, which leads to a significant increase in the use of antibiotics, prolonged hospitalization, increased medical expenses, and a mortality rate of 30-50%. How to scientifically prevent, diagnose and treat drug-resistant bacterial infections has become a focus of common concern in global public health and a major clinical problem that needs to be solved urgently. Especially in the intensive care unit (ICU), infections in critically ill patients are characterized by high incidence, high drug resistance, high transmissibility and high mortality. Many factors such as medical staff contact, mobile equipment, and invasive interventional operations can lead to cross-infection of drug-resistant bacteria. Strengthening the prevention and control of drug-resistant bacteria and preventing cross-infection are the primary key links in the diagnosis and treatment of infections in critically ill patients in ICU.

[0003] In daily medical activities in the ICU, contact isolation of patients with drug-resistant bacteria is the core link of the comprehensive prevention and control strategy, including wearing isolation gowns and hand hygiene to avoid direct contact between medical staff and patients and contaminated shared medical surfaces. However, in clinical practice, it was found that the overall compliance of medical staff with contact isolation was only 45-66%, among which the compliance with wearing isolation gowns, wearing gloves, and hand disinfection after contact was 34%, 58%, and 66%, respectively. Because the behavior of medical staff is not standardized (for example, not wearing gloves, forgetting to wear isolation gowns, forgetting to disinfect hands in time, touching objects in the ward and the patient's body by mistake, etc.), cross-infection of drug-resistant bacteria in wards (especially ICU wards) occurs. Therefore, it is crucial and imperative to improve the level of prevention and control of drug-resistant bacteria in wards, prevent cross-infection, and reduce mortality through objective and reliable technical means under the current medical conditions and resources.

[0004] To this end, based on various advanced technical means, the intelligent identification of medical supplies and medical staff behavior patterns in ICU wards is realized, the contact status of medical staff and medical supplies during contact isolation is monitored, and timely alarms and prompts are given, which can effectively reduce or even avoid the occurrence of cross-infection of drug-resistant bacteria. In addition, further analysis and processing of the massive information saved by these monitoring means can also clarify the implementation of ward infection control during contact isolation and existing work problems, further improve system norms, improve implementation quality, ensure the effect of drug-resistant bacteria prevention and control, and achieve the goal of reducing the spread of drug-resistant bacteria in ICU, reducing patients' exposure to drug-resistant bacteria and antibiotics, and improving the level of intelligent medical care in wards and even the entire hospital.

[0005] However, looking at ICU wards at home and abroad, the existing infection control means are basically blank, and there are no special ward infection control monitoring means and equipment. Basically, it all depends on the medical staff's awareness, self-discipline, and self-control. When the medical tasks are heavy and the medical activities are carried out continuously for a long time, the individual's awareness, self-discipline, and self-control ability will inevitably decline. Summary of the invention

[0006] The present invention aims at the current situation that the medical field lacks ICU ward infection control monitoring means, and proposes a vision-based ICU ward intelligent infection control system. A 2D visual sensor is placed above the entrance area to perform non-contact and non-sensing monitoring of the behavior norms of medical staff at the entrance. A 3D visual sensor is installed above the bed area, and an infrared laser point cloud is used to obtain the three-dimensional information of the target in the field of view, and touch recognition is performed through distance characteristics. When non-compliant phenomena are found, prompts can be given to avoid the occurrence and spread of drug-resistant bacteria price difference infection phenomena, and the touched parts can be prompted to prompt timely disinfection, which greatly improves the infection control level and actual effect of the ICU ward. The infrared light source is used to obtain the information of the target in the field of view, non-contact, non-sensing operation, which greatly eliminates the patient's resistance and improves the application scalability. A centralized monitoring system is used to summarize all monitoring information data and automatically save them, so that a digital twin system can be generated, and statistical analysis can be performed to quantitatively evaluate the infection control of the ward and the compliance of medical staff.

[0007] The present invention is achieved through the following technical solutions: The present invention provides a vision-based intelligent sensor control system for ICU wards, wherein the intelligent sensor control system comprises three parts: a 2D vision sensor, a 3D vision sensor, a controller and a monitoring system, wherein: The 2D visual sensor is arranged above the entrance area of ​​the ward and is used to monitor the behavioral norms of medical staff entering the ward; the 2D visual sensor is equipped with a 2D camera, a 2D lens, a light source, a prompter, a 2D protective cover, a driver and a 2D base; the 2D camera is a high-definition camera so that it can identify detailed information of the target in the field of view; the 2D lens is a medium-focus lens that focuses on the medical staff in the entrance area; the light source is an infrared light source to achieve non-sensing monitoring; the prompter is an acousto-optic prompter for prompting non-standard behavior of medical staff; the driver is a microcontroller for controlling the operation of the light source and the prompter; the 2D protective cover is a spherical protective cover for protecting the entire 2D visual sensor and facilitating daily disinfection; the 2D base is made of lightweight alloy material to ensure strength and reduce weight; The 3D visual sensor is arranged above the bed area in the ward and is used to monitor the behavioral norms of medical staff in the bed area; the 3D visual sensor is equipped with a 3D camera, a 3D lens, a projector, a light source, a prompter, a driver, a 3D protective cover and a 3D base; the 3D camera is a high-definition camera, there are two of them, and the three-dimensional target and its orientation in the field of view can be identified through binocular stereo vision technology; the 3D lens is a short-focus lens, there are two of them, which are installed on the 3D camera respectively, and the field of view can cover the bed and the medical staff and medical equipment around it; the projector is an infrared laser point cloud projector, which forms an infrared laser point cloud within the field of view for three-dimensional target monitoring; the light source is an infrared light source, which is used for scene target recognition so as to realize non-sensing monitoring; the prompter is an acousto-optic prompter, which is used to prompt the non-standard behavior of medical staff; the driver is a microcontroller, which is used to control the work of the projector, the light source and the prompter; the 3D protective cover is a rectangular protective cover, which is used to protect the entire 3D visual sensor and facilitate daily disinfection; the 3D base is made of lightweight alloy material to ensure strength and reduce weight; The core of the controller is a high-performance integrated fanless industrial computer, which can be directly connected to the 2D vision sensor and the 3D vision sensor to process and judge the data, and upload the results to the monitoring system; The monitoring system is a computer server system, which is responsible for receiving the processing results and key frame image data from the controller, and is used for various information display and data twinning required for background monitoring, recording various relevant data, and can perform statistical analysis and evaluation, and can also send relevant information to designated personnel as needed.

[0008] The vision-based ICU ward intelligent sensing control system of the present invention is also characterized in that the 3D vision sensor can also adopt a combination of a camera lens group and a digital projector to realize the monitoring of three-dimensional targets and their orientations in the field of view through a digital grating projection method.

[0009] The present invention provides a method for monitoring the behavior of medical staff in the entry area of ​​the above-mentioned vision-based ICU ward intelligent infection control system, which is as follows: (1) First, the controller turns off the light source in the 2D vision sensor and obtains a frame of image as the background image; (2) Then, the controller turns on the light source in the 2D vision sensor and obtains a frame of image as the current image; (3) The controller performs differential processing on the current image and the background image to obtain a differential image; (4) If the number of effective pixels of the differential image is less than the set threshold, it can be determined that no target appears, and the above process is repeated; (5) If the number of effective pixels of the differential image exceeds the set threshold, it can be determined that a target has appeared. At this time, the target is identified. If it is not a medical staff, the above process is repeated; (6) After confirming that the target is a medical staff member, the controller identifies whether the medical staff is fully dressed, including gloves, isolation clothing, hats, protective goggles, etc. If any of the equipment is missing, the controller will give an audible and visual reminder through the reminder and upload the information to the monitoring system to alert. (7) After confirming that the medical staff is fully dressed, the controller continuously captures the video stream to determine whether the medical staff has completed the hand disinfection process; if the hand disinfection process is not completed or the hand disinfection is not compliant, the controller will give an audible and visual prompt through the prompter and upload it to the monitoring system to alarm.

[0010] The present invention provides a method for monitoring the behavior of medical staff in the bed area of ​​the above-mentioned vision-based ICU ward intelligent sensory control system, which is as follows: (1) First, the controller turns off the projector in the 3D vision sensor and obtains a frame of image as the background image; (2) Then, the controller turns on the projector in the 3D vision sensor, projects the laser point cloud into the field of view, and obtains a frame of image as the current image; (3) The controller performs differential processing on the current image and the background image to obtain a point cloud image; (4) The controller extracts the three-dimensional coordinates of all point clouds in the point cloud image; (5) The controller uses the point cloud to reconstruct the 3D scene to identify specific targets such as hospital beds, medical equipment, and medical staff; (6) The controller calculates the three-dimensional position coordinates of each target and calculates the minimum distance between the medical staff and other targets; (7) If the distance between the medical staff and the designated target is less than the set upper threshold, it is determined that the medical staff has touched the target. The controller can then give an audible and visual prompt through the prompter and upload the alarm to the monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the composition of the vision-based ICU ward intelligent sensory control system of the present invention; Figure 2 This is a schematic diagram of the composition principle of the 2D vision sensor of the present invention; Figure 3 This is a schematic diagram of the composition principle of the 3D vision sensor of the present invention; Figure 4 It is a schematic diagram of the principle of the controller and two visual sensors of the present invention; Figure 5 It is a schematic diagram of the principle of monitoring the behavior norms of medical personnel in the entry area of ​​the present invention; Figure 6 This is a schematic diagram of the principle of monitoring the behavior of medical staff in the bed area of ​​the present invention; In the figure, 1-2D sensor, 2-3D sensor, 3-controller, 4-monitoring system, 5-2D camera, 6-2D lens, 7-light source, 8-prompter, 9-driver, 10-2D protective cover, 11-2D base, 12-3D camera, 13-3D lens, 14-projector, 15-3D protective cover, 16-3D base, 17-industrial computer. DETAILED DESCRIPTION

[0012] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0013] The present invention is achieved through the following technical solutions: The present invention provides a vision-based intelligent sensor control system for ICU wards, wherein the intelligent sensor control system comprises three parts: a 2D vision sensor, a 3D vision sensor, a controller and a monitoring system. Figure 1 As shown, where: The 2D visual sensor 1 is arranged above the entrance area of ​​the ward and is used to monitor the behavior of medical staff entering the ward; the 2D visual sensor 1 has a built-in 2D camera 5, a 2D lens 6, a light source 7, a prompter 8, a driver 9, a 2D protective cover 10 and a 2D base 11. Figure 2 As shown; the 2D camera 5 is a high-definition camera so that it can identify the detailed information of the target in the field of view; the 2D lens 6 is a medium-focus lens, focusing on the medical staff in the entry area; the light source 7 is an infrared light source to achieve non-sensing monitoring; the prompter 8 is an acoustic and optical prompter, which is used to prompt the non-standard behavior of medical staff; the driver 9 is a microcontroller, which is used to control the operation of the light source 7 and the prompter 8; the 2D protective cover 10 is a spherical protective cover, which is used to protect the entire 2D visual sensor 1 and facilitate daily disinfection; the 2D base 11 is a lightweight alloy material to ensure strength and reduce weight; The 3D visual sensor 2 is arranged above the bed area in the ward and is used to monitor the behavior of medical staff in the bed area; the 3D visual sensor 2 has a built-in 3D camera 12, a 3D lens 13, a projector 14, a light source 7, a prompter 8, a driver 9, a 3D protective cover 15 and a 3D base 16. Figure 3As shown; the 3D camera 12 is a high-definition camera, there are two of them, namely 12a and 12b, which can identify the three-dimensional target and its orientation in the field of view through binocular stereo vision technology; the 3D lens 13 is a short-focus lens, there are two of them, namely 13a and 13b, which are installed on the 3D cameras 12a and 12b respectively, and the field of view can cover the bed and the medical staff and medical equipment around it; the projector 14 is an infrared laser point cloud projector, which forms an infrared laser point cloud within the field of view for three-dimensional target monitoring; the light source 7 is an infrared light source, which is used for scene target recognition so as to achieve non-sensing monitoring; the prompter 8 is an acousto-optic prompter, which is used to prompt the non-standard behavior of medical staff; the driver 9 is a microcontroller, which is used to control the projector 14, the light source 7 and the prompter 8 to work; the 3D protective cover 15 is a rectangular protective cover, which is used to protect the entire 3D visual sensor 2 and facilitate daily disinfection; the 3D base 16 is a lightweight alloy material to ensure strength and reduce weight; The core of the controller 3 is a high-performance integrated fanless industrial computer 17, which can be directly connected to the 2D vision sensor 1 and the 3D vision sensor 2. Figure 4 As shown, data processing and judgment are performed, and the results are uploaded to the monitoring system 4; The monitoring system 4 is a computer server system, which is responsible for receiving the processing results and key frame image data from the controller 3, and is used for various information display and data twinning required for background monitoring, recording various relevant data, and can perform statistical analysis and evaluation, and can also send relevant information to designated personnel as needed.

[0014] The vision-based ICU ward intelligent sensing control system of the present invention is also characterized in that the 3D vision sensor 2 can also adopt a combination of a camera lens group and a digital projector to realize the monitoring of three-dimensional targets and their orientations in the field of view through a digital grating projection method. The present invention provides a method for monitoring the behavior of medical staff in the entry area of ​​the above-mentioned vision-based ICU ward intelligent sensory control system, such as Figure 5 As shown, the details are as follows: (1) First, the controller 3 turns off the light source 7 in the 2D vision sensor 2 and obtains a frame of image as the background image Ib; (2) Then, the controller 3 turns on the light source 7 in the 2D vision sensor 2 to obtain a frame of image as the current image Ic; (3) The controller 3 performs a differential process on the current image Ic and the background image Ib to obtain a differential image Id; (4) If the number of valid pixels in the differential image Id is less than the set threshold, it can be determined that no target appears, and the above process is repeated; (5) If the number of valid pixels in the differential image Id exceeds the set threshold, it can be determined that a target appears. At this time, target identification is performed. If it is not a medical staff, the above process is repeated; (6) After the controller 3 confirms that the target is a medical staff, the controller 3 identifies whether the medical staff is fully dressed, including gloves, isolation clothing, hats, protective goggles, etc. If any of the equipment is missing, the controller 3 gives an audible and visual reminder through the reminder 8 and uploads the information to the monitoring system 4 to alert. (7) After confirming that the medical staff is properly dressed, the controller 3 continuously captures the video stream to determine whether the medical staff has completed the hand disinfection process; if the hand disinfection process is not completed or the hand disinfection is not compliant, the controller 3 gives an audible and visual prompt through the prompter 8 and uploads it to the monitoring system 4 to alarm.

[0015] The present invention provides a method for monitoring the behavior of medical staff in the bed area of ​​the above-mentioned vision-based ICU ward intelligent sensory control system, such as Figure 6 As shown, the details are as follows: (1) First, the controller 3 turns off the projector 14 in the 3D vision sensor 2 and obtains a frame of image as the background image Ib; (2) Then, the controller 3 turns on the projector 14 in the 3D vision sensor 2, projects the laser point cloud into the field of view, and obtains a frame of image as the current image Ic; (3) The controller 3 performs differential processing on the current image Ic and the background image Ib to obtain a point cloud image Ip; (4) Controller 3 extracts the three-dimensional coordinates (x1, y1, z1), (x2, y2, z2), …, (xn, yn, zn) of all point clouds Pi (i=1, 2, …n) in the point cloud image Ip; (5) Controller 3 uses all point clouds Pi (i=1,2,…n) to reconstruct the three-dimensional scene to identify specific targets such as beds, medical equipment, and medical staff; (6) The controller 3 calculates the three-dimensional position coordinates of each target and calculates the minimum distance D between the medical staff and other targets; (7) If the distance between the medical staff and the designated target is less than the set upper threshold D0, it is determined that the medical staff has touched the target, and the controller 3 can give an audible and visual prompt through the prompter 8 and upload it to the monitoring system 4 to alarm. Compared with the prior art, the present invention has the following beneficial effects: (1) The vision-based intelligent infection control system for ICU wards proposed in this invention uses computer vision methods to obtain behavioral norms information of medical staff. When non-compliance occurs, on-site reminders are given in the first place to avoid the occurrence and spread of cross-infection of drug-resistant bacteria, greatly improving the infection control level and actual effect of the ICU ward.

[0016] (2) The vision-based intelligent sensor control system for ICU ward proposed in the present invention uses an infrared light source to obtain information about targets within the field of view. It operates without contact or feeling, which greatly eliminates the patient's resistance and improves the scalability of the application.

[0017] (3) The vision-based intelligent infection control system for ICU wards proposed in the present invention uses infrared laser point cloud to obtain three-dimensional information of targets in the field of view, and uses distance features to perform touch recognition. When an illegal touch occurs, an on-site reminder is given immediately to avoid the occurrence and spread of cross-infection of drug-resistant bacteria. The system can also indicate the touched parts and prompt timely disinfection, thereby greatly improving the infection control level and actual effect of the ICU ward.

[0018] (4) The vision-based intelligent infection control system for ICU wards proposed in the present invention adopts a centralized monitoring system to aggregate all monitoring information data and automatically save them, so as to generate a digital twin system, and perform statistical analysis to quantitatively evaluate the infection control in the wards and the compliance of medical staff.

[0019] (5) The vision-based intelligent sensor control system for ICU ward proposed by the present invention adopts a spherical or rectangular integral protective cover, which is convenient for disinfection, verification and standardization, and has strong versatility and good adaptability. The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A vision-based intelligent sensor control system for ICU wards, the intelligent sensor control system comprises three parts: a 2D vision sensor, a 3D vision sensor, a controller and a monitoring system, wherein: The 2D visual sensor is arranged above the entrance area of ​​the ward and is used to monitor the behavioral norms of medical staff entering the ward; the 2D visual sensor is equipped with a 2D camera, a 2D lens, a light source, a prompter, a 2D protective cover, a driver and a 2D base; the 2D camera is a high-definition camera so that detailed information of the target in the field of view can be identified; the 2D lens is a medium-focus lens that focuses on the medical staff in the entrance area; the light source is an infrared light source to achieve non-sensing monitoring; the prompter is an acoustic and optical prompter for prompting the non-standard behavior of medical staff; the driver is a microcontroller for controlling the operation of the light source and the prompter; the 2D protective cover is a spherical protective cover for protecting the entire 2D visual sensor and facilitating daily disinfection; The 2D base is made of lightweight alloy material to ensure strength and reduce weight; The 3D visual sensor is arranged above the bed area in the ward and is used to monitor the behavioral norms of medical staff in the bed area; the 3D visual sensor is equipped with a 3D camera, a 3D lens, a projector, a light source, a prompter, a driver, a 3D protective cover and a 3D base; the 3D camera is a high-definition camera, there are two of them, and the three-dimensional target and its orientation in the field of view can be identified through binocular stereo vision technology; the 3D lens is a short-focus lens, there are two of them, which are installed on the 3D camera respectively, and the field of view can cover the bed and the medical staff and medical equipment around it; the projector is an infrared laser point cloud projector, which forms an infrared laser point cloud within the field of view for non-sensing monitoring of three-dimensional targets; the light source is an infrared light source, which is used for scene target recognition in order to achieve non-sensing monitoring; the prompter is an acousto-optic prompter, which is used to prompt the non-standard behavior of medical staff; the driver is a microcontroller, which is used to control the work of the projector, the light source and the prompter; the 3D protective cover is a rectangular protective cover, which is used to protect the entire 3D visual sensor and facilitate daily disinfection; The 3D base is made of lightweight alloy material to ensure strength and reduce weight; The core of the controller is a high-performance integrated fanless industrial computer, which can be directly connected to the 2D vision sensor and the 3D vision sensor to process and judge the data, and upload the results to the monitoring system; The monitoring system is a computer server system, which is responsible for receiving the processing results and key frame image data from the controller, and is used for various information display and data twinning required for background monitoring, recording various relevant data, and can perform statistical analysis and evaluation, and can also send relevant information to designated personnel as needed.

2. The vision-based ICU ward intelligent sensory control system according to claim 1 is further characterized by: The 3D vision sensor may also be a combination of a camera lens group and a digital projector, and a digital grating projection method may be used to monitor the three-dimensional target and its position in the field of view.

3. The present invention provides a method for monitoring the behavior of medical staff in the entry area of ​​the above-mentioned vision-based ICU ward intelligent infection control system, which is as follows: (1) First, the controller turns off the light source in the 2D vision sensor and obtains a frame of image as the background image; (2) Then, the controller turns on the light source in the 2D vision sensor and obtains a frame of image as the current image; (3) The controller performs differential processing on the current image and the background image to obtain a differential image; (4) If the number of effective pixels of the differential image is less than the set threshold, it can be determined that no target appears, and the above process is repeated; (5) If the number of effective pixels of the differential image exceeds the set threshold, it can be determined that a target has appeared. At this time, the target is identified. If it is not a medical staff, the above process is repeated; (6) After confirming that the target is a medical staff member, the controller identifies whether the medical staff is fully dressed, including gloves, isolation clothing, hats, and protective goggles; if any of the equipment is missing, the controller gives an audible and visual reminder through the reminder and uploads it to the monitoring system to alarm; (7) After confirming that the medical staff is fully dressed, the controller continuously captures the video stream to determine whether the medical staff has completed the hand disinfection process; if the hand disinfection process is not completed or the hand disinfection is not compliant, the controller will give an audible and visual prompt through the prompter and upload it to the monitoring system to alarm.

4. The present invention provides a method for monitoring the behavior of medical staff in the bed area of ​​the above-mentioned vision-based ICU ward intelligent sensory control system, which is as follows: (1) First, the controller turns off the projector in the 3D vision sensor and obtains a frame of image as the background image; (2) Then, the controller turns on the projector in the 3D vision sensor, projects the laser point cloud into the field of view, and obtains a frame of image as the current image; (3) The controller performs differential processing on the current image and the background image to obtain a point cloud image; (4) The controller extracts the three-dimensional coordinates of all point clouds in the point cloud image; (5) The controller uses the point cloud to reconstruct the three-dimensional scene to identify specific targets such as hospital beds, medical equipment, and medical staff; (6) The controller calculates the three-dimensional position coordinates of each target and calculates the minimum distance between the medical staff and other targets; (7) If the distance between the medical staff and the designated target is less than the set high threshold, it is determined that the medical staff has touched the target. The controller can then give an audible and visual prompt through the prompter and upload the alarm to the monitoring system.

Images