Method, device, equipment and medium for identifying hand touch behavior of medical staff in hospital

Abstract

The application discloses a method, device, equipment and medium for identifying hand touch behavior of medical staff in a hospital, wherein the method comprises the following steps: collecting multiple video information of different angles of each bed in a ward in the hospital; calculating the tracking position and behavior of each medical staff or patient, the ROI of hand sanitizer, a bed and fixed medical equipment, and the Mask coordinates of the objects in the ROI in each video information in a first video stream and a second video stream; and identifying the hand touch behavior of each medical staff or patient in the ward according to the tracking position and behavior of each medical staff or patient, the ROI of hand sanitizer, a bed and fixed medical equipment, and the Mask coordinates of the objects in the ROI. The embodiment of the application can accurately observe, track and evaluate the hand touch behavior of medical staff through images, effectively reduce hospital infection, and effectively trace the source when a virus infection event occurs, thereby solving the problem that the hand contact condition of medical staff in a hospital cannot be monitored and counted in real time.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a method, apparatus, equipment and medium for identifying the hand-touching behavior of medical staff in a hospital. Background Technology

[0002] Hand hygiene is a measure taken to address the risk of cross-infection among healthcare workers. In the context of the COVID-19 pandemic, infection control in hospitals has become crucial in suppressing virus transmission, with a focus on isolating transmission routes. Hand hygiene among healthcare workers is the most important, simplest, most effective, and economical method for preventing and controlling hospital-acquired infections.

[0003] In related technologies, identifiers are installed in corridors and wards to identify the entry and exit of medical staff and switch the hand hygiene status of medical staff. At the same time, pressure sensors are added to hand sanitizer and hand disinfectant bottles to identify the usage of hand sanitizer and hand disinfectant bottles and record the hand hygiene status of medical staff.

[0004] However, the relevant technologies cannot accurately observe, track, and assess the hand-touching behavior of medical staff in real time through images, which makes it difficult to accurately identify the hand-touching behavior of medical staff in hospitals, making it difficult to reduce the risk of virus transmission. Consequently, it is impossible to effectively trace the source in the event of a virus transmission incident, which urgently needs to be solved. Summary of the Invention

[0005] This application provides a method, device, equipment, and medium for identifying the hand-touching behavior of medical staff in hospitals, in order to solve the problem that it is impossible to monitor and statistically analyze the hand-touching status of medical staff in hospitals in real time.

[0006] The first aspect of this application provides a method for identifying hand-touching behavior of medical staff in a hospital, comprising the following steps: collecting multiple video information from different angles of each bed in a hospital ward; calculating the tracking position and behavior of each medical staff member or patient in a first video stream and a second video stream, and the ROI (Region of Interest) of hand sanitizer, bed, and fixed medical equipment, as well as the mask coordinates of the object within the ROI; and identifying the hand-touching behavior of each medical staff member or patient in the ward based on the tracking position and behavior of each medical staff member or patient and the ROI of the hand sanitizer, bed, and fixed medical equipment, as well as the mask coordinates of the object within the ROI.

[0007] Optionally, in one embodiment of this application, the step of calculating the tracking position and behavior of each medical staff member or patient in the first video stream and the second video stream, and the ROI of the hand sanitizer, hospital bed, fixed medical equipment, and the mask coordinates of the object within the ROI, includes: for each camera, calling a preset YOLO object detection model based on the first video stream to distinguish between the medical staff member and the patient; calling a preset OpenPose framework to extract the human skeleton point coordinates for the medical staff member; assigning a unique identity to each medical staff member entering the ward, and unifying the identity of the same person in each video information, binding the human skeleton point coordinates with the unique identity to perform single-person position tracking in the camera's field of view, and recording the behavior of each unique identity.

[0008] Optionally, in one embodiment of this application, the step of calculating the tracking position and behavior of each medical staff member or patient in the first video stream and the ROI of the hand sanitizer, the hospital bed, and the fixed medical equipment, and the Mask coordinates of the object within the ROI, for each of the plurality of video information in the first video stream and the second video stream respectively, includes: for each camera, based on the second video stream, identifying the hand sanitizer, the hospital bed, and the fixed medical equipment using a preset Detectron2 object detection model; performing instance segmentation on the hand sanitizer, the hospital bed, and the fixed medical equipment, and marking the ROI of the hand sanitizer, the hospital bed, and the fixed medical equipment, and the Mask coordinates of the object within the ROI.

[0009] Optionally, in one embodiment of this application, the step of identifying the hand-touching behavior of each medical staff member or patient in the ward based on the tracking location and behavior of each medical staff member or patient and the ROI of the hand sanitizer, the hospital bed, the fixed medical device, and the mask coordinates of the object within the ROI includes: determining whether the wrist point coordinates in each frame of the image coincide with the mask coordinates of the object within the ROI; if it coincides with the hand sanitizer, then mark the hand washing / hand sanitizing action; if it coincides with the hospital bed, then mark the medical staff-patient contact action; if it coincides with the fixed medical device, then mark it as the device contact action.

[0010] Optionally, in one embodiment of this application, the step of identifying the hand-touching behavior of each medical staff member or patient in the ward based on the tracking location and behavior of each medical staff member or patient and the ROI of the hand sanitizer, the hospital bed, the fixed medical device, and the Mask coordinates of the object within the ROI, further includes: counting the cumulative number of markers for the handwashing / hand sanitizing action, the medical staff-patient contact action, and the device contact action; and recording the cumulative number of markers to the corresponding unique identifier when the cumulative number of markers is greater than a preset threshold.

[0011] A second aspect of this application provides an apparatus for identifying hand-touching behavior of medical staff in a hospital, comprising: a data acquisition module for acquiring multiple video information from different angles of each bed in a hospital ward; a calculation module for calculating the tracking position and behavior of each medical staff member or patient in a first video stream and a second video stream, respectively, relative to the ROI of hand sanitizer, the bed, and fixed medical equipment, and the mask coordinates of the object within the ROI; and an identification module for identifying the hand-touching behavior of each medical staff member or patient in the ward based on the tracking position and behavior of each medical staff member or patient and the ROI of the hand sanitizer, the bed, the fixed medical equipment, and the mask coordinates of the object within the ROI.

[0012] Optionally, in one embodiment of this application, the calculation module includes: a differentiation unit, used to differentiate between the medical staff and the patient based on the first video stream by calling a preset YOLO object detection model for each camera; an extraction unit, used to extract the coordinates of human skeleton points by calling a preset OpenPose framework for the medical staff; and a tracking unit, used to assign a unique identifier to each medical staff member entering the ward, unify the identifier of the same person in each video information, bind the coordinates of human skeleton points to the unique identifier, so as to perform single-person position tracking in the camera's field of view and record the behavior of each unique identifier.

[0013] Optionally, in one embodiment of this application, the calculation module further includes: a detection unit, configured to identify the hand sanitizer, the hospital bed, and the fixed medical device for each camera based on the second video stream using a preset Detectron2 target detection model; and a segmentation unit, configured to perform instance segmentation on the hand sanitizer, the hospital bed, and the fixed medical device, and mark the ROI of the hand sanitizer, the hospital bed, and the fixed medical device, as well as the Mask coordinates of the objects within the ROI.

[0014] Optionally, in one embodiment of this application, the identification module includes: a judgment unit, used to determine whether the wrist point coordinates in each frame of the image coincide with the mask coordinates of the object within the ROI; a first marking unit, used to mark handwashing / hand disinfection actions when coinciding with the hand sanitizer; a second marking unit, used to mark doctor-patient contact actions when coinciding with the hospital bed; and a third marking unit, used to mark as equipment contact actions when coinciding with the fixed medical equipment.

[0015] Optionally, in one embodiment of this application, the identification module further includes: a statistics unit, used to count the cumulative number of times the handwashing / hand disinfection action, the medical staff-patient contact behavior action, and the device contact action are marked; and a recording unit, used to record the cumulative number of marks into the corresponding unique identifier when the cumulative number of marks is greater than a preset threshold.

[0016] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for recognizing hand touch behavior of medical staff in a hospital as described in the above embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for recognizing hand-touching behaviors of medical staff in a hospital.

[0018] Therefore, the embodiments of this application have the following beneficial effects:

[0019] This application embodiment can collect multiple video feeds from different angles of each bed in a hospital ward; calculate the tracking position and behavior of each medical staff member or patient in the first and second video streams, and their relative positions (ROIs) with hand sanitizer, hospital beds, and fixed medical equipment, as well as the mask coordinates of the objects within the ROI; based on the tracking position and behavior of each medical staff member or patient, and their relative positions (ROIs) with hand sanitizer, hospital beds, and fixed medical equipment, as well as the mask coordinates of the objects within the ROI, the hand-touching behavior of each medical staff member or patient in the ward can be identified. This allows for accurate observation, tracking, and evaluation of medical staff's hand-touching behavior through images, effectively reducing hospital-acquired infections and enabling effective source tracing in the event of a viral outbreak. Therefore, it solves the problem of the inability to monitor and statistically analyze the hand contact status of medical staff in hospitals in real time.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart illustrating a method for recognizing hand-touching behavior of medical staff in a hospital, according to an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of a camera installation position according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the detection process of the Detectron2 target detection model according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the execution logic for recognizing hand-touching behavior of medical staff in a hospital, according to an embodiment of this application;

[0026] Figure 5 This is an example diagram of a device for recognizing hand-touching behavior of medical staff in a hospital, according to an embodiment of this application.

[0027] Figure 6 A schematic diagram of the structure of the electronic device provided in the application embodiment.

[0028] Explanation of reference numerals in the attached figures:

[0029] Device for recognizing hand touch behavior of medical staff in hospitals - 10; acquisition module - 100, calculation module - 200, recognition module - 300; memory - 601, processor - 602, communication interface - 603. Detailed Implementation

[0030] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0031] The following describes, with reference to the accompanying drawings, a method, apparatus, device, and medium for identifying hand-touching behavior of medical staff in hospitals according to embodiments of this application. Addressing the problems mentioned in the background art, this application provides a method for identifying hand-touching behavior of medical staff in hospitals. In this method, multiple video feeds from different angles of each bed in a hospital ward are collected. For each video feed, the tracking position and behavior of each medical staff member or patient are calculated in a first video stream and a second video stream, along with the Region of Interest (ROI) of hand sanitizer, the bed, and fixed medical equipment, and the mask coordinates of the objects within the ROI. Based on the tracking position and behavior of each medical staff member or patient, and the ROI of hand sanitizer, the bed, and fixed medical equipment, and the mask coordinates of the objects within the ROI, the hand-touching behavior of each medical staff member or patient in the ward is identified. This allows for accurate observation, tracking, and evaluation of hand-touching behavior through images, effectively reducing hospital-acquired infections and enabling effective source tracing in the event of a viral outbreak. Thus, it solves the problem of the inability to monitor and statistically analyze the hand contact status of medical staff in hospitals in real time.

[0032] Specifically, Figure 1 This is a flowchart illustrating a method for recognizing hand-touching behavior of medical staff in a hospital, as provided in an embodiment of this application.

[0033] like Figure 1 As shown, the method for identifying hand-touching behavior of medical staff in a hospital includes the following steps:

[0034] In step S101, multiple video images of each bed in the hospital ward from different angles are collected.

[0035] Understandably, in order to identify the hand-touching behavior of medical staff in hospitals, the embodiments of this application can first set up multiple cameras in appropriate locations in the hospital ward to collect multiple video information from different angles of each bed in the ward.

[0036] Specifically, in embodiments of this application, three cameras can be installed at different locations on each hospital bed, such as the headboard, footboard, and both sides of the bed, with the distance between the cameras within 5 meters. Different angles are selected to record three videos, and each video is calculated independently. The camera arrangement is as follows: Figure 2 As shown, this ensures that hand sanitizer, hospital beds, and fixed medical equipment are all within the camera's field of view, providing a reliable video data source for subsequent recognition of hand touch behavior.

[0037] It should be noted that, in the specific implementation process, those skilled in the art can set up multiple cameras at appropriate intervals in different locations in the ward according to the actual situation. The specific location, interval and number of cameras are not specifically limited here.

[0038] In step S102, for each video information in the multiple video information, the tracking position and behavior of each medical staff member or patient are calculated in the first video stream and the second video stream, respectively, along with the ROI of the hand sanitizer, the hospital bed, the fixed medical equipment, and the mask coordinates of the object within the ROI.

[0039] After acquiring multiple video feeds from different angles of each hospital bed, the video data recorded by each camera can be processed separately in two video streams using a multi-threaded approach. The first video stream identifies and distinguishes between medical staff and patients, and tracks the images of medical staff. The second video stream, tailored to the specific hospital environment, identifies objects such as hand sanitizer, hospital beds, and fixed medical equipment. This not only accurately identifies and distinguishes medical staff, patients, and objects like hand sanitizer and hospital beds in the video images but also significantly improves detection speed, providing reliable target recognition information for subsequent hand-touching behavior determination.

[0040] Optionally, in one embodiment of this application, the tracking position and behavior of each medical staff member or patient in each of the multiple video information is calculated in the first video stream and the second video stream, respectively, along with the ROI of the hand sanitizer, the hospital bed, the fixed medical equipment, and the mask coordinates of the object within the ROI. This includes: for each camera, using a preset YOLO (You Only Look Once) object detection model based on the first video stream to distinguish between medical staff and patients; using a preset OpenPose framework to extract the coordinates of human skeleton points for medical staff; assigning a unique identity to each medical staff member entering the ward, and unifying the identity of the same person in each video information, binding the human skeleton point coordinates with the unique identity to perform single-person position tracking in the camera's field of view, and recording the behavior of each unique identity.

[0041] Specifically, in the embodiments of this application, each camera installed in the ward can perform calculations on the captured video data in two separate video streams using a multi-threaded approach.

[0042] The first video stream can utilize a YOLO object detection model trained to distinguish between medical staff and patients. It should be noted that those skilled in the art can, depending on the specific requirements such as real-time performance and accuracy, also employ YOLOv5 or other YOLO series detection algorithms, or use algorithms such as Faster-RCNN for object detection; no specific limitations are imposed here.

[0043] After distinguishing between medical staff and patients in video recordings, embodiments of this application can extract information such as the center point coordinates and target bounding box coordinates of the medical staff. Using the OpenPose framework, 25 human skeleton coordinates are extracted for each medical staff member entering the ward, and each staff member is assigned a unique ID (Identity Document). Simultaneously, the identity of the same person from multiple cameras is unified, and the human skeleton coordinates are bound to the identity. Individual position tracking is performed on this person within the camera's field of view, and the behavior of each identity is recorded.

[0044] Understandably, by calling the YOLO object detection model to distinguish between medical staff and patients for each camera, and then using the OpenPose framework to extract the coordinates of the human skeleton points of the medical staff, single-person position tracking is performed in the camera's field of view, and the behavior of each medical staff member is recorded. This not only allows for accurate observation and tracking of the behavior of each medical staff member entering the ward through images, but also provides reliable data support for subsequent judgment of the medical staff's hand touch behavior.

[0045] Optionally, in one embodiment of this application, the tracking position and behavior of each medical staff member or patient in each of the multiple video information are calculated in the first video stream and the second video stream, respectively, along with the ROI of the hand sanitizer, the hospital bed, and the fixed medical equipment, and the mask coordinates of the object within the ROI. This includes: for each camera, based on the second video stream, identifying the hand sanitizer, the hospital bed, and the fixed medical equipment using a preset Detectron2 object detection model; performing instance segmentation on the hand sanitizer, the hospital bed, and the fixed medical equipment, and marking the ROI of the hand sanitizer, the hospital bed, and the fixed medical equipment, and the mask coordinates of the object within the ROI.

[0046] It should be noted that while the object detection model is used to distinguish between medical staff and patients in the first video stream, object detection can also be performed on specific hospital environments through the second video stream.

[0047] Specifically, embodiments of this application can use the Detectron2 object detection model to train and identify three types of objects: hand sanitizer, hospital bed, and fixed medical equipment. The model then performs instance segmentation on these three types of objects, marking the Regions of Interest (ROIs) for each object and the mask coordinates of the objects within those ROIs. Figure 3 As shown, by utilizing relevant object detection models and instance segmentation algorithms, the system can identify objects such as medical staff, hand sanitizers, hospital beds, and fixed medical equipment while ensuring the accuracy and real-time performance of the detection. This provides comprehensive and reliable data information for determining the hand-touching behavior of medical staff.

[0048] In step S103, the hand-touching behavior of each medical staff member or patient in the ward is identified based on the tracking location and behavior of each medical staff member or patient, the ROI of the hand sanitizer, the hospital bed, the fixed medical equipment, and the Mask coordinates of the objects within the ROI.

[0049] By identifying and distinguishing medical staff and patients through each camera, and after identifying objects such as hand sanitizer, hospital beds, and fixed medical equipment in a specific hospital environment, the embodiments of this application determine the hand-touching behavior of each medical staff member in the ward based on the tracking location of each medical staff member and the relevant information of hand sanitizer, hospital beds, and fixed medical equipment. This not only enables accurate observation, tracking, and evaluation of the hand-touching behavior of medical staff through images, effectively reducing hospital infections, but also allows for effective source tracing in the event of events such as the COVID-19 pandemic.

[0050] Optionally, in one embodiment of this application, the hand-touching behavior of each medical staff member or patient in the ward is identified based on the tracking location and behavior of each medical staff member or patient and the ROI of hand sanitizer, hospital bed, fixed medical equipment, and the mask coordinates of the object within the ROI. This includes: determining whether the wrist point coordinates in each frame of the image coincide with the mask coordinates of the object within the ROI; if it coincides with the hand sanitizer, then a hand washing / hand sanitizing action is marked; if it coincides with the hospital bed, then a medical staff-patient contact action is marked; if it coincides with the fixed medical equipment, then a device contact action is marked.

[0051] Specifically, in the embodiments of this application, each camera can calculate whether the wrist point coordinates in each frame of the image overlap with the mask coordinates of the three types of objects (hand sanitizer, hospital bed, and fixed medical equipment) within the ROI of that object in the second video stream. If it overlaps with the hand sanitizer, the medical staff's posture is marked as a hand washing / hand sanitizing action, and 1 vote is given for hand washing / hand sanitizing; if it overlaps with the hospital bed, the medical staff's posture is marked as a doctor-patient contact action, and 1 vote is given for doctor-patient contact; if it overlaps with the fixed medical equipment, the medical staff's posture is marked as an equipment contact action, and 1 vote is given for equipment contact. In this way, by combining the camera with posture recognition and other models, the voting method is used to estimate the hand contact behavior of medical staff, thereby improving the accuracy of behavior recognition and judgment.

[0052] Optionally, in one embodiment of this application, the hand-touching behavior of each medical staff member or patient in the ward is identified based on the tracking location and behavior of each medical staff member or patient and the ROI of hand sanitizer, hospital bed, fixed medical equipment and the Mask coordinates of the object within the ROI. The method further includes: counting the cumulative number of times the hand washing / hand sanitizing actions, medical staff-patient contact actions and equipment contact actions are marked; when the cumulative number of marks is greater than a preset threshold, it is recorded in the corresponding unique identifier.

[0053] It should be noted that each camera installed in the ward can perform the aforementioned processing on the recorded video, and then vote on hand-touching behavior based on the postures and movements of medical staff, combined with the relevant coordinates of objects such as hand sanitizer, hospital beds, and fixed medical equipment. Simultaneously, using multiple cameras installed at various angles within the ward, behavior estimation is performed on the existing scene through a voting method.

[0054] As one possible approach, embodiments of this application can set appropriate thresholds based on the number of cameras in the ward. For example, when more than half of the cameras vote for medical staff contact, the hand-touching behavior is marked as a medical staff contact action. This allows for voting on the hand-touching behaviors of medical staff, followed by subsequent data statistics operations. Figure 4 As shown.

[0055] For example, when three cameras are installed in a ward, if two or three cameras in the same frame identify a medical staff member's action as handwashing / hand sanitizing, then that action is marked as handwashing / hand sanitizing and recorded in the staff member's ID. Furthermore, after the medical staff member leaves the camera's range, their ID is statistically analyzed to summarize their hand-touching behavior in the ward area. This not only enables real-time and accurate observation, tracking, and evaluation of medical staff's hand-touching behavior within the hospital, ensuring the accuracy of behavioral judgments, but also provides valuable data for infection control.

[0056] The method for identifying hand-touching behavior of medical staff in hospitals, as proposed in the embodiments of this application, involves collecting multiple video images from different angles of each bed in a hospital ward. For each video image, the tracking position and behavior of each medical staff member or patient are calculated in a first video stream and a second video stream, along with the ROI (Region of Interest) of hand sanitizer, bed, and fixed medical equipment, and the mask coordinates of the objects within that ROI. Based on the tracking position and behavior of each medical staff member or patient, and the ROI of hand sanitizer, bed, and fixed medical equipment, and the mask coordinates of the objects within that ROI, the hand-touching behavior of each medical staff member or patient in the ward is identified. This allows for accurate observation, tracking, and evaluation of hand-touching behavior of medical staff through images, effectively reducing hospital-acquired infections and enabling effective source tracing in the event of a viral outbreak.

[0057] Next, with reference to the accompanying drawings, an apparatus for recognizing the hand-touching behavior of medical staff in a hospital, according to an embodiment of this application, is described.

[0058] Figure 5 This is a block diagram of a device for recognizing the hand-touching behavior of medical staff in a hospital, according to an embodiment of this application.

[0059] like Figure 5As shown, the device 10 for recognizing the hand-touching behavior of medical staff in a hospital includes: a data acquisition module 100, a calculation module 200, and a recognition module 300.

[0060] The acquisition module 100 is used to acquire multiple video messages from different angles of each bed in a hospital ward.

[0061] The calculation module 200 is used to calculate the tracking position and behavior of each medical staff member or patient in the first video stream and the second video stream, and the ROI of the hand sanitizer, hospital bed, fixed medical equipment, and the mask coordinates of the object within the ROI.

[0062] The identification module 300 is used to identify the hand touch behavior of each medical staff member or patient in the ward based on the tracking location and behavior of each medical staff member or patient, the ROI of hand sanitizer, hospital bed, fixed medical equipment, and the mask coordinates of the objects within the ROI.

[0063] Optionally, in one embodiment of this application, the calculation module 200 includes: a differentiation unit, an extraction unit, and a tracking unit.

[0064] The differentiation unit is used to distinguish between medical staff and patients for each camera by calling a preset YOLO object detection model based on the first video stream.

[0065] The extraction unit is used to call the preset OpenPose framework to extract the coordinates of human skeletal points from medical staff.

[0066] The tracking unit is used to assign a unique identifier to each medical staff member entering the ward and to unify the identifier of the same person in each video message. It binds the coordinates of human skeleton points to the unique identifier to track the position of a single person in the camera's field of view and record the behavior of each unique identifier.

[0067] Optionally, in one embodiment of this application, the calculation module 200 further includes a detection unit and a segmentation unit.

[0068] The detection unit is used to identify hand sanitizer, hospital bed, and fixed medical equipment for each camera based on the second video stream using a preset Detectron2 target detection model.

[0069] The segmentation unit is used to segment hand sanitizer, hospital bed, and fixed medical equipment into instances, marking the ROI of hand sanitizer, hospital bed, and fixed medical equipment, as well as the mask coordinates of the objects within the ROI.

[0070] Optionally, in one embodiment of this application, the identification module 300 includes: a judgment unit, a first marking unit, a second marking unit, and a third marking unit.

[0071] The judgment unit is used to determine whether the wrist point coordinates in each frame of the image coincide with the mask coordinates of the object within the ROI.

[0072] The first marking unit is used to mark the handwashing / hand sanitizing action when it overlaps with hand sanitizer.

[0073] The second marking unit is used to mark doctor-patient contact behaviors when it overlaps with the hospital bed.

[0074] The third marking unit is used to mark the contact action as a device when it overlaps with a fixed medical device.

[0075] Optionally, in one embodiment of this application, the identification module 300 further includes a statistics unit and a recording unit.

[0076] The statistics unit is used to count the cumulative number of times handwashing / hand disinfection actions, medical staff-patient contact actions, and equipment contact actions are marked.

[0077] The recording unit is used to record the cumulative number of times the markers are set to a preset threshold into the corresponding unique identifier.

[0078] It should be noted that the foregoing explanation of the method embodiment for recognizing the hand-touching behavior of medical staff in hospitals also applies to the device for recognizing the hand-touching behavior of medical staff in hospitals in this embodiment, and will not be repeated here.

[0079] The device for identifying hand-touching behavior of medical staff in hospitals, as proposed in the embodiments of this application, collects multiple video information from different angles of each bed in a hospital ward. For each video information, it calculates the tracking position and behavior of each medical staff member or patient in the first and second video streams, along with the ROI (Region of Interest) of hand sanitizer, bed, and fixed medical equipment, and the mask coordinates of the objects within the ROI. Based on the tracking position and behavior of each medical staff member or patient, and the ROI of hand sanitizer, bed, and fixed medical equipment, and the mask coordinates of the objects within the ROI, it identifies the hand-touching behavior of each medical staff member or patient in the ward. This allows for accurate observation, tracking, and evaluation of hand-touching behavior through images, effectively reducing hospital-acquired infections and enabling effective source tracing in the event of a viral outbreak.

[0080] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0081] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0082] When the processor 602 executes the program, it implements the method for recognizing the hand-touching behavior of medical staff in the hospital provided in the above embodiments.

[0083] Furthermore, electronic devices also include:

[0084] Communication interface 603 is used for communication between memory 601 and processor 602.

[0085] The memory 601 is used to store computer programs that can run on the processor 602.

[0086] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0087] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0088] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0089] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0090] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for recognizing hand-touching behavior of medical staff in a hospital.

[0091] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0092] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0093] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0094] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0095] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0096] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0098] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for identifying hand-touching behavior of medical staff in a hospital, characterized in that, Includes the following steps: Collect multiple video feeds from different angles of each bed in a hospital ward; For each of the multiple video information sets, the tracking position and behavior of each medical staff member or patient are calculated in the first and second video streams, along with the ROI (Region of Interest) of the hand sanitizer, hospital bed, and fixed medical equipment, and the mask coordinates of the object within the ROI. Specifically, the video data recorded by each camera is calculated separately in the two video streams using a multi-threaded approach. Based on the tracking location and behavior of each medical staff member or patient, and the ROI of the hand sanitizer, the hospital bed, the fixed medical equipment, and the mask coordinates of the objects within the ROI, the hand touch behavior of each medical staff member or patient in the ward is identified. This includes: determining whether the wrist point coordinates in each frame of the image coincide with the mask coordinates of the objects within the ROI; if it coincides with the hand sanitizer, a hand washing / hand sanitizing action is marked; if it coincides with the hospital bed, a doctor-patient contact action is marked; if it coincides with the fixed medical equipment, an equipment contact action is marked; further, it includes: counting the cumulative number of markings for the hand washing / hand sanitizing action, the doctor-patient contact action, and the equipment contact action; when the cumulative number of markings exceeds a preset threshold, it is recorded in the corresponding unique identifier.

2. The method according to claim 1, characterized in that, The step of calculating the tracking position and behavior of each medical worker or patient in the first and second video streams, and the ROI (Region of Interest) of the hand sanitizer, hospital bed, and fixed medical equipment, as well as the mask coordinates of the object within the ROI, includes: For each camera, a preset YOLO object detection model is invoked based on the first video stream to distinguish between the medical staff and the patients; The medical staff used the preset OpenPose framework to extract the coordinates of human skeletal points; Each medical staff member entering the ward is assigned a unique identity, and the identity of the same person in each video message is unified. The coordinates of the human skeleton points are bound to the unique identity to track the location of a single person in the camera's field of view and record the behavior of each unique identity.

3. The method according to claim 2, characterized in that, The step of calculating the tracking position and behavior of each medical worker or patient in the first and second video streams, and the ROI (Region of Interest) of the hand sanitizer, hospital bed, and fixed medical equipment, as well as the mask coordinates of the object within the ROI, includes: For each camera, based on the second video stream, the hand sanitizer, the hospital bed, and the fixed medical equipment are identified using a preset Detectron2 target detection model. The hand sanitizer, the hospital bed, and the fixed medical equipment are segmented into instances, and the ROIs of the hand sanitizer, the hospital bed, and the fixed medical equipment, as well as the mask coordinates of the objects within the ROIs, are marked.

4. A device for recognizing the hand-touching behavior of medical staff in a hospital, characterized in that, include: The acquisition module is used to collect multiple video images from different angles of each bed in a hospital ward. The calculation module is used to calculate the tracking position and behavior of each medical staff member or patient in the first and second video streams, and the ROI (Region of Interest) of the hand sanitizer, hospital bed, and fixed medical equipment, as well as the mask coordinates of the object within the ROI. The calculation is performed separately in the two video streams using a multi-threaded approach for the video data recorded by each camera. The identification module is used to identify the hand-touching behavior of each medical staff member or patient in the ward based on the tracking location and behavior of each medical staff member or patient, the ROI of the hand sanitizer, the hospital bed, the fixed medical equipment, and the Mask coordinates of the objects within the ROI. The identification module includes: The judgment unit is used to determine whether the wrist point coordinates in each frame of the image coincide with the Mask coordinates of the object within the ROI; The first marking unit is used to mark a handwashing / hand sanitizing action when it overlaps with the hand sanitizer; The second marking unit is used to mark doctor-patient contact actions when it overlaps with the hospital bed; The third marking unit is used to mark the device contact action when it overlaps with the fixed medical device; The statistics unit is used to count the cumulative number of times the handwashing / hand disinfection action, the medical staff-patient contact behavior action, and the equipment contact action are marked. The recording unit is used to record the cumulative number of times the markers are added to the corresponding unique identifier when the cumulative number of times the markers are added exceeds a preset threshold.

5. The apparatus according to claim 4, characterized in that, The computing module includes: The differentiation unit is used to distinguish between the medical staff and the patients based on the first video stream by calling a preset YOLO object detection model for each camera. The extraction unit is used to call the preset OpenPose framework to extract the coordinates of human skeletal points from the medical staff. The tracking unit is used to assign a unique identifier to each medical staff member entering the ward and to unify the identifier of the same person in each video message. The coordinates of the human skeleton points are bound to the unique identifier to perform single-person position tracking in the camera's field of view and to record the behavior of each unique identifier.

6. The apparatus according to claim 5, characterized in that, The computing module also includes: The detection unit is used to identify the hand sanitizer, the hospital bed, and the fixed medical equipment for each camera based on the second video stream using a preset Detectron2 target detection model. The segmentation unit is used to segment the hand sanitizer, the hospital bed, and the fixed medical equipment into instances, and to mark the ROI of the hand sanitizer, the hospital bed, and the fixed medical equipment, as well as the mask coordinates of the objects within the ROI.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method for recognizing hand-touching behavior of medical staff in a hospital as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method for recognizing hand-touching behavior of medical staff in a hospital as described in any one of claims 1-3.

Images