An artificial intelligence-based automatic monitoring method and system for hand hygiene

Abstract

This invention discloses an automatic hand hygiene monitoring method and system based on artificial intelligence. The method includes: acquiring video data of the handwashing posture in three directions (left, top, and right) at a frame rate of M frames per second; identifying several bone nodes of each hand, extracting the three-dimensional coordinates of the bone nodes from every N frames of video as input to a sub-discrimination space model, and realizing the judgment of handwashing posture and handwashing steps M / N times per second, where M≥N; using a smoothing algorithm to extract video segments of each handwashing step from the video data; using a repetitive action calculation module to calculate the number of handwashing rubs in the video segment; and jointly calculating a score for a certain handwashing step based on two index scores: the number of handwashing rubs and the average probability value of the handwashing step judgment result in the corresponding video segment, to monitor the hand hygiene of the handwashing user. This invention also discloses an automatic hand hygiene monitoring system based on artificial intelligence. This invention achieves hand hygiene monitoring while greatly reducing computational power.

Description

Technical Field

[0001] This invention relates to the field of medical and health technology, and in particular to an automatic hand hygiene scoring method and system based on artificial intelligence. Background Technology

[0002] Hand hygiene is the most effective measure for preventing hospital-acquired infections, especially in ICUs where the infection rate is higher than in general wards due to the complex infection pathways. Healthcare workers, including nursing assistants, have the highest chance of contact with critically ill patients during diagnosis and treatment. However, the current hand hygiene compliance rate among healthcare workers is not ideal, at only 33.17%. If healthcare workers do not wash their hands after contact with patients or after contamination, the total bacterial count exceeds the standard in 100% of cases. One-third of hospital-acquired infections can be effectively controlled through strict hand hygiene. How to assess whether doctors are implementing strict hand hygiene is a challenge facing hospitals in preventing hospital-acquired infection events.

[0003] Existing technologies are mainly divided into two types: one based on wearable devices and the other based on video.

[0004] Wearable device-based methods typically require users to wear wearable devices to wash their hands. Data is collected via gyroscopes and accelerometers in the armband, and then machine learning methods are used to analyze the data to ultimately identify and classify handwashing postures. However, this method is inherently inconvenient for users, requiring them to roll up their sleeves and wear the armband, significantly increasing the burden of handwashing. Furthermore, it is unusable in winter, hindering its large-scale application.

[0005] Current video-based handwashing methods directly input video data and use deep learning to identify handwashing postures. This method struggles to achieve real-time monitoring of handwashing postures and suffers from low accuracy. The biggest problem lies in its inability to be practically applied; the handwashing recognition performance is poor among participants not on the training set, with an accuracy rate below 60%. Furthermore, this method requires significant computing power; because it directly uses video as the judgment criterion, the sheer volume of data generated by the video stream necessitates a system operating on high-performance computing machines.

[0006] Furthermore, in real-world medical settings, individuals often fail to complete the seven-step handwashing technique correctly and thoroughly due to improper handwashing techniques or time constraints. Hospital infection control departments can only conduct random checks to verify whether doctors are strictly adhering to the proper handwashing method. Therefore, using AI to automatically score doctors' handwashing actions becomes crucial in preventing hospital-acquired infections. Current methods for accurate and real-time handwashing technique recognition in real-world medical scenarios are too dependent on the environment; changing the handwashing environment often renders the model ineffective, failing to generate a reliable handwashing score. Summary of the Invention

[0007] This invention provides an automatic hand hygiene monitoring method and system based on artificial intelligence, which realizes hand hygiene monitoring and avoids using the entire video as input to the classifier, greatly reducing the number of input parameters.

[0008] The present invention adopts the following technical solution:

[0009] On one hand, the present invention provides an automatic hand hygiene monitoring method based on artificial intelligence, comprising:

[0010] Video data of the handwashing posture is acquired from the left, top, and right directions of the handwashing station, with a video frame rate of M frames per second;

[0011] Several key bone nodes of each hand are identified from the video data, and the three-dimensional coordinates of each key bone node are obtained.

[0012] The three-dimensional coordinates of the key bone nodes of each N frames of video are extracted and used as input to the sub-discrimination space model to realize the judgment of the handwashing posture of the handwashing person in the handwashing method M / N times per second, and then to determine the handwashing step to which the handwashing posture belongs, where M≥N;

[0013] A smoothing algorithm is used to extract the sequence of judgment results for consecutive identical handwashing steps in each handwashing step from the video data. The judgment result for each handwashing posture represents N frames of video, thereby extracting the video segment corresponding to each handwashing step.

[0014] The repetitive motion calculation module is used to calculate the number of times the hands are rubbed in the video segment for each handwashing step;

[0015] The score for a particular handwashing step is calculated by combining the scores of two indicators: the number of times the hands are rubbed and the average probability value of the handwashing step judgment results in the corresponding video segment. This allows for the monitoring of the hand hygiene performance of the handwashing user.

[0016] On the other hand, the present invention provides an artificial intelligence-based automatic hand hygiene monitoring system, comprising:

[0017] The video acquisition module is used to acquire video data of the handwashing person's posture in the left, top, and right directions of the handwashing position, with a video frame rate of M frames per second;

[0018] The bone node extraction module identifies several key bone nodes of each hand from the video data and obtains the three-dimensional coordinates of each key bone node.

[0019] The handwashing posture judgment module is used to extract the three-dimensional coordinates of the key bone nodes of each N frames of video, and use them as input to the sub-discrimination space model to realize the judgment of the handwashing posture of the handwashing person in the handwashing method M / N times per second, and then determine the handwashing step to which the handwashing posture belongs, where M≥N;

[0020] The confidence level judgment module for handwashing posture is used to extract the judgment result sequence of consecutive identical handwashing steps in each handwashing step from the video data using a smoothing algorithm. The judgment result of each handwashing posture represents N frames of video, thereby extracting the video segment corresponding to each handwashing step and calculating the average probability value of the handwashing step judgment result of the corresponding video segment.

[0021] The rubbing count calculation module is used to calculate the number of times the hands are rubbed in the video segment of each handwashing step using the repetitive action calculation module;

[0022] The scoring module combines the scores of the number of rubbing strokes and the average probability value of the handwashing step judgment results in the corresponding video segment to calculate the score of the handwashing person for a certain handwashing step, thereby monitoring the hand hygiene performance of the handwashing person.

[0023] Beneficial effects

[0024] The method and system of this invention utilize deep learning skeletal extraction technology to extract the hand bones of the handwashing participant from three directions, extracting the coordinate information of several key skeletal points from each hand. This avoids using the entire video as input to the classifier, greatly reducing the number of input parameters. Simultaneously, the powerful analytical capabilities of this invention can accurately identify the doctor's handwashing posture and assign a score to each posture, which is of great significance for promoting infection control in hospitals.

[0025] The method and system of this invention are highly scalable and can significantly improve the quality of handwashing in hospitals and communities. Especially for hospitals, the system can track whether each doctor follows the "seven-step handwashing technique" each time they wash their hands, as well as the quality of each step, and provide real-time feedback on the monitoring of each handwashing session. Through the internet, doctors can receive timely reminders and feedback; hospitals can accurately monitor the handwashing of each doctor and manage the process comprehensively. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the extraction of key skeletal points using the skeletal extraction technology provided in various embodiments of the present invention;

[0027] Figure 2 This is a flowchart illustrating an automatic hand hygiene monitoring method based on artificial intelligence, provided in Embodiment 1 of the present invention.

[0028] Figure 3 This is a flowchart illustrating a specific method for automatic hand hygiene monitoring based on artificial intelligence, as provided in Embodiment 2 of the present invention.

[0029] Figure 4 This is a schematic diagram of a specific artificial intelligence-based automatic hand hygiene monitoring system provided in Embodiment 3 of the present invention. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0031] The World Health Organization recommends the "six-step handwashing method," which hospital departments often add a "wrist washing" step to become the standard handwashing method, known as the "seven-step handwashing method." The seven-step handwashing method is a method used by medical personnel before performing procedures, requiring them to use seven different handwashing techniques. This invention provides an AI-based automatic hand hygiene monitoring method that judges the correctness of the handwashing posture primarily based on the seven techniques of the seven-step handwashing method.

[0032] See Figure 2 This is a flowchart illustrating an automated hand hygiene monitoring method based on artificial intelligence, provided in Embodiment 1 of the present invention. The method includes:

[0033] Step S101: Acquire video data of the handwashing person's posture in the left, top, and right directions of the handwashing position, with a video frame rate of M frames per second.

[0034] In this step, cameras are specifically deployed in the left, top, and right directions of the washbasin to provide video data of the handwashing posture of the person washing their hands.

[0035] Step S102: Identify several key bone nodes of each hand from the video data and obtain the three-dimensional coordinates of each key bone node.

[0036] In this step, artificial intelligence hand skeleton extraction technology is used to extract the skeletal points of the hand in each frame of the video. The left camera extracts the skeletal points of the left hand, the right camera extracts the skeletal points of the right hand, and the top camera extracts the skeletal points of both hands, thus realizing the modeling of each hand.

[0037] Step S103: Create a filtering algorithm that filters out actions involving posture transitions and / or rinsing that are not part of the seven-step handwashing method based on the spacing between the key bone nodes. The filtering algorithm is as follows:

[0038]

[0039] Where S represents the square of the distance between two skeletal points, n represents the number of key skeletal points in each hand's fingers, and X... k The X-axis coordinate of the Kth skeletal node, Y-axis coordinate of the Kth skeletal node. k The Y-axis coordinate of the Kth vertices, Z k Let Z represent the Z-axis coordinate of the Kth bone node. Then, this formula calculates the square of the distance from the first point of the hand bone node to the nth point.

[0040] Preferably, the filtration threshold S for non-handwashing individuals is set to 0.391–0.693.

[0041] In this step, hands that enter the frame due to passing by or other reasons can affect the recognition of hands of genuine handwashers in the video data, thus requiring filtering out non-handwashing individuals. Non-handwashing individuals are farther from the camera than handwashing individuals, therefore their hands appear smaller relative to the camera. This algorithm calculates the distance between the identified hand skeletal points and filters out hands that are too close, thus filtering out the hands of non-handwashing individuals.

[0042] Step S104: Extract the three-dimensional coordinates of the key bone nodes of each N frames of video, and use them as input to the sub-discrimination space model to realize the judgment of the handwashing posture of the handwashing person in the handwashing method M / N times per second, and then determine the handwashing step to which the handwashing posture belongs, where M≥N.

[0043] This step extracts key skeletal nodes of the hand in the video, avoiding the impact of different shooting scenes on recognition, and thus achieving noise reduction.

[0044] The handwashing posture determination involves identifying whether the currently identified handwashing posture is a handwashing posture from the handwashing posture training set. The handwashing step determination involves identifying which step in the handwashing technique the identified handwashing posture belongs to.

[0045] Step S105: Use a smoothing algorithm to extract the sequence of judgment results for consecutive identical handwashing steps from the video data. Each handwashing posture judgment result represents N frames of video, thereby extracting the video segment corresponding to each handwashing step.

[0046] In this step, specifically, in the step of extracting the sequence of consecutive identical judgment results corresponding to each handwashing posture from the video data using a smoothing algorithm based on the judgment results of the handwashing posture, if a difference occurs once in the consecutive sequence, it is ignored.

[0047] Step S106: Use the repetitive motion calculation module to calculate the number of times the hands are rubbed in the video segment of each handwashing step.

[0048] Step S107: Based on the scores of the number of times the hands are rubbed and the average probability value of the handwashing steps in the corresponding video segment, the score of the handwasher for a certain handwashing step is calculated in combination, thereby monitoring the hand hygiene performance of the handwasher.

[0049] In this step, the score for a particular handwashing step is determined by summing the scores of two indicators: the number of times the hands are rubbed and the average probability value of the handwashing step judgment results in the corresponding video segment.

[0050] In summary, this invention utilizes deep learning-based skeletal extraction technology to extract the hand bones of the person washing their hands from three directions, extracting the coordinate information of several key skeletal points from each hand. This avoids using the entire video as input to the classifier, significantly reducing the number of input parameters and alleviating the computational burden on the video stream. The handwashing posture recognition model is environmentally independent and boasts a recognition rate of up to 99%. This method can statistically analyze whether each doctor follows the "seven-step handwashing technique" for each handwashing session, as well as the quality of each step, and provide real-time feedback on the monitoring status of each handwashing session. Through the internet, doctors can receive timely reminders and feedback; hospitals can accurately monitor the handwashing status of each doctor and manage the process comprehensively.

[0051] To ensure accuracy, the seven handwashing techniques can be further subdivided into 19 or more techniques.

[0052] The seven-step handwashing method consists of: washing palms, washing the backs of hands and between fingers, washing the palms and between fingers, washing the backs of fingers, washing thumbs, washing fingertips, and washing wrists. Steps two, four, five, six, and seven require alternating between both hands. Therefore, identifying the seven-step handwashing method actually involves identifying 12 different steps: washing palms, washing the backs of the left hand and between fingers, washing the backs of the right hand and between fingers, washing the palms and between fingers, washing the backs of the left fingers, washing the backs of the right fingers, washing the left thumb, washing the right thumb, washing the left fingertips, washing the right fingertips, washing the left wrist and forearm, and washing the right wrist and forearm. However, doctors typically don't use the same method for every step of the handwashing process. Therefore, even if the person using different approved handwashing postures or different hand positions, the model must be able to identify which handwashing posture it belongs to. Because there are many handwashing postures, we cannot exhaust all of them. Therefore, depending on the handwashing method or the different hand placement angles, each of the 12 handwashing steps has 1 to 2 different handwashing postures, resulting in a total of 19 different handwashing postures. This requires the discrimination model to have the ability to accurately classify these 19 postures. The 19 handwashing postures included in the 12 handwashing steps are detailed in Table 1 below:

[0053]

[0054] See Figure 3 This is a flowchart illustrating a specific method for automatic hand hygiene monitoring based on artificial intelligence, provided in Embodiment 2 of the present invention. The method includes:

[0055] Step S201: Deploy cameras in the left, top, and right directions of the handwashing area to acquire video data of the handwashing person's posture at a frame rate of 30 frames per second.

[0056] Step S202: Use skeletal extraction technology to extract the position of the hand in each frame of the video data. Only the skeleton of one hand is extracted from each of the left and right cameras, while the skeleton of both hands is extracted from the top camera; therefore, four hands are extracted from each frame of the video.

[0057] Step S203: After locating the hand, use bone extraction technology to identify 21 key bone nodes of each hand, obtain the three-dimensional coordinates (x, y, z) of each key bone node, and realize the modeling of the hand.

[0058] Step S204: Create a filtering algorithm that filters out actions involving posture transitions and / or rinsing that are not part of the seven-step handwashing method based on the spacing between the key bone nodes; wherein the filtering algorithm is...

[0059]

[0060] Where S represents the square of the distance between two skeletal points, n represents the number of key skeletal points in each hand's fingers (20, i.e., n = 20), X k The X-axis coordinate of the Kth skeletal node, Y-axis coordinate of the Kth skeletal node. k The Y-axis coordinate of the Kth vertices, Z k Let Z represent the Z-axis coordinate of the Kth bone node. Then, this formula calculates the square of the distance from the first point of the hand bone nodes to the 20th point.

[0061] When a person washing their hands passes by a sink without washing their hands, the hands of the non-hand-washing person are easily captured by cameras deployed on both sides of the sink, thus affecting the judgment of the handwashing posture. To avoid this impact, a filtering algorithm was created to filter out the hands of non-hand-washing people, with a filtering threshold S of 0.5 set for non-hand-washing people.

[0062] Step S205: Extract the three-dimensional coordinates of the key bone nodes of every 10 frames of video, and use them as input to the sub-discrimination space model to realize the judgment of the handwashing posture of the handwashing person in the handwashing method 3 times per second, and then determine the handwashing step to which the handwashing posture belongs.

[0063] In this step, every 10 frames of video are used as input, so each input has 10 frames of video * 4 hands * 21 key bone nodes * 3 judgments = 2520 variables.

[0064] This can also be understood as follows: the handwashing video of the person who needs to be scored is input into the pre-trained handwashing posture discrimination model at 10 frames per second. Since the video is 30 frames per second, the model will make 3 judgments on the handwashing steps per second.

[0065] In the seven-step handwashing method, for precise classification, it can be further subdivided into 19 or more postures based on different handwashing methods or hand placement angles. This embodiment preferably uses 19 handwashing postures, as shown in Table 1, which serves as the training set for the handwashing posture discrimination model. Using 2520 variables generated from every 10 frames of video as input, the model determines which of the 19 handwashing postures the person washing their hands uses in those 10 frames. After training, the model's accuracy is verified using 10-fold cross-validation. The model achieves an accuracy of 97.5% in discriminating the 19 handwashing postures. Then, the 19 discriminations are converted into 12 handwashing steps. Since both the first and second handwashing methods belong to the first step, when the model classifies either the first or second handwashing method, it is considered the same judgment, and the model's accuracy reaches 99%.

[0066] Step S206: A smoothing algorithm is used to extract the sequence of judgment results for consecutive identical handwashing steps from the video data. The judgment result for each handwashing posture represents 10 frames of video, thereby extracting the video segment corresponding to each handwashing step. For example, the model output is: 1, 3, 4, 5, 1, 2, 4, 1, 1, 1, 1, 1, 1, 3, 1, 1, 4, 1, 1, 1, 1, 5, 6, 7, 4, where the model output result is 1, representing the first step in 12 handwashing steps. The smoothing algorithm extracts consecutive identical judgment results, ignoring any difference in the sequence. The extracted handwashing video segment for the first step is: 1, 1, 1, 1, 1, 3, 1, 1, 4, 1, 1, 1, 1.

[0067] Step S207: Use the repetitive motion calculation module to calculate the number of times the hands are rubbed in the video segment of each handwashing step.

[0068] Step S208: Based on the sum of the scores of the two indicators, namely the number of times the hands are rubbed and the average probability value of the handwashing step judgment result in the corresponding video segment, the score of the handwasher for a certain handwashing step is calculated, thereby monitoring the hand hygiene performance of the handwasher.

[0069] Specifically, the number of times hands are rubbed is as follows: if the number of rubbing exceeds 6 (the infection control requirements stipulate that each posture should last no less than 3 seconds, and on average, rubbing twice per second, then the number of rubbing hands should not be less than 6), then this indicator will receive a full score of 50 points; if it does not exceed 6 times, then the score will be 50 * (number of times / 6).

[0070] The average of the model's output probability values ​​is calculated as follows: For example, if the first step of the handwashing video segment is: 1, 1, 1, 1, 1, 1, 3, 1, 1, 4, 1, 1, 1, 1, the probability values ​​where the model outputs 1 in this segment are summed and then averaged. The score for this indicator is then 50 * the average probability value.

[0071] The score for this handwashing step is the sum of the scores from the two indicators.

[0072] In summary, this embodiment uses deep learning skeletal extraction technology to extract the hand bones of the handwasher from three directions, extracting the coordinate information of several key skeletal points from each hand. This avoids using the entire video as input to the classifier, greatly reducing the number of input parameters and alleviating the computational burden on the video stream. The hand hygiene of the handwasher is monitored by evaluating two metrics: the number of rubbing motions and the average probability value of the model output for each handwashing posture throughout the entire video. This method can statistically analyze whether each doctor follows the "seven-step handwashing technique" for each handwashing session, as well as the quality of each step, and provide real-time feedback on the monitoring status of each handwashing session. Through the internet, doctors can receive timely reminders and feedback; hospitals can accurately monitor the handwashing status of each doctor and manage the process comprehensively.

[0073] See Figure 4 This is a schematic diagram of a specific artificial intelligence-based automatic hand hygiene monitoring system provided in Embodiment 3 of the present invention. The system includes a video acquisition module 301, a bone node extraction module 302, a handwashing posture judgment module 303, a handwashing posture confidence judgment module 304, a rubbing count calculation module 305, and a scoring module 306.

[0074] The video acquisition module is used to acquire video data of the handwashing posture, with a video capture rate of M frames per second.

[0075] The bone node extraction module identifies several key bone nodes of each hand from the video data and obtains the three-dimensional coordinates of each key bone node.

[0076] The handwashing posture judgment module is used to extract the three-dimensional coordinates of the key bone nodes of each N frames of video, and use them as input to the sub-discrimination space model to realize the judgment of the handwashing posture of the handwashing person in the handwashing method M / N times per second, where M≥N.

[0077] The confidence level judgment module for handwashing posture is used to extract a sequence of consecutive identical judgment results corresponding to each handwashing posture from the video data using a smoothing algorithm. Each judgment result represents N frames of video, thereby extracting the entire video segment corresponding to each handwashing posture and calculating the average value of the average probability value output by the judgment model for each judgment in the entire video segment.

[0078] The rubbing count calculation module is used to calculate the number of times the hands are rubbed during each handwashing posture in the entire video using the repetitive motion calculation module.

[0079] The scoring module is used to calculate the final score of the handwashing user for a certain handwashing step by combining the scores of the number of rubbings and the average of the average probability values ​​output by the judgment model for each handwashing posture in the entire video. This is used to monitor the hand hygiene of the handwashing user.

[0080] In one possible embodiment, the system further includes a filtering module 307 for creating a filtering algorithm that filters out actions involving posture transitions and / or rinsing that are not part of the seven-step handwashing technique, based on the spacing between the key skeletal nodes; wherein the filtering algorithm is...

[0081]

[0082] Where S represents the square of the distance between two skeletal points, n represents the number of key skeletal points in each hand's fingers, and X... k The X-axis coordinate of the Kth skeletal node, Y-axis coordinate of the Kth skeletal node. k The Y-axis coordinate of the Kth vertices, Z k The formula, representing the Z-axis coordinate of the Kth bone node, calculates the sum of squares of the distances from the first point to the nth point of the hand bone. To prevent non-handwashing individuals from being easily captured by cameras deployed on both sides of the sink while the handwashing person is washing their hands, thus affecting the judgment of the handwashing posture, a filtering module was created to filter out the hands of non-handwashing individuals.

[0083] In one possible embodiment, the system further includes a threshold setting module 308 for filtering non-handwashing individuals, wherein the threshold S is set to 0.391 to 0.693.

[0084] In one possible embodiment, the system further includes cameras deployed on the left, top, and right sides of the washbasin, for connection to the video acquisition module. This enables precise monitoring of the handwashing user's posture in a three-dimensional, multi-dimensional manner.

[0085] It should be noted that the AI-based automatic hand hygiene monitoring system provided in the above embodiments is only illustrated by the division of the functional modules described above when performing monitoring and scoring. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the server serving as the automatic hand hygiene monitoring system can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the automatic hand hygiene monitoring system and the automatic hand hygiene monitoring method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0086] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0087] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based automatic monitoring method of hand hygiene, characterized by, include: Video data of the handwashing posture is acquired from the left, top, and right directions of the handwashing station, with a video frame rate of M frames per second; Several key bone nodes of each hand are identified from the video data, and the three-dimensional coordinates of each key bone node are obtained. The three-dimensional coordinates of the key bone nodes in each N frames of video are extracted and used as input to the sub-discrimination space model to realize the judgment of the handwashing posture of the handwashing person in the handwashing method M / N times per second, and then to determine the handwashing step to which the handwashing posture belongs, where M≥N; A smoothing algorithm is used to extract the sequence of judgment results for consecutive identical handwashing steps in each handwashing step from the video data. The judgment result for each handwashing posture represents N frames of video, thereby extracting the video segment corresponding to each handwashing step. The repetitive motion calculation module is used to calculate the number of times the hands are rubbed in the video segment for each handwashing step; Based on the scores of two indicators—the number of times the hands are rubbed and the average probability value of the handwashing steps in the corresponding video segment—a score for a particular handwashing step is calculated to monitor the hand hygiene performance of the handwasher. Before the step of extracting the three-dimensional coordinates of the key bone nodes for each N frames of video as input to the sub-discrimination space model, the following step is also included: A filtering algorithm is created that filters out actions involving posture transitions and / or rinsing that are not part of the seven-step handwashing technique, based on the spacing between the key skeletal nodes; wherein the filtering algorithm is... Where S represents the square of the distance between two skeletal points, n represents the number of key skeletal points in each hand's fingers, and X... k The X-axis coordinate of the k-th skeletal node is represented by the Y-axis coordinate. k The Z-axis coordinate represents the Y-coordinate of the k-th skeletal node. k Let Z represent the Z-axis coordinate of the k-th bone node. Then, this formula calculates the square of the distance from the first point of the hand bone node to the n-th point. In the step of extracting a sequence of consecutive identical judgment results corresponding to each handwashing posture from the video data using a smoothing algorithm, if a difference occurs once in a consecutive sequence, it is ignored. The filtering threshold S for non-handwashing individuals was set to 0.391~0.693; M is 30, N is 10, and the number of key bone nodes is 21, located on the five fingers, palm, and wrist respectively.

2. An automatic hand hygiene monitoring system based on artificial intelligence, characterized in that include: The video acquisition module is used to acquire video data of the handwashing person's posture in the left, top, and right directions of the handwashing position, with a video frame rate of M frames per second; The bone node extraction module identifies several key bone nodes of each hand from the video data and obtains the three-dimensional coordinates of each key bone node. The handwashing posture judgment module is used to extract the three-dimensional coordinates of the key bone nodes of each N frames of video, and use them as input to the sub-discrimination space model to realize the judgment of the handwashing posture of the handwashing person in the handwashing method M / N times per second, and then determine the handwashing step to which the handwashing posture belongs, where M≥N; The confidence level judgment module for handwashing posture is used to extract the judgment result sequence of consecutive identical handwashing steps in each handwashing step from the video data using a smoothing algorithm. The judgment result of each handwashing posture represents N frames of video, thereby extracting the video segment corresponding to each handwashing step and calculating the average probability value of the handwashing step judgment result of the corresponding video segment. The rubbing count calculation module is used to calculate the number of times the hands are rubbed in the video segment of each handwashing step using the repetitive action calculation module; The scoring module is used to calculate the score of the handwashing person for a certain handwashing step by combining the scores of the number of rubbing and the average probability value of the handwashing step judgment results of the corresponding video segment, thereby monitoring the hand hygiene performance of the handwashing person; The system also includes a filtering module for creating a filtering algorithm that filters out actions involving posture transitions and / or rinsing that are not part of the seven-step handwashing method, based on the spacing between the key skeletal nodes; wherein the filtering algorithm is... Where S represents the square of the distance between two skeletal points, n represents the number of key skeletal points in each hand's fingers, and X... k The X-axis coordinate of the k-th skeletal node is represented by the Y-axis coordinate. k The Z-axis coordinate represents the Y-coordinate of the k-th skeletal node. k Let Z represent the Z-axis coordinate of the k-th bone node. Then, this formula calculates the square of the distance from the first point of the hand bone node to the n-th point. The system also includes cameras deployed on the left, top, and right sides of the washbasin, for connection to the video acquisition module; It also includes a threshold setting module for filtering non-handwashing individuals, wherein the threshold S is set to 0.391~0.693; M is 30, N is 10, and the number of key bone nodes is 21, located on the five fingers, palm, and wrist respectively.

Images