Multi-person parallel hand disinfection behavior video data processing method and system based on region instance segmentation

Abstract

The application provides a multi-person parallel hand disinfection behavior video data processing method and system based on region instance segmentation, and relates to the technical field of computer vision. The method comprises the following steps: acquiring a monitoring video stream in which multiple target objects perform hand disinfection operations concurrently, and performing frame-by-frame feature coding on the monitoring video stream to obtain an original space-time feature tensor; performing region instance segmentation processing on the original space-time feature tensor to obtain independent instance masks of each hand washing point; and performing space feature cropping and adjacent region interference suppression operations on the original space-time feature tensor by using the independent instance masks to obtain a space decoupled instance feature set which is one-to-one mapped with each hand washing point. The application realizes independent and accurate time sequence behavior tracking and compliance evaluation of each hand washing point under multi-person parallel hand disinfection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for processing video data of multi-person parallel hand disinfection behavior based on region instance segmentation. Background Technology

[0002] Automatic monitoring and compliance assessment of hand disinfection procedures are of great significance for infection control in hospitals. Currently, there are some computer vision-based hand disinfection recognition systems that can capture video streams through cameras, use facial recognition to confirm the operator's identity, and combine motion recognition algorithms to guide and evaluate hand operation steps. However, in actual deployment and application, especially in environments where multiple people perform hand disinfection procedures simultaneously, existing technologies have some limitations.

[0003] For example, at the entrance to the operating room, there are usually multiple handwashing stations arranged side by side. Medical staff need to frequently enter this area to disinfect their hands. When two medical staff members start operating at two adjacent handwashing stations almost simultaneously, the existing system may have difficulty accurately distinguishing their respective hand movements and identities. Since the camera's field of view may cover two handwashing station areas at the same time, the hand movement trajectories of the two operators are likely to intersect or overlap in the image space, causing the system to misjudge operator A's rubbing motion as part of operator B's process, or causing confusion during the identity binding process. In addition, when one operator's body temporarily obscures the hand area of ​​another operator, the motion tracking of the existing system may be interrupted, resulting in incomplete operation records. These situations may cause the compliance assessment results to be inconsistent with the actual situation, resulting in biased data reported to the hospital infection control platform and affecting the accurate judgment of medical staff's hand hygiene compliance. Summary of the Invention

[0004] This invention provides a method and system for processing video data of multi-person parallel hand disinfection behavior based on region instance segmentation, which enables independent and accurate temporal behavior tracking and compliance assessment of each handwashing station under multi-person parallel hand disinfection.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A first aspect is a method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation, the method comprising: Step 1: Obtain the monitoring video stream of multiple target objects performing hand disinfection operations concurrently, and perform frame-by-frame feature encoding on the monitoring video stream to obtain the original spatiotemporal feature tensor; Step 2: Perform region instance segmentation on the original spatiotemporal feature tensor to obtain independent instance masks for each handwashing station; use the independent instance masks to perform spatial feature clipping and adjacent region interference suppression operations on the original spatiotemporal feature tensor to obtain a spatially decoupled instance feature set that maps one-to-one with each handwashing station. Step 3: Perform temporal trajectory tracking processing on the spatial decoupled instance feature set to extract the inter-frame displacement sequence of limb key points within the mask coverage area of ​​each independent instance; based on the inter-frame displacement sequence of limb key points, perform motion continuity interpolation compensation on the trajectory break segments caused by dynamic occlusion, and reassemble the compensated discrete motion segments across frames according to the instance identifier and bind them to the identity to obtain the temporally isolated behavior sequence chain. Step 4: Perform stage state mapping processing on the time-series isolation behavior sequence chain, extract the features of each motion segment, and perform multi-dimensional feature similarity matching with the preset discrete state nodes of the standard hand disinfection process; perform state transition probability threshold determination based on multi-dimensional feature similarity, and perform weight decay filtering on non-standard deviation actions to obtain the stage compliance state vector corresponding to each handwashing area. Step 5: Perform parallel compliance decision processing on the stage compliance status vector, and perform multi-instance concurrent logic matching and verification by combining the preset process timing topology constraints and the standard operation duration threshold of each stage to obtain the staged hand disinfection behavior evaluation result data stream that strictly corresponds to the spatial location of the handwashing area.

[0006] Secondly, a multi-person parallel hand disinfection behavior video data processing system based on region instance segmentation includes: The acquisition module is used to acquire the monitoring video stream of multiple target objects performing hand disinfection operations concurrently, and to perform frame-by-frame feature encoding on the monitoring video stream to obtain the original spatiotemporal feature tensor; The segmentation module is used to perform region instance segmentation processing on the original spatiotemporal feature tensor to obtain an independent instance mask for each handwashing station; the independent instance mask is used to perform spatial feature clipping and adjacent region interference suppression operations on the original spatiotemporal feature tensor to obtain a spatially decoupled instance feature set that is mapped one-to-one with each handwashing station. The compensation module is used to perform temporal trajectory tracking processing on the spatial decoupled instance feature set, extract the inter-frame displacement sequence of limb key points in the mask coverage area of ​​each independent instance; based on the inter-frame displacement sequence of limb key points, it performs motion continuity interpolation compensation on the trajectory break segments caused by dynamic occlusion, and reassembles the compensated discrete motion segments across frames according to the instance identifier and binds them to the identity to obtain the temporally isolated behavior sequence chain. The determination module is used to perform stage state mapping processing on the time-series isolation behavior sequence chain, extract the features of each motion segment, and perform multi-dimensional feature similarity matching with the preset discrete state nodes of the standard hand disinfection process; based on the multi-dimensional feature similarity, it performs state transition probability threshold determination, and performs weight decay filtering on non-standard deviation actions to obtain the stage compliance state vector corresponding to each handwashing area. The evaluation module is used to perform parallel compliance decision processing on the stage compliance status vector. It combines the preset process timing topology constraints with the standard operation duration threshold of each stage to perform multi-instance concurrent logical matching and verification, and obtains a data stream of staged hand disinfection behavior evaluation results that strictly correspond to the spatial location of the handwashing area.

[0007] The above-described solution of the present invention has at least the following beneficial effects: By segmenting regional instances and clipping spatial features, independent operation instances of multiple handwashing stations and multiple target objects are separated, effectively suppressing motion interference from adjacent areas. Combined with inter-frame displacement sequences of limb key points and motion continuity interpolation compensation, stable identity binding and behavior reorganization across frame sequences are achieved, improving the integrity and temporal coherence of motion trajectories in complex environments. Through stage state mapping and multi-dimensional feature similarity matching, combined with state transition probability thresholds and weight attenuation filtering of non-standard deviation actions, the accuracy and anti-interference ability of compliance judgment are improved. Based on temporal topological constraints and standard operation duration thresholds, parallel compliance judgment is achieved, enabling independent and synchronous evaluation of each handwashing station for multiple handwashing areas and multiple target objects. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a method for processing video data of multi-person parallel hand disinfection behavior based on region instance segmentation, provided by an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram of a multi-person parallel hand disinfection behavior video data processing system based on region instance segmentation provided by an embodiment of the present invention. Detailed Implementation

[0010] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0011] like Figure 1 As shown, embodiments of the present invention propose a method for processing video data of multiple people's parallel hand disinfection behavior based on region instance segmentation. The method includes the following steps: Step 1: Obtain the monitoring video stream of multiple target objects performing hand disinfection operations concurrently, and perform frame-by-frame feature encoding on the monitoring video stream to obtain the original spatiotemporal feature tensor; Step 2: Perform region instance segmentation on the original spatiotemporal feature tensor to obtain independent instance masks for each handwashing station; use the independent instance masks to perform spatial feature clipping and adjacent region interference suppression operations on the original spatiotemporal feature tensor to obtain a spatially decoupled instance feature set that maps one-to-one with each handwashing station. Step 3: Perform temporal trajectory tracking processing on the spatial decoupled instance feature set to extract the inter-frame displacement sequence of limb key points within the mask coverage area of ​​each independent instance; based on the inter-frame displacement sequence of limb key points, perform motion continuity interpolation compensation on the trajectory break segments caused by dynamic occlusion, and reassemble the compensated discrete motion segments across frames according to the instance identifier and bind them to the identity to obtain the temporally isolated behavior sequence chain. Step 4: Perform stage state mapping processing on the time-series isolation behavior sequence chain, extract the features of each motion segment, and perform multi-dimensional feature similarity matching with the preset discrete state nodes of the standard hand disinfection process; perform state transition probability threshold determination based on multi-dimensional feature similarity, and perform weight decay filtering on non-standard deviation actions to obtain the stage compliance state vector corresponding to each handwashing area. Step 5: Perform parallel compliance decision processing on the stage compliance status vector, and perform multi-instance concurrent logic matching and verification by combining the preset process timing topology constraints and the standard operation duration threshold of each stage to obtain the staged hand disinfection behavior evaluation result data stream that strictly corresponds to the spatial location of the handwashing area.

[0012] In this embodiment of the invention, by segmenting regional instances and pruning spatial features, independent operation instances of multiple handwashing stations and multiple target objects are separated, effectively suppressing motion interference in adjacent areas. Combined with inter-frame displacement sequences of limb key points and motion continuity interpolation compensation, stable identity binding and behavior reorganization across frame sequences are achieved, improving the integrity and temporal coherence of motion trajectories in complex environments. By using stage state mapping and multi-dimensional feature similarity matching, combined with state transition probability thresholds and weight attenuation filtering of non-standard deviation actions, the accuracy of compliance judgment and anti-interference ability are improved. Based on temporal topological constraints and standard operation duration thresholds, parallel compliance judgment is achieved, enabling independent and synchronous evaluation of each handwashing station for multiple handwashing areas and multiple target objects.

[0013] In a preferred embodiment of the present invention, step 1 above may include: The video capture device is a color network camera with a resolution of 1920×1080 pixels and a frame rate of 30 frames per second, installed approximately 2.2 meters above the handwashing area, simultaneously covering the operating areas of three handwashing sinks from a top-down perspective. Video data processing is performed in real time on an edge computing device equipped with a GPU accelerator card. This edge computing device is deployed in a server rack near the handwashing area and connected to the camera via gigabit Ethernet. The edge computing device continuously receives real-time video streams from the top camera through a network interface. The video stream enters the processing pipeline in the form of a continuous frame sequence, with each frame being an RGB three-channel color image. The video decoding module in the edge computing device decompresses the compressed video stream into a frame-by-frame raw pixel matrix sequence. Each decoded image frame is input into a pre-trained deep convolutional neural network backbone for frame-by-frame feature encoding. In this embodiment, the backbone network adopts a ResNet-50 structure pre-trained on the ImageNet large-scale image classification dataset, but only retains the part from the input layer to the output of the fourth-stage convolutional module. The global average pooling layer and fully connected classification layer at the end of the original ResNet-50 are removed, so that the backbone network outputs a feature map with a spatial size of one-sixteenth of the input image size and 1024 channels for each frame of input image. This feature map is the two-dimensional spatial feature representation of the frame, where the feature vector at each spatial location encodes the texture, edge, shape and semantic information of the corresponding receptive field region in the input image.

[0014] The two-dimensional spatial feature maps of T consecutive frames in the video stream are stacked along the time dimension to form a four-dimensional tensor structure. Its four dimensions are, in order, the time frame number dimension (length T), the feature channel dimension (length 1024), the feature map height dimension (length H / 16, where H is the height of the input frame in pixels), and the feature map width dimension (length W / 16, where W is the width of the input frame in pixels). This four-dimensional tensor is the original spatiotemporal feature tensor, which contains both the spatial distribution information of the operation actions in each handwashing area in the monitoring scene and the information on the evolution of the actions along the time axis.

[0015] In actual deployment, the value of T is determined based on the memory capacity and processing latency requirements of the edge computing device. In this embodiment, T is set to 32 frames, corresponding to a continuous video segment of about 1 second. This window length is sufficient to cover the complete execution cycle of a single standard handwashing action, while not introducing excessive computational latency. For a video frame with an input resolution of 1920×1080, the original spatiotemporal feature tensor after backbone network encoding has a shape of [32, 1024, 120, 68] (assuming the height dimension is 1920 / 16=120 and the width dimension is 1080 / 16≈68, rounded down).

[0016] In a preferred embodiment of the present invention, step 2 above may include: After obtaining the original spatiotemporal feature tensor, it is necessary to separate the spatial information of multiple people operating concurrently at different handwashing stations, so that subsequent trajectory tracking and action recognition can be executed independently at each handwashing station without interference from operations in adjacent areas. In this embodiment, a Mask R-CNN is used as the core model for region instance segmentation. This model adds a fully convolutional mask prediction branch parallel to the bounding box regression branch to the object detection framework of Faster R-CNN, which can generate a pixel-level binary mask for each candidate object region while detecting it. Specifically, the feature map of the current frame (i.e., the T-th frame, taking the last frame of the video segment to ensure that the latest arriving scene is processed) in the original spatiotemporal feature tensor is input into Mask R-CNN. In the R-CNN region proposal network, a small convolutional network slides across the feature map output by the backbone network. At each sliding window position, it simultaneously predicts whether multiple anchor boxes of different scales and aspect ratios contain the target object (i.e., the hand sanitizing operator) and the offset of the anchor box relative to the real target bounding box. Through the preset anchor box scale set and aspect ratio set, the region proposal network can cover the scale changes caused by different body types of operators and different shooting distances that may occur in the scene.

[0017] The candidate regions output by the region proposal network undergo non-maximum suppression based on the intersection-union ratio (IU) threshold. Redundant detection boxes with an overlap greater than a preset threshold with higher-confidence candidate boxes are removed, and only a few candidate regions with the highest confidence are retained. In this embodiment, the IU threshold is set to 0.7, that is, when the ratio of the overlap area to the union area of ​​two candidate boxes exceeds 0.7, the one with higher confidence is retained. The upper limit of the number of candidate regions retained after non-maximum suppression is set to a maximum of 3 candidate regions per handwashing station. The retained candidate regions extract fixed-size feature blocks from the feature map through the region of interest alignment operation and input them into the three parallel output branches of Mask R-CNN. The classification branch outputs the probability score of the candidate region belonging to the operator category. The bounding box regression branch outputs the fine-tuning offset of the candidate box position. The mask prediction branch outputs a two-dimensional probability map of a fixed size (28×28 pixels in this embodiment), where the value of each pixel represents the probability that the pixel belongs to the current operator's foreground region.

[0018] The probability map output from the mask prediction branch is binarized pixel-by-pixel. Pixels with a probability value greater than 0.5 are set as foreground (value 1), and the remaining pixels are set as background (value 0). This yields a binarized independent instance mask that corresponds one-to-one with the spatial location of each handwashing station. Each mask uniquely identifies a person performing hand sanitization and the handwashing basin area they occupy. Next, spatial feature clipping and adjacent region interference suppression operations are performed using each independent instance mask. For each independent instance mask corresponding to a handwashing station, based on its contour coordinates, a local spatiotemporal feature sub-map covering only the pixels inside the current handwashing station mask is extracted frame-by-frame from the original spatiotemporal feature tensor. The specific operation involves expanding the binarized independent instance mask along the spatial dimensions (height and width) into a binary matrix with the same spatial size as the original spatiotemporal feature tensor. Then, this binary matrix is ​​multiplied element-wise with each frame feature map of the original spatiotemporal feature tensor. The features of the region inside the mask (pixels with a mask value of 1) are preserved, while the features of the region outside the mask (pixels with a mask value of 0) are set to zero. After this operation, the non-zero feature region within the minimum bounding rectangle of the mask is extracted to obtain a local spatiotemporal feature sub-map containing only the operator information of the current handwashing station. This local spatiotemporal feature sub-map is the initial spatial decoupling feature, which initially separates the features of different handwashing stations in space.

[0019] Because adjacent handwashing stations are spatially close, the initial spatial decoupling features may still be affected by the operations of adjacent areas. For example, the hands of operators at adjacent handwashing stations may partially enter the mask area of ​​the current handwashing station. To suppress such interference, an adjacent area interference suppression operation is further performed. For the currently processed handwashing station, independent instance masks of its adjacent handwashing stations (handwashing stations located to the left and right of the current handwashing station in physical layout) are obtained. Morphological dilation operation is performed on each adjacent mask. A circular structuring element with a radius of r pixels is used to dilate the adjacent masks, so that the boundaries of the adjacent masks are extended outward by r pixels. After normalization, the dilated masks form a position weight matrix with values ​​between 0 and 1. The values ​​in this weight matrix represent the intensity of interference from adjacent handwashing station operations on the corresponding pixel position in the original image. The weight value is higher in the dilated area (the closer to the original boundary of the adjacent mask), indicating stronger interference. The weight value of the area far from the adjacent mask is close to zero. In this embodiment, the value of the dilation radius r is related to the physical distance between the handwashing stations and the image resolution. In this implementation scenario, r = 15 pixels.

[0020] The position weight matrix is ​​inverted (i.e., each position's weight value is subtracted from 1) to obtain the interference suppression weight matrix. In this matrix, the weight values ​​of regions closer to adjacent handwashing stations are lower (tending to 0), and the weight values ​​of regions farther from adjacent handwashing stations are higher (tending to 1). The interference suppression weight matrix is ​​multiplied element-wise with the local spatiotemporal feature sub-map of the initial spatial decoupling features to obtain the local spatiotemporal feature sub-map after interference suppression in adjacent regions. In this process, the feature values ​​of spatial locations affected by the operation in adjacent regions are attenuated proportionally to the interference intensity, while the features of undisturbed regions are completely preserved. The local spatiotemporal feature sub-maps of each handwashing station after interference suppression in adjacent regions are structured and encapsulated according to their corresponding handwashing station indices (e.g., numbered from left to right as handwashing station 1, handwashing station 2, and handwashing station 3) to form a spatial decoupling instance feature set. This feature set is a dictionary structure with the handwashing station index as the key and the corresponding local spatiotemporal feature sub-map as the value. Each value contains the complete spatiotemporal features of the handwashing station within a multi-frame time range, and the spatial interference from adjacent handwashing stations has been excluded.

[0021] In a preferred embodiment of the present invention, step 3 above may include: After obtaining the independent spatial decoupling instance feature sets for each handwashing station, temporal trajectory tracking processing is performed on each handwashing station to extract the movement trajectory of the operator's limbs within the mask coverage area and to compensate for trajectory breaks caused by dynamic occlusion. According to the time sequence of video frames, the instance feature sub-map corresponding to the current frame is extracted from the spatial decoupling instance feature set frame by frame. The instance feature sub-map is input into the multi-person skeleton key point detection process based on heatmap regression. This embodiment adopts a cascaded prediction network with a convolutional attitude machine architecture. This network contains multiple sequentially connected prediction stages. Each stage further refines the heatmap prediction of key points based on the prediction results of the previous stage.

[0022] In each prediction stage, the network performs a series of convolutional operations on the input feature map and outputs heatmap channels with the same number of human keypoint categories. Each channel corresponds to a human keypoint (such as wrist joint, elbow joint, and shoulder joint). The value of each pixel in the heatmap represents the probability score of the pixel location belonging to the corresponding keypoint. In this embodiment, for the evaluation needs of hand disinfection operation, the keypoint categories include six keypoints: left wrist joint, right wrist joint, left elbow joint, right elbow joint, left shoulder joint, and right shoulder joint. The motion trajectory of these keypoints can completely characterize the spatial motion pattern of the arm when performing each step of the seven-step handwashing method.

[0023] Non-maximum suppression is performed on each heatmap channel to find local maxima in each heatmap. Second-largest points whose distance to higher response points is less than a preset spacing threshold are removed, ensuring that only a few mutually separated high-confidence keypoints are retained in each heatmap channel. Coordinate regression is performed on the retained local maxima. Using the integer coordinates of the pixel containing the local maximum as the initial estimate, the centroid of the heatmap response values ​​in its neighborhood is taken as the sub-pixel precision keypoint coordinates. Let a local maximum be located at pixel coordinates (i, j), and the heatmap response values ​​of each pixel in its neighborhood (e.g., a 3×3 window) be h(p, j). If q), then the sub-pixel coordinates of the keypoint are calculated as follows: Calculate the sum of the heatmap response values ​​of each pixel in the neighborhood to obtain the total neighborhood response; accumulate the product of the horizontal coordinate of each pixel in the neighborhood with the heatmap response value of that pixel to obtain the weighted sum of the horizontal coordinates; divide the weighted sum of the horizontal coordinates by the total neighborhood response to obtain the sub-pixel horizontal coordinates of the keypoint; accumulate the product of the vertical coordinate of each pixel in the neighborhood with the heatmap response value of that pixel to obtain the weighted sum of the vertical coordinates; divide the weighted sum of the vertical coordinates by the total neighborhood response to obtain the sub-pixel vertical coordinates of the keypoint.

[0024] The above operations are performed independently for each detected keypoint, ultimately obtaining the sub-pixel precision coordinates of the wrist, elbow, and shoulder joints of each operator in the current frame in the image coordinate system, forming the limb keypoint coordinate set of the current frame. This coordinate set is stored in the form of a two-dimensional array, with each row of the array corresponding to a detected keypoint, including the following fields: keypoint category identifier (indicating whether the point is left wrist / right wrist / left elbow / right elbow / left shoulder / right shoulder), horizontal coordinate value (pixels), vertical coordinate value (pixels), and confidence score (heatmap response peak). Next, based on the limb keypoint coordinate set of the current frame, the limb keypoint coordinate set of the same handwashing point and the same operator stored in the previous frame is retrieved. For the same keypoint category (such as the right wrist joint of the same operator), the difference between the coordinates of the keypoint in the current frame and the coordinates in the previous frame is calculated to obtain the Euclidean displacement vector of the keypoint between two adjacent frames. This displacement vector is a two-dimensional vector, with its two components being the horizontal coordinate difference and the vertical coordinate difference, which represent the pixel displacement of the keypoint in the horizontal and vertical directions during the time interval between two adjacent frames, respectively.

[0025] The Euclidean displacement vector of each keypoint is sequentially appended to the displacement sequence buffer corresponding to the handwashing station and the keypoint category in ascending order of frame number. This buffer is a two-dimensional array structure with keypoint category as the first-level index and frame number as the second-level index. As video frames continue to arrive, the displacement sequence in the buffer continuously grows, forming the original inter-frame displacement sequence. This original inter-frame displacement sequence records the frame-by-frame motion trajectory changes of each keypoint of each operator's limb within the video time period. The original inter-frame displacement sequence is then associated with the corresponding handwashing station instance identifier (indicating which handwashing station the displacement sequence belongs to) and the timestamp of the current frame (precisely...). The displacement record entry is obtained by combining the temporary tracking index with the video frame timeline (accurate to milliseconds). The temporary tracking index is used to associate the same operation target across frames within the same handwashing station. It is a temporary number assigned to each detected operator within the mask area of ​​the current handwashing station to ensure that the same operator maintains the same temporary tracking index in the preceding and following frames. At this time, the final user identity (such as employee number or face recognition ID) is not bound. The final binding of the user identity will be completed in subsequent steps. The displacement record entry is structured and stored in the order of the video frame timeline and in units of handwashing stations to obtain the inter-frame displacement sequence of limb key points.

[0026] In real-world scenarios with multiple concurrent users, certain key points in some frames may be temporarily invisible due to body occlusion between operators or the crossing of operators' arms, leading to breaks in the inter-frame displacement sequence. Therefore, it is necessary to perform motion continuity interpolation compensation for these trajectory breaks caused by dynamic occlusion based on the inter-frame displacement sequence of limb key points. Specifically, this involves iterating through the inter-frame displacement sequence of limb key points corresponding to each handwashing station, checking the inter-frame displacement vector of each key point frame by frame. For each key point, the magnitude of the displacement vector in the current frame (i.e., the square root of the sum of the squares of the differences between the horizontal and vertical coordinates) is calculated and compared with the magnitude of the displacement vector in the previous frame. When a sudden change in the magnitude of the displacement vector is detected between two adjacent frames, i.e., the magnitude of the displacement vector in the current frame exceeds the threshold, the process is considered complete. When the preset motion speed threshold is exceeded, the continuous frame interval that meets the mutation condition is marked as a dynamic occlusion candidate segment. The motion speed threshold is set based on the biomechanical characteristics of human upper limb movement. In normal hand disinfection operation, the movement speed of the hand between adjacent frames is constrained by the range of motion of bones and joints. Its inter-frame displacement usually does not exceed a certain pixel distance (depending on the frame rate and camera imaging parameters). When the inter-frame displacement of a key point suddenly exceeds the upper limit of the constraint, it indicates that the key point may be lost due to occlusion in the middle frame or has been incorrectly matched to the key point of other objects. The occlusion interval is marked accordingly. Under the deployment parameters of this embodiment (30 frames / second, about 2.2 meters shooting distance), the motion speed threshold is 45 pixels per frame.

[0027] For marked dynamic occlusion candidate segments, motion continuity interpolation compensation needs to be performed on the missing frame intervals of the segment. This involves retrieving the nearest normal displacement vector sequence segment before the starting frame of the occlusion interval (i.e., the most recent frame interval not marked as occluded) and the nearest normal displacement vector sequence segment after the ending frame of the occlusion interval. Based on the displacement vector information in these two normal segments, the displacement vectors of each key point within the missing frame interval are predicted and filled frame by frame. This embodiment uses a linear interpolation method based on the assumption of uniform motion for prediction. The occlusion interval is assumed to start from frame number m and end at frame number n (a total of missing segments). (1 frame), the displacement vector of the nearest normal frame before occlusion is (Its horizontal and vertical components are) and The displacement vector of the nearest normal frame after occlusion is (Its horizontal and vertical components are) and For the i-th frame (i from m to n) within the occlusion interval, the horizontal component of its predicted displacement vector is calculated by weighting the horizontal component of the displacement vector of the normal frame before occlusion and the horizontal component of the displacement vector of the normal frame after occlusion. The weight changes linearly with the frame number (the closer to the frame before occlusion, the greater the weight of the displacement vector before occlusion; the closer to the frame after occlusion, the greater the weight of the displacement vector after occlusion). The vertical component is calculated using the same weighting method.

[0028] After the above interpolation processing, each frame within the occlusion interval obtains a displacement vector predicted by the motion continuity assumption, forming a compensated continuous displacement vector sequence. The compensated continuous displacement vector sequence is then concatenated with the normal displacement vector sequence without occlusion, following frame number order, to obtain a complete and continuous displacement trajectory sequence for each keypoint within each handwashing station. This trajectory sequence represents the entire motion history from the first detection to the last detection of the keypoint, without any interrupted frames. Based on the instance identifier of the same handwashing station, the complete and continuous displacement trajectory sequences of all keypoints (left and right wrists, left and right elbows, and left and right shoulders, a total of six keypoints) within that handwashing station are then processed along the time axis. Alignment is performed through cross-frame association verification using spatial constraints of multiple key points. Utilizing prior knowledge of human structure—that any two key points have fixed connections within the human skeleton model (such as forearm length constraints and upper arm length constraints)—the system verifies whether the relative positions of the six key points of the same operator in each frame conform to the geometric constraints of the human skeleton. If the distance between key points remains stable across multiple frames (with fluctuations within a preset tolerance range), then these key points are confirmed to belong to the same operator during the same operation period. If the spatial constraint relationship between a key point and the other key points suddenly breaks or deviates significantly in a particular frame, it indicates that the key point may have been mismatched in that frame, requiring re-verification.

[0029] After the spatial constraint verification described above, displacement trajectory segments belonging to the same operator and the same operation time period are recombined into a single motion link. Simultaneously, this motion link is persistently bound to the operator's identity identifier, identified through facial recognition or work card reading. The identity identifier is obtained as follows: an auxiliary identification camera or RFID (Radio Frequency Identification) reader is installed at the entrance of the handwashing area or above the front of the handwashing station. When an operator enters the handwashing area, their identity is determined through facial recognition or work card information reading, and a globally unique identifier is assigned. This identifier is associated with the motion link through a timestamp and spatial location (handwashing station number). That is, during the time period when the operator enters a certain handwashing station area, the motion link detected at that handwashing station is bound to the operator's operation record. The motion links bound with the identity identifier are stored according to the handwashing station index, forming a temporally isolated behavioral sequence chain for each handwashing station, with clear operator identification and continuous temporal sequence. Each sequence chain contains the following information: handwashing station number, operator identity identifier, start timestamp, end timestamp, and a sequence of pixel coordinates of six key points in each frame, indexed by the frame number.

[0030] In a preferred embodiment of the present invention, step 4 above may include: After obtaining each time-series isolated behavior sequence chain, it is necessary to perform stage state mapping on the behavior sequence and match it with each step of the standard hand disinfection process to determine whether the operator's behavior in each handwashing stage is compliant. For the time-series isolated behavior sequence chain corresponding to each handwashing point, a sliding window segment is performed according to the preset time window length. The behavior sequence chain is divided according to a fixed time window length, and a preset overlap ratio is set between adjacent windows. In this embodiment, the time window length is 0.5 seconds (corresponding to 15 frames), and the overlap ratio of adjacent windows is 50% (i.e., sliding once every 7 to 8 frames) to balance time resolution and computational efficiency. The key point displacement data within each time window constitute a motion segment.

[0031] For each motion segment, three types of motion features are extracted to form the original motion feature vector of the segment. The first type of feature is the spatial distribution feature of the displacement trajectory of the limb key points. For each of the six key points, the mean and standard deviation of the horizontal coordinate, the mean and standard deviation of the vertical coordinate, and the covariance between the horizontal and vertical coordinates are calculated for all frame coordinates in the image plane within the time window. These statistics describe the overall trend of the spatial position center, range of motion, and direction of movement of the operator's hand and arm within the time window. Five statistics are provided for each key point, and the six key points have a total of 30-dimensional spatial distribution features.

[0032] The second type of feature is the motion direction change feature. For each key point, the direction angle of the displacement vector between adjacent frames (i.e., the angle between the displacement vector and the horizontal direction) is calculated frame by frame, and the change range of the direction angle within the time window is statistically analyzed, including the mean, standard deviation, and mean of the absolute value of the difference of the direction angle between adjacent frames. This describes whether the operator's hand is performing operations with specific directional change patterns such as reciprocating rubbing or rotating within the time window. There are six key points, each providing three directional change features, for a total of 18 dimensions.

[0033] The third type of feature is the temporal feature of motion rate. For each key point, the displacement vector magnitude (i.e., motion rate) is calculated frame by frame, and the mean, maximum, minimum, median and the slope of the rate change over time are statistically analyzed within the time window (the slope is obtained by linearly fitting the temporal curve of the rate). The magnitude of the rate reflects the strength and rhythm of the operator's kneading action, and the degree of fluctuation of the rate reflects the regularity and uniformity of the action. Each of the six key points provides five temporal features of the rate, for a total of 30 dimensions.

[0034] The three types of features (30-dimensional spatial distribution, 18-dimensional orientation change, and 30-dimensional temporal velocity) are concatenated into a 78-dimensional original motion feature vector. This 78-dimensional original motion feature vector is then input into a pre-trained motion feature embedding network. This motion feature embedding network adopts a three-layer fully connected neural network structure. The first layer maps the 78-dimensional input to 256 dimensions, the second layer maps the 256-dimensional input to 128 dimensions, and the third layer maps the 128-dimensional input to a 64-dimensional output. Each fully connected layer is followed by a batch normalization layer and a non-linear activation layer (using the ReLU activation function). The output of the network is a standardized high-dimensional semantic feature vector (64-dimensional) for the motion segment. This 64-dimensional high-dimensional semantic feature vector is a high-level semantic abstraction of the original 78-dimensional motion features. It maps the underlying motion statistics to a more discriminative semantic space, allowing motion segments with the same handwashing steps to cluster together in this space, while motion segments with different steps are separated from each other in this space.

[0035] The training process of this motion feature embedding network is as follows: A large amount of hand disinfection operation video data labeled with each step of the standard seven-step handwashing method is collected as a training sample set. For each video segment in the training sample set, motion segments are extracted and 78-dimensional original motion feature vectors are calculated using the same sliding window segmentation method as described above. During training, a seven-class classification layer (Softmax classifier) ​​is temporarily connected after the 64-dimensional output layer. The network weights are updated using the backpropagation algorithm with the cross-entropy loss function as the optimization objective. During training, the learning rate adopts a cosine annealing strategy. After the initial value is set, it is gradually reduced according to the preset decay schedule during training. The training batch size is dynamically determined according to the GPU memory capacity, and random sampling is performed from the training set in each iteration. An early stopping strategy is adopted, and training is terminated when the performance of the validation set does not improve within the preset observation window. After training is completed, the temporary classification layer is removed, and only the feature embedding part from the 78-dimensional input to the 64-dimensional output is retained as the motion feature embedding network.

[0036] The preset set of discrete state nodes for the standard hand disinfection procedure is retrieved. In this embodiment, the set includes the following seven discrete state nodes in sequence according to the standard seven-step handwashing procedure: the first state node corresponds to the palm rubbing state (the operator rubs their palms together with their fingers interlaced); the second state node corresponds to the back of the hand rubbing state (one palm rubs the back of the other hand along the finger gaps); the third state node corresponds to the finger gap rubbing state (both palms face each other with fingers interlaced and rubbing along the finger gaps); the fourth state node corresponds to the back of the finger rubbing state (bending each finger joint and rubbing in the palm of the other hand); the fifth state node corresponds to the thumb rubbing state (one hand grasps the thumb of the other hand and rubs it in rotation); the sixth state node corresponds to the fingertip rubbing state (five fingertips are rubbed together in the palm of the other hand); and the seventh state node corresponds to the wrist rubbing state (one hand rubs the wrist of the other hand). Each state node is pre-configured with a corresponding reference motion semantic feature template. This template is obtained by taking the average of the 64-dimensional high-dimensional semantic feature vectors of a large number of training samples labeled as that state, representing the standard motion pattern of that state.

[0037] For the high-dimensional semantic feature vector of the current motion segment, the cosine similarity and Euclidean distance between it and the reference motion semantic feature templates of seven preset discrete state nodes are calculated respectively, and the two are fused into a comprehensive similarity score. The cosine similarity measures the closeness of two vectors in direction, and the value ranges from -1 to 1 (the larger the value, the more consistent the direction). The Euclidean distance measures the closeness of two vectors in absolute position, and the smaller the distance, the closer they are. In order to unify the two into a comprehensive score, this embodiment adopts the following fusion method: calculate the cosine similarity between the semantic vector of the current segment and a certain state template vector to obtain the cosine similarity score; calculate the Euclidean distance between the semantic vector of the current segment and a certain state template vector to obtain the Euclidean distance value; after taking the negative sign of the Euclidean distance value, it is scaled by a preset scale parameter so that its numerical range is comparable to the cosine similarity score; the cosine similarity score and the scaled negative Euclidean distance value are weighted and summed (the weight of cosine similarity is 0.5, and the weight of scaled negative Euclidean distance is 0.5) to obtain the comprehensive similarity score.

[0038] After performing the above calculations on each of the seven state nodes, a seven-dimensional multi-dimensional feature similarity vector is generated for the current motion segment. The value of the j-th dimension in this vector represents the comprehensive similarity score between the current motion segment and the j-th standard state node. Based on the multi-dimensional feature similarity, a state transition probability threshold determination is performed. Following the hand disinfection standard procedure, the discrete state nodes are pre-set in temporal order (progressing sequentially from the first state to the seventh state). The motion segment sequence is processed segment by segment. For the current motion segment, the state transition probability from the previously matched state node to the current candidate state node is calculated. The state transition probability is calculated as follows: if the comprehensive similarity score between the current motion segment and the candidate state node j is the highest among all seven state nodes, and the difference between this score and the second highest score exceeds a preset discrimination threshold, then the current segment is matched to state node j. If the difference between the highest score and the second highest score is small (below the discrimination threshold), or the highest score itself is below a preset minimum matching threshold, then the current segment is considered unable to obtain a reliable state match.

[0039] In this embodiment, the preset transition probability threshold includes two threshold parameters: a minimum matching threshold (within the range of zero to one) used to filter matches with excessively low overall similarity; and a discrimination threshold (within the range of zero to one) used to ensure the exclusivity of the match. The specific judgment rules are as follows: if the comprehensive similarity score between the current motion segment and a certain standard state node exceeds the minimum matching threshold, and the difference between this score and the second highest comprehensive similarity score exceeds the discrimination threshold, then the current motion segment is determined to have successfully matched the state node, and the identifier of the state node (such as rubbing the palm) is written into the corresponding temporal position of the phase compliance state vector being constructed; if the highest comprehensive similarity score of the current motion segment is lower than the minimum matching threshold, or the difference between the highest score and the second highest score is less than the discrimination threshold, then the current motion segment is determined to have failed to match any standard state node, triggering a non-standard deviation action marker.

[0040] For motion segments marked as non-standard deviations, further weighted attenuation filtering is performed to extract the motion direction change features and motion rate temporal features from the original motion feature vector of the segment. The degree of deviation between the motion features of the segment and the successfully matched state nodes in the adjacent time window is calculated. The degree of deviation is calculated by using the Euclidean distance between the direction change feature vector of the segment and the direction change feature vector of the adjacent matched segments as the direction deviation measure; and the Euclidean distance between the velocity temporal feature vector of the segment and the velocity temporal feature vector of the adjacent matched segments as the velocity deviation measure. The larger of the two values ​​is taken as the comprehensive degree of deviation.

[0041] The overall similarity score of the segment is weighted and attenuated based on the degree of deviation. The greater the deviation, the smaller the attenuation coefficient (i.e., the overall similarity score is multiplied by a coefficient that decreases with the degree of deviation). After attenuation, if the overall similarity score drops to an extremely low level, the movement segment is classified as a non-compliant movement segment. If the overall similarity score remains at a medium or higher level after attenuation, the movement segment is classified as a vague movement segment and participates in the statistics with a lower weight in subsequent evaluations. The state node identifier (for successfully matched segments), compliance judgment result (compliant / non-compliant / fuzzy), and timestamp information (segment start time and end time) corresponding to each movement segment obtained after state transition judgment and non-standard deviation action attenuation filtering are structured and organized according to the handwashing point index and time order to obtain the stage compliance state vector corresponding to each handwashing area.

[0042] The phase compliance status vector is a list of records arranged in chronological order. Each record is a multi-dimensional element containing the following four fields: status node name (e.g., rubbing palm, rubbing back of hand, etc.; for non-compliant segments, it is a non-standard action), entry time (absolute timestamp of the start of the status or action segment), exit time (absolute timestamp of the end of the status or action segment), and compliance judgment result (values ​​are compliant, ambiguous, or non-compliant).

[0043] In a preferred embodiment of the present invention, step 5 above may include: After obtaining the stage compliance state vector corresponding to each handwashing station, parallel compliance decision processing is performed. Multi-instance concurrent logic matching and verification are performed by combining the preset process time topology constraints and the standard operation duration thresholds for each stage. The state node sequence in the stage compliance state vector is extracted and sorted by timestamp in ascending order (only compliant state nodes are taken, and segments marked as ambiguous or non-compliant are skipped). The state node sequence is matched and verified with the preset process time topology constraints in a directed path. The process time topology constraints are defined as a directed acyclic topology path with seven standard handwashing states as nodes. The directed edges represent the legal transition directions between states. Only transitions from the k-th state node to the (k+1)-th state node (adjacent successor) are allowed. Cross-node jumps (such as jumping directly from the first state to the third state) are prohibited, reverse transitions (such as going back from the third state to the second state) are prohibited, and multi-state unordered branch jumps (such as multiple states being marked in parallel within the same time period) are prohibited.

[0044] The specific execution method of directed path matching verification is as follows: the sequence of state nodes extracted from the stage compliance state vector is compared node by node with the standard directed acyclic topology path. For the first state node in the sequence, it is checked whether it is the first state node (rubbing palm); for the i-th state node in the sequence (i>1), it is checked whether it is the i-th state node in the standard path, and whether its immediate preceding state node is the (i-1)-th state node in the standard path. The following abnormalities are detected when each handwashing point executes the process: stage missing, a step in the standard seven steps is completely absent in the entire operation sequence; reverse jump, a step appears before its proper preceding step; illegal branch, two or more contradictory state markers appear in the same time period. The detection results form the intermediate data of temporal topology verification. This data records in tabular form the existence (existence / missing), jump direction (forward compliance / reverse anomaly) of each step in the process execution of each handwashing point, and whether there is a branch conflict.

[0045] Based on the time span of each consecutive state node in the intermediate data of the time-series topology verification, the actual duration of each standard hand disinfection stage is calculated. For each group of adjacent compliant state nodes, the difference between the entry time of the next state and the entry time of the previous state is taken as the actual execution time of the previous stage. The actual duration of each stage is compared with the preset standard operation time threshold for each stage for determination. In this embodiment, the standard operation time threshold for each stage is set based on the hand hygiene standards of medical institutions (such as the hand hygiene guidelines issued by the World Health Organization). The recommended duration of each rubbing step is usually between 10 and 15 seconds. There are some individual differences between each step. Therefore, a lower limit (such as 5 seconds) and an upper limit (such as 20 seconds) are set for each stage.

[0046] The interval comparison judgment rule is as follows: if the actual duration is between the lower and upper limits of the duration, the duration of that stage is judged to be up to standard; if the actual duration is lower than the lower limit of the duration, the duration of that stage is judged to be insufficient (the duration deviation is recorded as the difference between the actual duration and the lower limit of the duration); if the actual duration is higher than the upper limit of the duration, the duration of that stage is judged to be excessive (the duration deviation is recorded as the difference between the actual duration and the upper limit of the duration). For abnormal periods that exceed the upper limit of the duration or are lower than the lower limit of the duration, a threshold truncation filter is performed. In the compliance statistics, the truncation value is used instead of the actual value for calculation to avoid the excessive impact of extreme deviation values ​​on the overall assessment. The result is a duration compliance verification result that includes the duration compliance status of each stage (up to standard / insufficient duration / excessive duration) and the quantitative value of the duration deviation.

[0047] Based on the duration compliance verification results, multi-instance concurrent logical matching verification is performed. The purpose of multi-instance concurrent logical matching verification is that multiple handwashing stations may have operators performing hand disinfection operations simultaneously within the same period. It is necessary to perform spatiotemporal cross-fusion processing on the evaluation results of each handwashing station within the parallel monitoring period to form a unified evaluation output. The topological connectivity status of each handwashing station within the parallel monitoring period (i.e., the completion status of the seven stages) and duration compliance characteristics (duration deviation values ​​of each stage) are cross-fused and weighted in the spatiotemporal dimensions. For each handwashing station, the topological completion degree of the seven stages (the number of stages that have passed compliance divided by seven), duration compliance degree (the normalized reciprocal of the mean of the absolute values ​​of the duration deviations of all stages), and process completeness score (the weighted product of topological completion degree and duration compliance degree, with each weight of 0.5) are calculated. The process completeness score ranges from 0 to 1, where 1 indicates full compliance (topological completeness and duration compliance), and 0 indicates complete non-compliance.

[0048] A serialized data packet is generated, which contains the following: spatial coordinate mapping of each handwashing station (describing the spatial location information of each handwashing station in the monitoring screen, represented by the center pixel coordinates of the outer rectangle of the mask), compliance level identifier for each stage (each stage is independently labeled as one of four levels: insufficient compliance time, excessive time, or missing time), and overall process completeness score. The serialized data packet is structured and encapsulated according to the video stream synchronization frame rate. Whenever a new frame of video data arrives and all the above analysis steps are completed, the updated evaluation result is appended to the output data stream, forming a phased hand disinfection behavior evaluation result data stream that strictly corresponds to the spatial location of the handwashing area.

[0049] The assessment results data stream is reported to the hospital infection control management platform in real time via a network interface. The platform's visualization interface displays each handwashing station as an independent display unit, showing the current operator's identity information, the handwashing steps being performed or completed, the time taken for each step, and the overall compliance score. When a missing step, insufficient time, or other serious non-compliance is detected, the edge computing device can also issue a real-time corrective reminder to the operator through an audio-visual prompt device connected to the handwashing area.

[0050] like Figure 2 As shown, embodiments of the present invention also provide a multi-person parallel hand disinfection behavior video data processing system based on region instance segmentation, including: The acquisition module is used to acquire the monitoring video stream of multiple target objects performing hand disinfection operations concurrently, and to perform frame-by-frame feature encoding on the monitoring video stream to obtain the original spatiotemporal feature tensor; The segmentation module is used to perform region instance segmentation processing on the original spatiotemporal feature tensor to obtain an independent instance mask for each handwashing station; the independent instance mask is used to perform spatial feature clipping and adjacent region interference suppression operations on the original spatiotemporal feature tensor to obtain a spatially decoupled instance feature set that is mapped one-to-one with each handwashing station. The compensation module is used to perform temporal trajectory tracking processing on the spatial decoupled instance feature set, extract the inter-frame displacement sequence of limb key points in the mask coverage area of ​​each independent instance; based on the inter-frame displacement sequence of limb key points, it performs motion continuity interpolation compensation on the trajectory break segments caused by dynamic occlusion, and reassembles the compensated discrete motion segments across frames according to the instance identifier and binds them to the identity to obtain the temporally isolated behavior sequence chain. The determination module is used to perform stage state mapping processing on the time-series isolation behavior sequence chain, extract the features of each motion segment, and perform multi-dimensional feature similarity matching with the preset discrete state nodes of the standard hand disinfection process; based on the multi-dimensional feature similarity, it performs state transition probability threshold determination, and performs weight decay filtering on non-standard deviation actions to obtain the stage compliance state vector corresponding to each handwashing area. The evaluation module is used to perform parallel compliance decision processing on the stage compliance status vector. It combines the preset process timing topology constraints with the standard operation duration threshold of each stage to perform multi-instance concurrent logical matching and verification, and obtains a data stream of staged hand disinfection behavior evaluation results that strictly correspond to the spatial location of the handwashing area.

[0051] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0052] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for processing video data of multi-person parallel hand disinfection behavior based on region instance segmentation, characterized in that, The method includes: Step 1: Acquire the monitoring video stream of multiple target objects performing hand disinfection operations concurrently, and perform frame-by-frame feature encoding on the monitoring video stream to obtain the original spatiotemporal feature tensor; Step 2: Perform region instance segmentation on the original spatiotemporal feature tensor to obtain independent instance masks for each handwashing station; use the independent instance masks to perform spatial feature clipping and adjacent region interference suppression operations on the original spatiotemporal feature tensor to obtain a spatially decoupled instance feature set that maps one-to-one with each handwashing station. Step 3: Perform temporal trajectory tracking processing on the spatial decoupled instance feature set to extract the inter-frame displacement sequence of limb key points within the mask coverage area of ​​each independent instance; based on the inter-frame displacement sequence of limb key points, perform motion continuity interpolation compensation on the trajectory break segments caused by dynamic occlusion, and reassemble the compensated discrete motion segments across frames according to the instance identifier and bind them to the identity to obtain the temporally isolated behavior sequence chain. Step 4: Perform stage state mapping processing on the time-series isolation behavior sequence chain, extract the features of each motion segment, and perform multi-dimensional feature similarity matching with the preset discrete state nodes of the standard hand disinfection process; perform state transition probability threshold determination based on multi-dimensional feature similarity, and perform weight decay filtering on non-standard deviation actions to obtain the stage compliance state vector corresponding to each handwashing area. Step 5: Perform parallel compliance decision processing on the stage compliance status vector, and perform multi-instance concurrent logic matching and verification by combining the preset process timing topology constraints and the standard operation duration threshold of each stage to obtain the staged hand disinfection behavior evaluation result data stream that strictly corresponds to the spatial location of the handwashing area.

2. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 1, characterized in that, The original spatiotemporal feature tensor is processed by region instance segmentation to obtain independent instance masks for each handwashing station. Spatial feature clipping and adjacent region interference suppression operations are then performed on the original spatiotemporal feature tensor using these independent instance masks to obtain a spatially decoupled instance feature set that maps one-to-one with each handwashing station, including: The original spatiotemporal feature tensor is input into a pre-trained mask region convolutional neural network, and pixel-level semantic segmentation and instance boundary regression are performed frame by frame to obtain a multi-channel probability response map including all candidate areas of handwashing stations. Based on the multi-channel probability response map, a non-maximum suppression operation based on the cross-union ratio threshold is performed on each candidate region to remove redundant detection boxes with an overlap higher than the preset threshold, and retain the detection results with the highest confidence to obtain a binary independent instance mask that corresponds one-to-one with the spatial location of each handwashing point. Based on the contour coordinates of the binarized independent instance mask, local spatiotemporal feature sub-images covering only the pixels inside the current handwashing point are extracted frame by frame from the original spatiotemporal feature tensor as local spatiotemporal feature sub-images of the initial spatial decoupling features. Based on the local spatiotemporal feature subgraph of the initial spatial decoupling feature, independent instance masks of the adjacent handwashing points of the current handwashing point are obtained. Morphological dilation operation is performed on the adjacent masks to obtain the position weight matrix representing the spatial distribution of neighborhood interference. The position weight matrix is ​​then inverted and multiplied element-wise with the local spatiotemporal feature subgraph to obtain the local spatiotemporal feature subgraph after the interference of the adjacent region is suppressed. Each local spatiotemporal feature sub-map is structured and encapsulated according to its corresponding handwashing point index, forming a spatially decoupled instance feature set that maps one-to-one with each handwashing point.

3. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 2, characterized in that, Temporal trajectory tracking processing is performed on the spatially decoupled instance feature set to extract the inter-frame displacement sequence of limb key points within the mask coverage area of ​​each independent instance, including: The instance feature sub-map of the current frame is extracted from the spatially decoupled instance feature set frame by frame according to the temporal order of the video frames. The instance feature sub-map is then input into the multi-person skeleton key point detection process based on heat map regression to obtain the heat map response distribution of user limb key points within the mask coverage area of ​​each handwashing point in the current frame. Non-maximum suppression and coordinate regression calculations are performed on the heatmap response distribution to extract the pixel coordinates of each user's wrist, elbow and shoulder joints in the image coordinate system in the current frame, forming the limb key point coordinate set of the current frame. Based on the set of limb key point coordinates, retrieve the set of limb key point coordinates of the same handwashing point and the same user stored in the previous frame, calculate the coordinate difference of the same joint point between two adjacent frames, and obtain the Euclidean displacement vector of each key point in the current frame relative to the previous frame. The Euclidean displacement vectors of each key point are sequentially appended to the corresponding handwashing point and the displacement sequence buffer of the corresponding key point in ascending order of frame number, forming the original inter-frame displacement sequence with key point category as sub-index and frame number as main index. The original inter-frame displacement sequence is jointly bound with the corresponding handwashing station instance identifier, the current frame timestamp, and the temporary tracking index to obtain displacement record entries with partial spatiotemporal labels. The temporary tracking index is used to associate the same operation target across frames within the same handwashing station and is not bound to the end user identity identifier for the time being. The displacement record entries are structured and stored in the order of the video frame timeline, with handwashing stations as the unit, to obtain the inter-frame displacement sequence of limb key points.

4. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 3, characterized in that, Based on the inter-frame displacement sequence of limb key points, motion continuity interpolation compensation is performed on trajectory break segments caused by dynamic occlusion. The compensated discrete motion segments are then reassembled across frames and bound to identities according to instance identifiers to obtain a temporally isolated behavior sequence chain, including: Obtain the inter-frame displacement sequence of limb key points corresponding to each handwashing station, traverse the inter-frame displacement vector of each key point in the sequence, detect the time position when the displacement vector magnitude changes abruptly between adjacent frames or exceeds the preset motion speed threshold, and mark the continuous frame interval that meets the abrupt change condition as dynamic occlusion candidate segments. For the marked dynamic occlusion candidate segments, retrieve the nearest normal displacement vector sequence segment before the start frame and the nearest normal displacement vector sequence segment after the end frame of the interval, and perform frame-by-frame prediction and filling of the displacement vectors of each key point in the missing frame interval to obtain the compensated continuous displacement vector sequence. The compensated continuous displacement vector sequence is spliced ​​with the normal displacement vector sequence without occlusion according to the frame number order to obtain a complete and continuous displacement trajectory sequence for each key point in each handwashing station. Based on the instance identifier of the same handwashing station, the complete and continuous displacement trajectory sequence of all key points is aligned according to the time axis, and cross-frame association verification is performed through the spatial constraint relationship of multiple key points. The displacement trajectory segments belonging to the same user and the same operation time period are recombined into a single motion link, and the motion link is persistently bound to the identified user identity. The bound motion links are stored separately according to the handwashing point index, forming a time-series isolated behavior sequence chain where each handwashing point is independent, the user identity is clear, and the time is continuous and unbroken.

5. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 4, characterized in that, The temporal isolation behavior sequence chain is processed by stage state mapping, features of each motion segment are extracted, and multi-dimensional feature similarity matching is performed with the preset discrete state nodes of the standard hand disinfection procedure, including: For each handwashing station, the sequential isolation behavior sequence chain is divided into several continuous motion segments by sliding window segmentation according to the preset time window length. For each motion segment, the spatial distribution features of the displacement trajectory of the limb key points, the features of the change in motion direction, and the temporal features of the motion rate are extracted and used to form the original motion feature vector of the segment. The original motion feature vector is input into a pre-trained action feature embedding network and mapped to a standardized high-dimensional semantic feature vector, which serves as the semantic representation of the motion segment. Retrieve a set of discrete state nodes for a pre-defined standard hand disinfection process, where each state node is pre-configured with a corresponding reference motion semantic feature template; calculate the cosine similarity and Euclidean distance between the high-dimensional semantic feature vector of the current motion segment and the reference motion semantic feature templates of each pre-defined discrete state node, and fuse the cosine similarity and Euclidean distance to obtain a comprehensive similarity score; A multidimensional feature similarity vector is generated for each motion segment. Each dimension of the vector represents the comprehensive similarity score between the current motion segment and a corresponding standard state node, thus completing the multidimensional feature similarity matching.

6. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 5, characterized in that, The discrete state node set of the standard hand disinfection process is based on the standard seven-step handwashing method, which includes the palm rubbing state, the back of the hand rubbing state, the interdigital rubbing state, the back of the fingers rubbing state, the thumb rubbing state, the fingertip rubbing state, and the wrist rubbing state.

7. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 5, characterized in that, Based on multidimensional feature similarity, a state transition probability threshold is determined, and non-standard deviation actions are filtered by weight decay to obtain the stage compliance state vector corresponding to each handwashing area, including: For each motion segment, the multidimensional feature similarity vector is used to calculate the state transition probability of the current motion segment from the previous state node to the current state node in turn, according to the time sequence of the discrete state nodes preset in the standard hand disinfection procedure. The state transition probability is compared with a preset transition probability threshold. If the transition probability is greater than or equal to the threshold value, the current motion segment is determined to have successfully matched the current state node, and the corresponding state node identifier is written into the corresponding temporal position of the stage compliance state vector. If the transition probability is less than the threshold value, the current motion segment is determined to have failed to match any standard state node, and a non-standard deviation action marker is triggered. For motion segments marked as non-standard deviation actions, the motion direction change features and motion rate temporal features are extracted from the original motion feature vector. The degree of deviation between the segment and the motion features of successfully matched state nodes in the adjacent time window is calculated. Based on the degree of deviation, the comprehensive similarity score of the segment is weighted and attenuated, and the attenuated motion segments are classified as non-compliant action segments. The state node identifiers, compliance judgment results, and timestamp information corresponding to each motion segment obtained after state transition judgment and non-standard deviation motion attenuation filtering are structured according to the handwashing point index and time sequence to obtain the stage compliance state vector corresponding to each handwashing area.

8. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 7, characterized in that, Each element in the phased compliance status vector includes the status node name, entry time, exit time, and compliance judgment result.

9. The method for processing multi-person parallel hand disinfection behavior video data based on region instance segmentation according to claim 7, characterized in that, Parallel compliance decision processing is performed on the phased compliance state vector. Multi-instance concurrent logical matching and verification are conducted by combining preset process timing topology constraints with standard operation duration thresholds for each phase. This yields a phased hand disinfection behavior evaluation result data stream that strictly corresponds to the spatial location of the handwashing area, including: Extract the state node sequence from the compliance state vector of each stage, sorted by timestamp in ascending order. Perform directed path matching verification between the state node sequence and the preset process time-series topology constraints to detect whether there are missing stages, reverse jumps, or illegal branches when each handwashing station executes the process, and obtain intermediate data for time-series topology verification. The preset process time-series topology constraints are directed acyclic topology paths, and their fixed directed jump paths are to jump sequentially according to the standard hand disinfection steps. Only forward ordered jumps between adjacent state nodes are allowed, and cross-node jumps, reverse order jumps, and multi-state unordered branch jumps are prohibited. Based on the time span of each continuous state node in the intermediate data of the time-series topology verification, the actual duration of each standard hand disinfection stage is calculated. The actual duration is compared with the preset standard operation duration threshold for each stage. For abnormal periods that exceed the upper limit or fall below the lower limit, threshold truncation filtering is performed to obtain the duration compliance verification results, including the duration standard status and duration deviation quantification value of each stage. Based on the duration compliance verification results, multi-instance concurrent logic matching verification is performed. The topological connectivity status of each handwashing point in the parallel monitoring cycle is cross-fused and weighted with the duration compliance characteristics in the spatiotemporal dimensions to obtain a serialized data packet including the spatial coordinate mapping of the point, the compliance level identifier of each stage and the overall process integrity score. The serialized data packet is then structured and encapsulated according to the video stream synchronization frame rate to obtain a data stream of phased hand disinfection behavior evaluation results that strictly correspond to the spatial location of the handwashing area.

10. A multi-person parallel hand disinfection behavior video data processing system based on region instance segmentation, wherein the system implements the method as described in any one of claims 1 to 9, characterized in that, include: The acquisition module is used to acquire the monitoring video stream of multiple target objects performing hand disinfection operations concurrently, and to perform frame-by-frame feature encoding on the monitoring video stream to obtain the original spatiotemporal feature tensor; The segmentation module is used to perform region instance segmentation processing on the original spatiotemporal feature tensor to obtain an independent instance mask for each handwashing station; the independent instance mask is used to perform spatial feature clipping and adjacent region interference suppression operations on the original spatiotemporal feature tensor to obtain a spatially decoupled instance feature set that is mapped one-to-one with each handwashing station. The compensation module is used to perform temporal trajectory tracking processing on the feature set of spatially decoupled instances and extract the inter-frame displacement sequence of limb key points within the mask coverage area of ​​each independent instance. Based on the inter-frame displacement sequence of limb key points, motion continuity interpolation compensation is performed on trajectory break segments caused by dynamic occlusion. The compensated discrete motion segments are then recombined across frames and bound to identities according to instance identifiers to obtain a temporally isolated behavior sequence chain. The determination module is used to perform phase state mapping processing on the time-series isolated behavior sequence chain, extract the features of each motion segment, and perform multi-dimensional feature similarity matching with the preset discrete state nodes of the standard hand disinfection process. Based on multidimensional feature similarity, the state transition probability threshold is determined, and non-standard deviation actions are filtered by weight decay to obtain the stage compliance state vector corresponding to each handwashing area. The evaluation module is used to perform parallel compliance decision processing on the stage compliance status vector. It combines the preset process timing topology constraints with the standard operation duration threshold of each stage to perform multi-instance concurrent logical matching and verification, and obtains a data stream of staged hand disinfection behavior evaluation results that strictly correspond to the spatial location of the handwashing area.

Images