Newborn hand health monitoring method and system based on machine vision

Abstract

The invention provides a newborn hand health monitoring method and system based on machine vision, and relates to the technical field of image processing, and the method comprises the steps: collecting a dual-mode image of a newborn hand; the dual-mode image is preprocessed; a newborn hand detection model is constructed, and the newborn hand detection model comprises a feature voxel conversion network, an improved feature pyramid network, a target detection network, a dynamic deformation convolutional network and a correction network; inputting the preprocessed dual-mode image into a newborn hand detection model, and outputting a hand candidate frame; taking the hand candidate frame as an ROI region, and extracting texture features in the ROI region through an improved local five-element mode algorithm; on the basis of the texture features, through a pre-trained naive Bayesian model, recognizing the sanitary state of the hand of the newborn; and outputting a newborn hand hygiene monitoring result according to an identification result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for monitoring the hand hygiene of newborns based on machine vision. Background Technology

[0002] Machine vision is a technology that allows computers to "see like human eyes." It acquires images or videos through hardware such as cameras and sensors, and then uses image processing, pattern recognition, and artificial intelligence algorithms to analyze and understand the images, thereby achieving functions such as detection, recognition, measurement, and judgment. A machine vision-based method for monitoring newborn hand hygiene refers to a technique that uses computer vision technology to automatically acquire and analyze images of newborns' hands to determine whether they are clean or contaminated, and then provides hygiene monitoring results.

[0003] Newborns have weak immune systems and are highly susceptible to infections from bacteria and viruses in the external environment. Their hands are the most vulnerable part of their bodies to contact with the outside world and the spread of germs. Therefore, timely and accurate monitoring of the hygiene of newborns' hands is crucial for reducing the risk of hospital-acquired infections and cross-infection.

[0004] However, traditional monitoring methods rely on manual observation and recording. For example, caregivers visually assess whether there is visible contamination on a newborn's hands. This method is not only highly subjective and easily affected by factors such as lighting conditions, differences in caregiver experience, and fatigue, leading to inconsistent and unobjective results, but also often involves intermittent manual checks. This cannot achieve continuous, real-time monitoring of a newborn's hands, which may result in the risk of contamination not being detected and addressed in a timely manner. This increases the probability of cross-infection and nosocomial infection, leading to decreased accuracy and a lack of stability and robustness. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a machine vision-based method for monitoring neonatal hand hygiene. This method can solve the problems of traditional monitoring methods that rely on manual observation and recording, such as nurses visually judging whether there is visible contamination on the newborn's hands. This method is not only highly subjective and easily affected by factors such as lighting conditions, differences in nurses' experience, and fatigue, resulting in inconsistent and unobjective results, but also often involves intermittent manual checks, which increases the probability of cross-infection and nosocomial infection, leading to a decrease in recognition accuracy and a lack of stability and robustness.

[0006] A first aspect of this invention provides a machine vision-based method for monitoring neonatal hand hygiene, comprising: S1: Acquire dual-mode images of the newborn's hand, including infrared and visible light images; S2: Preprocess the dual-mode image; S3: Construct a newborn hand detection model, which includes a feature voxel transformation network, an improved feature pyramid network, an object detection network, a dynamic deformation convolutional network, and a correction network. S4: Input the preprocessed dual-mode image into the newborn hand detection model and output the hand candidate box; S5: Using the hand candidate box as the ROI region, the texture features within the ROI region are extracted through the improved local pentagonal pattern algorithm. S6: Based on texture features, a pre-trained Naive Bayes model is used to identify the hand hygiene status of newborns. S7: Based on the recognition results, output the newborn's hand hygiene monitoring results.

[0007] A second aspect of the present invention provides a newborn hand hygiene monitoring system based on machine vision, comprising: a processor and a memory; The memory stores a program or instructions that can run on a processor, which, when executed by the processor, implement the steps of the machine vision-based neonatal hand hygiene monitoring method of the first aspect.

[0008] A third aspect of the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the machine vision-based neonatal hand hygiene monitoring method of the first aspect.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, by preprocessing the acquired dual-mode images of newborn hands, the original data is ensured to be clear and stable. A newborn hand detection model is constructed, and the preprocessed dual-mode images are input into the newborn hand detection model to output hand candidate boxes, thereby clarifying the ROI region. Through an improved local five-element pattern algorithm, texture features within the ROI region are extracted, quantifying subtle differences in contamination. Finally, based on the texture features, a pre-trained Naive Bayes model is used to identify the hygiene status of newborn hands, achieving continuous and real-time monitoring, reducing the workload of nursing staff, reducing the probability of cross-infection and nosocomial infection, improving the recognition accuracy, and enhancing the scientific nature and consistency of the measurement. Attached Figure Description

[0010] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0011] Figure 1 This is a flowchart illustrating a newborn hand hygiene monitoring method based on machine vision, provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of a newborn hand hygiene monitoring system based on machine vision provided in an embodiment of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] The following description, in conjunction with the accompanying drawings, details the machine vision-based neonatal hand hygiene monitoring method provided by the present invention through specific embodiments and application scenarios.

[0015] Reference manual attached Figure 1 The diagram shows a flowchart of a newborn hand hygiene monitoring method based on machine vision provided by an embodiment of the present invention.

[0016] This invention provides a machine vision-based method for monitoring neonatal hand hygiene, which may include the following steps: S1: Acquire dual-mode images of the newborn's hand, including infrared and visible light images.

[0017] Dual-mode images are images acquired simultaneously using two imaging methods: infrared images and visible light images. Infrared images refer to thermal radiation images obtained using infrared sensors, which are independent of ambient light and can reflect hand contours and temperature differences. Visible light images refer to images within the range of human vision captured using a regular camera, clearly showing the color and texture details of the skin surface.

[0018] It's important to note that the complementarity of infrared and visible light images enhances the completeness of the information. Visible light images accurately reflect the skin color, texture, and surface contamination characteristics of a newborn's hands, but their clarity can decrease in insufficient light or under occlusion. Infrared images, on the other hand, can stably acquire contour and thermal distribution features in dark or complex lighting environments, unaffected by visible light conditions. Combining the two effectively improves the robustness and accuracy of hand detection. Secondly, dual-mode data provides richer feature dimensions for subsequent models, including both appearance details and deep physical information, making subsequent target detection, texture feature extraction, and classification more discriminative. Thirdly, the non-contact optical acquisition method poses no discomfort or infection risk to newborns, ensuring both safety and practicality.

[0019] S2: Preprocess the dual-mode image.

[0020] In one possible implementation, the preprocessing includes: performing median filtering on the infrared image and gamma correction on the visible light image.

[0021] Median filtering replaces the current pixel with the "median level" of surrounding pixels, thus removing extreme outliers (noise) while preserving the main structure. Gamma correction is a non-linear brightness adjustment method that improves the visibility of bright and dark areas by adjusting the grayscale distribution of the image, making details clearer.

[0022] It's important to note that, firstly, median filtering of the infrared images effectively eliminates sensor noise and environmental interference, particularly salt-and-pepper noise. This improves image smoothness while preserving edge and contour details, providing more accurate structural information for subsequent target detection. Secondly, gamma correction of the visible light images improves contrast unevenness caused by insufficient or excessive lighting, making surface contamination features and textures more prominent, thus improving the accuracy of texture feature extraction. Thirdly, after unified preprocessing, the dual-mode images exhibit a more stable data distribution, reducing the impact of external factors such as ambient lighting and device differences, which helps improve the model's robustness and generalization ability across multiple scenarios.

[0023] S3: Construct a newborn hand detection model, which includes a feature voxel transformation network, an improved feature pyramid network, an object detection network, a dynamic deformation convolutional network, and a correction network.

[0024] Among them, the newborn hand detection model refers to an intelligent detection network specifically built for the newborn hand area, which is used to automatically identify and locate the hand position.

[0025] Among them, the voxel transformation network refers to a network structure that transforms image information into high-dimensional feature voxels. It can fuse dual-mode image information and provide a more comprehensive feature representation for subsequent detection.

[0026] Among them, the improved Feature Pyramid Network (FPN) is a structure that fuses feature maps at multiple scales, improving the detection accuracy of small targets (such as details like a newborn's fingers).

[0027] Among them, the object detection network is a deep learning model used to automatically locate and outline regions of interest (hands) in images. Dynamic deformation convolutional networks are a type of structure that improves upon traditional convolutional neural networks (CNNs). They mainly introduce learnable spatial offsets, so that convolution operations are no longer limited to fixed sampling points, but can automatically adjust the sampling position according to the input features to adapt to the geometric deformation and complex poses of the target.

[0028] Among them, the correction network refers to further fine-tuning the detected candidate boxes to make the hand positioning more accurate.

[0029] It's important to note that, firstly, the feature voxel transformation network maps information from dual-mode images (infrared and visible light) into high-dimensional feature voxels. This allows the model to not only capture the texture and color features of the hand but also utilize infrared signals to supplement shape and temperature differences, fundamentally enhancing feature representation capabilities. Secondly, the improved feature pyramid network addresses the issue of imbalanced target features at different scales, enabling the model to consider both the overall hand and detailed areas like fingers, thus ensuring comprehensive and accurate detection. Thirdly, the object detection network rapidly generates candidate regions, while the dynamic deformation convolution adaptively adjusts convolution sampling points to effectively handle complex situations such as varying hand poses, bending, or occlusion, significantly improving the model's adaptability in real-world environments. Finally, the correction network further optimizes the candidate boxes, eliminating detection biases and ensuring precise localization of the hand region.

[0030] S4: Input the preprocessed dual-mode image into the newborn hand detection model and output the hand candidate box.

[0031] Among them, the hand candidate box is a commonly used term in computer vision object detection tasks, referring to the rectangular bounding box (bounding box) that may contain the hand region and is automatically generated by the detection model in the input image.

[0032] It should be noted that by inputting the preprocessed dual-mode image into the detection model and gradually outputting the hand candidate box, automated and refined localization is achieved.

[0033] In one possible implementation, S4 specifically includes: S401: The preprocessed dual-mode image is converted into feature voxels through a feature voxel conversion network.

[0034] S402: Based on the feature voxels, extract multiple multi-scale feature maps of the preprocessed dual-mode image through convolution operations.

[0035] Among them, multi-scale feature maps refer to feature maps extracted at different resolutions, which are used to capture information about the hand at different sizes and levels of detail.

[0036] S403: By using an improved feature pyramid network, the various multi-scale feature maps are fused to obtain a fused feature map.

[0037] In one possible implementation, S403 specifically includes: S4031: Calculate the semantic importance of each multi-scale feature map: in, S ( F i ) indicates the first i Multi-scale feature maps F i The semantic importance of This represents the Sigmoid activation function. Conv ( ) represents a convolution operation. W s Indicates the convolution kernel weights. b s This indicates the convolution bias.

[0038] Semantic importance measures the contribution of a feature map at a certain scale to the task. It is calculated using convolution and the Sigmoid activation function, and its value ranges from 0 to 1.

[0039] S4032: Determine the adaptive weights of each multi-scale feature map based on the semantic importance of each element. in, w i Indicates the first i The adaptive weights of the multi-scale feature maps, where exp() represents the exponential function. S ( F j ) indicates the first j Multi-scale feature maps F j The semantic importance of This represents the temperature coefficient.

[0040] S4033: Based on each adaptive weight, the multi-scale feature maps are fused through upsampling to obtain a fused feature map: in, F merged Represents the fused feature map. U ( ) indicates an upsampling operation, and size indicates the target size.

[0041] Upsampling refers to adjusting feature maps of different scales to a uniform size to facilitate weighted fusion.

[0042] It's important to note that by calculating semantic importance, each scale of feature map receives different weights based on its actual value in the task, highlighting key scales and suppressing redundant information. Secondly, the introduction of a softmax weighting mechanism with a temperature coefficient γ allows for flexible adjustment of the relative differences between features at different scales, achieving better results in various application scenarios. Thirdly, the upsampling operation ensures that feature maps of different resolutions can be fused within a unified spatial dimension, resulting in a fused feature map that retains both low-level detail and high-level semantic information, forming a more comprehensive representation. Overall, this fusion method improves the ability to detect small targets (such as a newborn's finger) and large targets (such as the entire hand), enhancing the discriminativeness and robustness of the feature maps and providing more accurate and stable input for subsequent object detection and candidate box generation.

[0043] S404: Extract the first target candidate box from the multi-scale feature map through the object detection network.

[0044] S405: Based on the first target candidate box and the fused feature map, the second target candidate box is determined through a dynamic deformation convolutional network.

[0045] In one possible implementation, S405 specifically includes: S4051: Calculate the key offsets for each spatial location in the fused feature map, where the key offsets include: position offset and shape offset.

[0046] Among them, position offset refers to the adjustment of the spatial location of the sampling points, so that the convolution kernel is no longer limited to fixed regular grid points. Shape offset refers to the adjustment of the shape of the convolution kernel or the weight distribution, so that the convolution kernel can adapt to different geometric features.

[0047] The specific formula for calculating the position offset is as follows: in, Indicates position offset, The scaling factor is represented by tanh, and the hyperbolic tangent activation function is represented by tanh. xRepresents the fused feature map. W p This represents the weights of the convolution kernel.

[0048] The specific formula for calculating shape offset is as follows: in, Indicates shape offset. Indicates the scaling factor. W s This represents the weights of the convolution kernel.

[0049] S4052: Adjust the spatial position of the convolution kernel based on the key offset.

[0050] S4053: Adjust the weights of the convolution kernel based on the key offset and the scaling factor.

[0051] S4054: Based on the adjusted weights and spatial locations of the convolutional kernels, perform a convolution operation on the fused feature map to generate output feature map values: in, y ( p ) indicates the spatial location of the output feature map. p Pixel value at that location, p n Indicates the reference point offset. R Represents the set of reference points. Indicates the offset. This indicates the weights of the adjusted convolutional kernel. This indicates the spatial location of the adjusted convolution kernel.

[0052] S4055: Based on the output feature map value and the preliminary position information of the first target candidate box, the first target candidate box is corrected to determine the second target candidate box.

[0053] Among them, the second target candidate box is a candidate box determined by the convolution feature extraction result after offset correction, which more accurately marks the position of the hand.

[0054] It's important to note that, firstly, traditional convolutional kernels have fixed sampling positions, making it difficult to handle features of a newborn's hand under different postures, angles, bends, or partial occlusions. The introduction of a position offset Δp allows the convolutional kernel to flexibly sample at more suitable locations, capturing the true contour and structural information of the hand. Secondly, the shape offset Δs dynamically adjusts the convolutional kernel weights, essentially giving the kernel the ability to "adapt to shape," better adapting to local differences and irregular structures. Scaling factors θ and φ ensure that the offset magnitude is moderate, avoiding instability. By combining the offset position and corrected weights, the convolution operation is no longer limited by fixed geometry, resulting in an output feature map that more closely resembles the actual hand features. Finally, the second candidate box generated based on these improved features has more accurate boundaries, reducing detection errors and invalid regions.

[0055] S406: By modifying the network, the second target candidate box is modified to obtain the hand candidate box.

[0056] It's worth noting that the advantage of this process lies in achieving high-precision detection of newborn hands through multi-level, multi-stage optimization. First, a feature voxel transformation network converts the dual-mode image into three-dimensional feature voxels, enriching the representation capabilities. Subsequently, multi-scale feature extraction and an improved feature pyramid network effectively fuse semantic and detailed information at different levels, ensuring the model has stronger robustness to changes in hand size, pose, and lighting. An object detection network initially generates candidate boxes, while a dynamic deformation convolutional network adaptively corrects convolutional sampling by introducing position and shape offsets, improving the model's adaptability to non-rigid deformations of the hand. Finally, a correction network further refines the candidate boxes, making the hand region localization more accurate.

[0057] S5: Using the hand candidate box as the ROI region, the texture features within the ROI region are extracted through an improved local pentagonal pattern algorithm.

[0058] The Region of Interest (ROI) is the area of ​​the newborn's hand image defined by the candidate bounding boxes, serving as the target range for subsequent feature extraction. The improved local pentad pattern algorithm is a more refined and robust texture coding method. It classifies local gray-level differences into five categories and combines local + global thresholds with a unified pattern to improve stability and discriminative power. Texture features are feature vectors obtained based on the statistics of these encoded histograms, used to quantitatively describe the texture differences on the hand surface and serving as the core basis for subsequent hygiene status identification.

[0059] It should be noted that the improved local quinary mode algorithm, compared to the traditional local binary mode (LBP), divides the differences between the neighborhood and the center pixel into five categories, capturing brightness and texture variations in greater detail. Furthermore, by combining local and global thresholds, it can sensitively reflect subtle local differences while maintaining global consistency, thus possessing a strong ability to identify even minor signs of contamination.

[0060] In one possible implementation, S5 specifically includes: S501: Determine the neighboring pixels and center pixel of the ROI region.

[0061] The center pixel serves as the analysis benchmark, while the neighboring pixels form a ring of pixels around it, used to compare grayscale differences.

[0062] S502: Determine the local threshold based on the median absolute deviation of the neighboring pixel set.

[0063] Among them, the median absolute deviation (MAD) is a robust statistic that reflects the degree of fluctuation in the gray level of the neighborhood. It is used as a local threshold and has strong noise resistance.

[0064] S503: Determine the global threshold based on the average saliency value of the center pixel: in, Indicates the global threshold. GSV Indicates the globally significant value. M Indicates the width of the image. N Indicates the height of the image. Indicates the ROI region at pixel ( i , j The degree of local difference at ().

[0065] Among them, the Global Significance Value (GSV) is the average gray-scale contrast of the entire ROI region, which is used as a global threshold to ensure global consistency.

[0066] S504: Combine local and global thresholds to construct a five-value pattern.

[0067] Among them, the five-value mode is divided into five categories based on pixel differences: strong positive, positive, neutral, negative, and strong negative, which is more detailed than the traditional binary mode.

[0068] S505: Divide the five-valued pattern into multiple binary patterns: in, LLQP1 indicates a strong positive mode. c Indicates a quinary mode category label, when c When =2, LLQP A value of 1 indicates that the grayscale difference between neighboring pixels and the center pixel belongs to a strong positive mode. LLQP 2 indicates positive mode, when c =2 or c When =1, LLQP A value of 2 equal to 1 indicates that the grayscale difference between neighboring pixels and the center pixel is in positive mode. LLQP 3 indicates a negative mode, when c =-2 or c When =-1, LLQP A value of 3 equal to 1 indicates that the grayscale difference between neighboring pixels and the center pixel is in negative mode. LLQP 4 indicates a strong negative mode, when c When =-2, LLQP A value of 4 equal to 1 indicates that the grayscale difference between the neighboring pixels and the center pixel belongs to a strong negative mode.

[0069] Among them, the binary mode (LLQP1–LLQP4) is a four-channel decomposition of the five-value mode, representing strong positive, positive, negative, and strong negative respectively.

[0070] In one possible implementation, the binary mode specifically includes: strong positive mode, positive mode, negative mode, and strong negative mode.

[0071] It should be noted that if a newborn's hands are clean, the surface texture is relatively uniform, and the distribution of the four patterns will be relatively balanced with minimal overall difference. If the hands are contaminated (such as with liquids, stains, or residue), noticeable bright or dark spots will form in certain areas, significantly increasing the frequency of the "strong positive pattern" or "strong negative pattern." Contaminants often alter the fineness of skin texture and light reflection characteristics, thus changing the distribution pattern of the statistical histograms for the four patterns. Therefore, these four patterns do not directly represent "clean / dirty," but rather quantify the distribution of differences in texture and brightness.

[0072] S506: Mark each binary pattern using a uniformity method: in, ILQP riu2 Indicates an improved local five-valued model ILQP The eigenvalues ​​of the rotation-invariant unified mode riu2, P Indicates the total number of neighboring pixels. R Represents the neighborhood radius. ch Indicates channel (referring to) LLQP 1- LLQP 4. One of the binary patterns). Indicates the firstp The neighboring pixels in the th order ch The encoding result in a binary pattern No. p +1 neighboring pixels in the... ch The encoding result in a binary pattern Indicates the first p The last neighboring pixel and the first neighboring pixel In the ch Differences in binary patterns This represents the uniformity metric function, specifically the number of binary transitions in the encoding. The number of binary transitions... Less than or equal to the maximum number of transitions allowed by the neighboring pixel encoding threshold When the binary pattern is a uniform pattern category, the number of binary transitions... Greater than the maximum number of transitions allowed by the neighboring pixel encoding threshold At that time, the binary pattern belongs to the non-uniform pattern category, that is, the output is the encoding tag of the non-uniform pattern. P +1.

[0073] S507: Based on the labeling results, multiple rotation-invariant texture histograms are determined by constructing histograms for each uniform binary mode: in, Indicates the first ch The feature histogram vector constructed from each binary pattern F 0 represents the statistical frequency of all non-uniform patterns. F 1 indicates the statistical frequency of the uniform pattern. F P+1 This indicates the statistical frequency of the all-1s or all-0s pattern.

[0074] Among them, rotation-invariant histograms are distribution features formed by statistically analyzing the frequency of different patterns, ensuring that the features are insensitive to rotation changes.

[0075] It should be noted that the uniform binary pattern is the result obtained by labeling each binary pattern in a uniform manner. The uniform binary pattern represents the most common and discriminative patterns in hand texture, which can reduce feature dimensionality and enhance robustness.

[0076] S508: Merge the rotation-invariant texture histograms to generate a feature vector, and use the feature vector as the texture feature: in, This represents the feature vector extracted by the ILQP algorithm, i.e., texture features. Indicates in parameter ( P . R Under the first binary pattern, the rotation-invariant unified pattern histogram (riu2) features are shown. Indicates in parameter ( P . R Under the second binary pattern, the rotation-invariant unified pattern histogram (riu2) features are shown. Indicates in parameter ( P . R Under the third binary pattern, the rotation-invariant unified pattern histogram (riu2) features are shown. Indicates in parameter ( P . R Under the fourth binary pattern, the rotation-invariant unified pattern histogram (riu2) features.

[0077] It's important to note that, firstly, combining local thresholding (MAD) and global thresholding (GSV) allows features to balance local differences and overall consistency, maintaining stability even under varying lighting conditions or noise interference. Secondly, the five-value pattern, compared to the traditional binary pattern, offers more detailed segmentation, capturing subtle staining or minute texture changes and improving sensitivity to hand hygiene. Thirdly, decomposing the five-value pattern into four binary patterns and introducing a unified pattern (riu2) and rotation-invariant histograms effectively reduces dimensionality and redundant information while enhancing the feature's adaptability to directional changes. Finally, the final feature vector obtained by fusing multiple histograms is rich in information and highly discriminative, better supporting accurate judgments by the classifier. Overall, this method achieves high-precision, low-redundancy, and robust representation of newborn hand textures in complex environments, providing reliable input for subsequent hygiene status recognition.

[0078] S6: Based on texture features, a pre-trained Naive Bayes model is used to identify the hand hygiene status of newborns.

[0079] Among them, texture features are statistical feature vectors extracted by an improved local pentatonic pattern algorithm, reflecting the gray-scale differences and texture distribution patterns of the hand surface. The pre-trained Naïve Bayes Classifier is a probabilistic statistical classification model that has been learned on a labeled training dataset, preserving the prior probabilities and conditional probability distributions of various states for the recognition of new data.

[0080] Among them, the hygiene status of newborns' hands is a systematic classification output, usually divided into "contaminated status" and "uncontaminated status", which is the core result of the final monitoring.

[0081] It's important to note that, firstly, it utilizes the texture features extracted in the previous step as input, ensuring that the classification is based on objective, quantified image features rather than subjective human judgment, thus improving reliability. Secondly, the Naive Bayes model, based on probability statistics and assuming independent features, is computationally simple and efficient, making it suitable for real-time applications in medical scenarios. Furthermore, the Naive Bayes model possesses strong generalization ability and stability, performing well even on small datasets, making it particularly suitable for fields like neonatal care where data acquisition is costly and sample sizes are limited. Overall, S6 not only ensures the scientific rigor and efficiency of identification but also provides transparent and traceable decision-making support for the monitoring system, significantly enhancing the practical value of neonatal hand hygiene monitoring.

[0082] In one possible implementation, the hygiene status of a newborn's hands includes: a contaminated state and a non-contaminated state.

[0083] A contaminated state refers to a condition where the surface of the hands contains stains, liquid residue, or bacteria. This condition often indicates a higher risk of infection.

[0084] Among them, the uncontaminated state refers to the clean surface of the hands, without obvious stains, residues or signs of contamination, which is a safe and healthy state.

[0085] It should be noted that this binary classification method is simple and clear, facilitating rapid discrimination during algorithm training and inference, and is also easy for medical staff to understand and use intuitively. Secondly, it meets actual clinical needs: for neonatal care, the most crucial aspect is distinguishing whether there is a potential risk of infection on the hands, and the binary classification results precisely correspond to the two categories of nursing operation scenarios: "requires treatment" and "does not require treatment." Thirdly, it facilitates automation: the simplified output labels reduce the computational complexity of the model, helping to improve the system's real-time performance and stability, enabling the monitoring system to operate efficiently in a real hospital environment.

[0086] In one possible implementation, S6 specifically includes: S601: Input the texture feature vector into the pre-trained Naive Bayes model.

[0087] Among them, the texture feature vector is a set of numerical features extracted by the S5 improved local pentagonal pattern algorithm. It is used to reflect the gray-scale differences and structural distribution of the hand surface texture and is the input of the classification model.

[0088] Among them, the pre-trained Naive Bayes model is a probability-based classifier that has been trained using labeled training data and has learned the distribution patterns and prior probabilities of texture features under contaminated and uncontaminated states.

[0089] S602: Calculate the class conditional log-likelihood for each hand hygiene state in a pre-trained Naive Bayes model.

[0090] Among them, the class-conditional log-likelihood refers to the logarithmic probability of an input feature vector appearing given a certain class (contaminated / uncontaminated), and is used to measure the degree of matching between the feature and the class.

[0091] S603: Calculate the nonnormalized log-posterior probability score for each hand hygiene status based on the class-conditional log-likelihood and prior probability.

[0092] Among them, the prior probability is the probability of occurrence of each category itself learned by the model during the training phase, which is used to reflect the distribution of the category in the overall data.

[0093] Among them, the non-normalized logarithmic posterior probability score is a composite score obtained by adding the prior probability and the class conditional log likelihood. Although it is not normalized, it can be directly compared for classification decision-making.

[0094] S604: Select the hand hygiene status with the highest non-normalized logarithmic posterior probability score as the output of the newborn's hand hygiene status.

[0095] It's important to note that, firstly, inputting texture feature vectors into the Naive Bayes model ensures that the classification is based on objective, quantified image features, rather than subjective human judgment, thus improving scientific rigor and consistency. Secondly, Naive Bayes achieves lightweight probabilistic inference by calculating class-conditional log-likelihood and combining prior probabilities. Its computational process is simple and efficient, suitable for real-time applications, especially in medical monitoring where rapid response is crucial for newborns. Thirdly, using non-normalized log-posterior probability scores as the discrimination criterion not only ensures computational speed but also maintains the transparency and interpretability of the results, allowing medical staff to clearly understand the basis for classification. Furthermore, this process guarantees the uniqueness of the output, explicitly selecting contaminated or uncontaminated states through the "maximum a posteriori probability principle," avoiding ambiguous judgments. Overall, the S6 design tightly integrates texture features with a statistical learning model, ensuring both the efficiency and accuracy of the method while enhancing the credibility and operability of the results, providing a reliable guarantee for intelligent monitoring of newborn hand hygiene.

[0096] S7: Based on the recognition results, output the newborn's hand hygiene monitoring results.

[0097] Among them, the neonatal hand hygiene monitoring results are the monitoring conclusions that the system finally outputs to medical staff or management systems. It is an intuitive presentation of the identification results, usually expressed in the form of labels, prompts or data records, and used in actual clinical operations.

[0098] It's important to note that, firstly, it transforms the identification results into intuitive monitoring outputs, allowing medical staff to directly obtain "contaminated / non-contaminated" conclusions without needing to understand complex image processing or statistical models, significantly improving operational convenience and efficiency. Secondly, the output process can interface with hospital information systems or nursing management platforms, forming a complete data chain that facilitates archiving, statistical analysis, and long-term tracking, providing reliable data support for nursing quality assessment and infection control. Thirdly, real-time output can immediately issue alerts upon detecting contamination, helping nursing staff take timely cleaning or protective measures to reduce the risk of cross-infection and ensure the safety of newborns. Finally, the standardized output format also enhances the system's scalability and portability, enabling it to adapt to different medical environments and application scenarios.

[0099] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, by preprocessing the acquired dual-mode images of newborn hands, the original data is ensured to be clear and stable. A newborn hand detection model is constructed, and the preprocessed dual-mode images are input into the newborn hand detection model to output hand candidate boxes, thereby clarifying the ROI region. Through an improved local five-element pattern algorithm, texture features within the ROI region are extracted, quantifying subtle differences in contamination. Finally, based on the texture features, a pre-trained Naive Bayes model is used to identify the hygiene status of newborn hands, achieving continuous and real-time monitoring, reducing the workload of nursing staff, reducing the probability of cross-infection and nosocomial infection, improving the recognition accuracy, and enhancing the scientific nature and consistency of the measurement.

[0100] Reference manual attached Figure 2 The diagram shows a schematic of a newborn hand hygiene monitoring system based on machine vision provided in an embodiment of the present invention.

[0101] This invention provides a newborn hand hygiene monitoring system 20 based on machine vision, including: a processor 201 and a memory 202; The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-described machine vision-based newborn hand hygiene monitoring method and achieve the same technical effect. To avoid repetition, the present invention will not repeat the above-described steps.

[0102] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0103] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0104] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0105] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0108] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0110] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0111] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] This invention provides a readable storage medium that stores a program or instructions on the medium. When the program or instructions are executed by a processor, they implement the steps of the above-described machine vision-based newborn hand hygiene monitoring method and achieve the same technical effect. To avoid repetition, this invention will not repeat the above description.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for monitoring neonatal hand hygiene based on machine vision, characterized in that, include: S1: Acquire dual-mode images of the newborn's hand, wherein the dual-mode images include infrared images and visible light images; S2: Preprocess the dual-mode image; S3: Construct a newborn hand detection model, wherein the newborn hand detection model includes a feature voxel transformation network, an improved feature pyramid network, an object detection network, a dynamic deformation convolutional network, and a correction network; S4: Input the preprocessed dual-mode image into the newborn hand detection model and output the hand candidate box; S5: Using the candidate hand box as the ROI region, extract the texture features within the ROI region using the improved local pentagonal pattern algorithm; S6: Based on the texture features, the hand hygiene status of newborns is identified using a pre-trained Naive Bayes model; S7: Based on the recognition results, output the newborn's hand hygiene monitoring results.

2. The method for monitoring neonatal hand hygiene based on machine vision according to claim 1, characterized in that, The preprocessing includes: performing median filtering on the infrared image; and performing gamma correction on the visible light image.

3. The method for monitoring neonatal hand hygiene based on machine vision according to claim 1, characterized in that, S4 specifically includes: S401: The preprocessed dual-mode image is converted into feature voxels through the feature voxel conversion network; S402: Based on the feature voxels, extract multiple multi-scale feature maps of the preprocessed dual-mode image through convolution operations; S403: The improved feature pyramid network is used to fuse the various multi-scale feature maps to obtain a fused feature map; S404: Extract the first target candidate box from the multi-scale feature map using the target detection network; S405: Based on the first target candidate box and the fused feature map, a second target candidate box is determined through the dynamic deformation convolutional network; S406: The second target candidate box is corrected through the correction network to obtain the hand candidate box.

4. The method for monitoring neonatal hand hygiene based on machine vision according to claim 3, characterized in that, Specifically, S403 includes: S4031: Calculate the semantic importance of each of the multi-scale feature maps; S4032: Determine the adaptive weights of each of the multi-scale feature maps based on the semantic importance of each of them; S4033: Based on each of the adaptive weights, the multi-scale feature maps are fused through an upsampling operation to obtain the fused feature map.

5. The method for monitoring neonatal hand hygiene based on machine vision according to claim 3, characterized in that, Specifically, S405 includes: S4051: Calculate the key offset for each spatial location in the fused feature map, wherein the key offset includes: position offset and shape offset; S4052: Adjust the spatial position of the convolution kernel according to the key offset; S4053: Adjust the weights of the convolution kernel based on the key offset and the scaling factor; S4054: Based on the adjusted weights and spatial positions of the convolution kernels, perform a convolution operation on the fused feature map to generate output feature map values; S4055: Based on the output feature map value and the preliminary position information of the first target candidate box, the first target candidate box is corrected to determine the second target candidate box.

6. The method for monitoring neonatal hand hygiene based on machine vision according to claim 1, characterized in that, S5 specifically includes: S501: Determine the neighboring pixels and the center pixel of the ROI region; S502: Determine the local threshold based on the median absolute deviation of the neighboring pixel set; S503: Determine the global threshold based on the average saliency value of the center pixel; S504: Combine the local threshold and the global threshold to construct a five-value pattern; S505: Divide the five-valued pattern into multiple binary patterns; S506: Each of the binary patterns is marked using a uniformity method; S507: Based on the labeling results, multiple rotation-invariant texture histograms are determined by constructing histograms for each uniform binary mode; S508: The rotation-invariant texture histograms are fused to generate a feature vector, and the feature vector is used as the texture feature.

7. The method for monitoring neonatal hand hygiene based on machine vision according to claim 6, characterized in that, The binary modes specifically include: strong positive mode, positive mode, negative mode, and strong negative mode.

8. The method for monitoring neonatal hand hygiene based on machine vision according to claim 1, characterized in that, The hygiene status of the newborn's hands includes: contaminated and uncontaminated.

9. The method for monitoring neonatal hand hygiene based on machine vision according to claim 1, characterized in that, S6 specifically includes: S601: Input the texture feature vector into the pre-trained Naive Bayes model; S602: In the pre-trained Naive Bayes model, calculate the class conditional log-likelihood for each hand hygiene state; S603: Calculate the nonnormalized log-posterior probability score of each of the hand hygiene states based on the class-conditional log-likelihood and prior probability. S604: Select the hand hygiene status with the highest non-normalized logarithmic posterior probability score as the output of the newborn's hand hygiene status.

10. A machine vision-based neonatal hand hygiene monitoring system, characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the machine vision-based neonatal hand hygiene monitoring method as described in any one of claims 1 to 9.

Images