Hand contamination assessment method and system fusing multi-modal images, and storage medium

Abstract

The application relates to the field of health monitoring, and discloses a hand pollution evaluation method and system fusing multi-modal images and a storage medium. The method comprises the following steps: synchronously collecting hand visible light, fluorescence and thermal imaging data and carrying out denoising processing; extracting a hand contour and a fluorescence intensity distribution, and generating an initial pollution distribution feature map; calculating a pollution concentration gradient and classifying the pollution intensity of a hand region; spatially registering and weighting fusing the thermal imaging data and the fluorescence intensity, and obtaining a temperature-weighted pollution distribution map; correcting and normalizing the high-temperature area pollution proportion; combining the pollution degree distribution result to carry out temperature-dependent diffusion simulation and spatial mapping, and generating a hand pollution comprehensive distribution map; and carrying out regional comparative analysis based on the map and generating an evaluation report. The application solves the problems of insufficient accuracy and inability to reflect dynamic risks when a single imaging mode is used to evaluate hand pollution, and improves the comprehensiveness and accuracy of evaluation.

Description

Technical Field

[0001] This application relates to the field of health monitoring, and in particular to a method, system, and storage medium for assessing hand contamination by integrating multimodal images. Background Technology

[0002] Hand hygiene assessment is a crucial step in preventing hospital-acquired infections and foodborne illnesses, especially in industries with high hygiene standards such as healthcare and food processing. This field primarily utilizes non-contact imaging technology to objectively assess hand cleanliness. Common technical approaches include using visible light imaging to analyze hand morphology and contaminant residues, or using fluorescence imaging to excite and capture microbial marker signals on the hand surface, thereby achieving a quantitative assessment of handwashing effectiveness.

[0003] Existing assessment methods suffer from three main limitations. First, reliance on a single imaging modality leads to insufficient information dimensions. For example, methods based solely on visible light images can identify hand contours and visible dirt, but cannot detect microbial contamination. Methods relying solely on fluorescence imaging are susceptible to interference from background fluorescence, hand creases and shadows, and uneven application of fluorescent agents, and struggle to distinguish between live microorganisms and non-specific fluorescent substances. Second, these methods neglect the combined influence of key physiological and environmental parameters. Hand surface temperature distribution varies significantly; high-temperature areas (such as the palm and nail groove) may enhance microbial activity and promote contamination migration. However, existing methods do not incorporate thermal imaging data into their analysis, failing to correct for the potential impact of temperature on fluorescence signal intensity and contamination diffusion behavior, potentially causing assessment results to deviate from the true risk. Finally, the assessment results are static, lacking dynamic risk warning capabilities. Most methods only output the current contamination distribution, failing to incorporate temperature field simulations of potential diffusion trends in hidden areas such as between fingers and wrists, thus failing to provide forward-looking guidance for continuous hygiene monitoring and preventative cleaning. The above-mentioned shortcomings collectively limit the accuracy, robustness, and practical application value of existing technologies in complex real-world scenarios.

[0004] To address the above deficiencies, this application solves the problems of one-sided assessment, neglect of temperature interference, and lack of dynamic early warning by simultaneously acquiring and fusing multimodal data of visible light, fluorescence, and thermal imaging, combined with temperature-weighted correction mechanism and temperature-dependent diffusion dynamic simulation, thereby improving the comprehensiveness, accuracy, and risk prediction capability of hand contamination assessment. Summary of the Invention

[0005] This application provides a method, system, and storage medium for assessing hand contamination by fusing multimodal images. It solves the problems of single-modal assessment being one-sided, ignoring temperature interference, and lacking dynamic early warning, thereby improving the comprehensiveness, accuracy, and risk prediction capabilities of hand contamination assessment.

[0006] In a first aspect, this application provides a method for assessing hand contamination by fusing multimodal images, the method comprising:

[0007] Step S101: Simultaneously acquire visible light images, microbial fluorescence signals, and thermal imaging data of the hand surface to form a raw multimodal dataset, and obtain hand multimodal data after denoising processing;

[0008] Step S102: Extract hand contour features and fluorescence signal intensity distribution from the hand multimodal data, refine the hand contour features, and obtain an initial hand contamination distribution feature map;

[0009] Step S103: Calculate the pollution concentration gradient based on the initial hand pollution distribution feature map, and evaluate the pollution intensity of different hand areas by combining the fluorescence signal intensity distribution to determine the pollution degree distribution result;

[0010] Step S104: Based on the pollution degree distribution results, perform spatial registration of thermal imaging data and calibration of fluorescence signal intensity, then perform superposition processing, and obtain a temperature-weighted hand pollution distribution map by calculating the temperature field distribution weight.

[0011] Step S105: Determine whether the proportion of pollution in the high-temperature region in the temperature-weighted hand pollution distribution map exceeds the preset proportion threshold. If so, reduce the pollution intensity weight of the high-temperature region by the temperature correction coefficient and normalize the fluorescence signal intensity to determine the corrected pollution distribution feature map.

[0012] Step S106: Based on the corrected pollution distribution feature map and the pollution degree distribution results, perform temperature-dependent diffusion simulation and concentration distribution spatial mapping to generate a comprehensive hand pollution distribution map;

[0013] Step S107: Based on the comprehensive distribution map of hand contamination, conduct a comparative analysis of contamination between regions to determine the hygiene status of specific areas of the hands and generate a targeted assessment report.

[0014] Secondly, this application provides a hand contamination assessment system that integrates multimodal images, used to implement the aforementioned hand contamination assessment method integrating multimodal images, the system comprising:

[0015] The data acquisition module is used to simultaneously acquire visible light images, microbial fluorescence signals and thermal imaging data of the hand surface to form a raw multimodal dataset, which is then processed to obtain hand multimodal data.

[0016] The contour segmentation module is used to extract hand contour features and fluorescence signal intensity distribution from the hand multimodal data, and to finely segment the hand contour features to obtain an initial hand contamination distribution feature map.

[0017] The pollution grading module is used to calculate the pollution concentration gradient based on the initial hand pollution distribution feature map, and to grade and evaluate the pollution intensity of different hand areas in combination with the fluorescence signal intensity distribution to determine the pollution degree distribution result;

[0018] The weighted fusion module is used to perform spatial registration of thermal imaging data and calibration of fluorescence signal intensity based on the pollution degree distribution results, and then perform superposition processing. By calculating the temperature field distribution weight, a temperature-weighted hand pollution distribution map is obtained.

[0019] The correction and optimization module is used to determine whether the proportion of pollution in the high-temperature area in the temperature-weighted hand pollution distribution map exceeds a preset proportion threshold. If so, the pollution intensity weight in the high-temperature area is reduced by the temperature correction coefficient, and the fluorescence signal intensity is normalized to determine the corrected pollution distribution feature map.

[0020] The visual presentation module is used to perform temperature-dependent diffusion simulation and concentration distribution spatial mapping based on the corrected pollution distribution feature map, and generate a comprehensive distribution map of hand pollution.

[0021] The report generation module is used to conduct inter-regional pollution comparison analysis based on the comprehensive distribution map of hand contamination, determine the hygiene status of specific areas of the hands, and generate a targeted assessment report.

[0022] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when run by a processor, causes the processor to execute the hand contamination assessment method fused with multimodal images.

[0023] This application proposes a method, system, and storage medium for hand contamination assessment that integrates multimodal images. It solves the problems of single-modal assessment being one-sided, ignoring temperature interference, and lacking dynamic early warning, thereby improving the comprehensiveness, accuracy, and risk prediction capability of hand contamination assessment. Compared with existing technologies, the beneficial effects of this application's technical solution are at least as follows:

[0024] First, by simultaneously acquiring and fusing data from three modalities—visible light, microbial fluorescence, and thermal imaging—the limitations of a single imaging modality were overcome. This enabled multi-dimensional information complementarity, from hand contours and biological contamination signals to temperature fields, providing a comprehensive data foundation for a full assessment.

[0025] Second, the pollution concentration gradient is calculated based on the initial hand pollution distribution feature map, and the classification is carried out by combining the hand anatomy and regional health risk weights. This realizes the quantitative mapping from fluorescence intensity to actual health risk, making the pollution classification more consistent with the actual situation of different areas of the hand.

[0026] Third, the thermal imaging data and the pollution distribution results were spatially registered and the fluorescence signal intensity was calibrated. Through temperature field weighted fusion, temperature was systematically incorporated into the evaluation model as a key variable affecting the activity and distribution of pollutants for the first time. This made the generated pollution distribution map more consistent with the actual physiological conditions and reduced misjudgments caused by temperature differences.

[0027] Fourth, by analyzing the pollution ratio in high-temperature areas and dynamically adjusting the pollution weight using a temperature correction coefficient, it is possible to effectively identify and correct abnormal enhancement of fluorescence signals caused by local high temperatures (such as areas with active blood circulation), thus avoiding false positive or false negative pollution assessments and improving the system's anti-interference ability and robustness.

[0028] Fifth, based on the corrected pollution distribution and pollution level results, temperature-dependent diffusion simulation and spatial mapping were performed, enabling dynamic prediction and visualization of potential pollution diffusion paths. This allows the assessment results to not only reflect the current state but also provide early warnings of future risks, enhancing the foresight and guidance of the assessment. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the hand contamination assessment method that integrates multimodal images in this application;

[0031] Figure 2 This is a performance comparison chart for this application;

[0032] Figure 3 The bar chart shows the performance improvement of each indicator in this application;

[0033] Figure 4 This is a schematic diagram of the hand contamination assessment system that integrates multimodal images in this application. Detailed Implementation

[0034] This application provides a method, system, and storage medium for assessing hand contamination by fusing multimodal images. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 An embodiment of a hand contamination assessment method that integrates multimodal images, as described in this application, includes:

[0036] Step S101: Simultaneously acquire visible light images, microbial fluorescence signals, and thermal imaging data of the hand surface to form a raw multimodal dataset. After denoising, obtain hand multimodal data.

[0037] In one specific embodiment, step S101 includes the following steps:

[0038] Visible light images, microbial fluorescence signals, and thermal imaging data of the hand surface are acquired simultaneously using visible light sensors, fluorescence imaging devices, and thermal imaging devices.

[0039] The acquired visible light images, microbial fluorescence signals, and thermal imaging data are time-stamped to generate the original multimodal dataset;

[0040] For the original multimodal dataset, median filtering was applied to the visible light images, threshold segmentation was applied to the microbial fluorescence signals, and Gaussian filtering was applied to the thermal imaging data to obtain hand multimodal data.

[0041] Specifically, in the field of hygiene monitoring, especially in high-hygiene-standard scenarios such as medical and food processing, to address the technical problems of insufficient information dimensions, neglect of temperature effects, and lack of dynamic early warning in single imaging modes, it is necessary to simultaneously collect relevant data on the hand surface using visible light sensors, fluorescence imaging devices, and thermal imaging devices. For example, a visible light sensor captures a 1920×1080 resolution visible light image of the hand surface at a frame rate of 30 frames per second for subsequent extraction of hand contour features; a fluorescence imaging device emits excitation light with a wavelength of 365 nm to induce fluorescence signals from microorganisms on the hand surface, and the collected fluorescence signal intensity ranges from 0 to 255 grayscale values ​​to reflect the distribution of microbial contamination; a thermal imaging device records the hand temperature distribution at the same frame rate, with a temperature measurement range of 20–40°C, providing data support for analyzing the impact of temperature on contamination diffusion. During the acquisition process, a shared clock signal is used to mark the data collected by the three types of devices with a unified time stamp, ensuring that the timestamp deviation is less than 10 milliseconds, achieving synchronous data acquisition, and then integrating them to form a raw multimodal dataset containing visible light images, microbial fluorescence signals, and thermal imaging data.

[0042] For the original multimodal dataset, corresponding denoising methods need to be adopted according to the characteristics of different data types and subsequent application requirements. For visible light images, which are susceptible to random particle noise in the environment, affecting the accuracy of hand contour extraction, a median filtering algorithm with a 3×3 pixel window is used. This algorithm effectively removes random noise by replacing the center pixel value with the median grayscale value of the pixels within the window, while preserving hand edge details. After processing, the signal-to-noise ratio of the visible light image is improved to over 20dB, laying the foundation for accurate hand contour recognition by subsequent edge detection algorithms. For microbial fluorescence signals, which are easily interfered with by background fluorescence, making it difficult to accurately distinguish microbial contamination areas, threshold segmentation is used. A threshold is calculated based on the dataset histogram, retaining pixels with a signal intensity greater than 50 as valid signals and eliminating background interference signals. This makes the fluorescence signal in microbial contamination areas purer, ensuring the accuracy of subsequent contamination intensity quantification. For thermal imaging data, temperature noise can cause uneven temperature distribution, affecting the calculation of temperature field weights. Therefore, Gaussian filtering is used, with a Gaussian kernel standard deviation of 1.2. By performing convolution operations on the thermal imaging data, temperature noise is smoothed, making the hand temperature distribution more consistent with actual physiological conditions and ensuring the reliability of temperature field distribution information. After the above processing, the visible light image, microbial fluorescence signal, and thermal imaging data are respectively transformed into a denoised clear image, clean fluorescence signal data, and smoothed temperature data, which together constitute multimodal hand data.

[0043] Step S102: Extract hand contour features and fluorescence signal intensity distribution from hand multimodal data, refine the hand contour features, and obtain an initial hand contamination distribution feature map.

[0044] In one specific embodiment, step S102 includes the following steps:

[0045] The visible light image in the multimodal data of the hand is processed by an edge detection algorithm to identify the edge lines of the hand and form the contour features of the hand.

[0046] Intensity mapping was performed on microbial fluorescence signal data in hand multimodal data to quantify the fluorescence value distribution of each pixel and obtain the fluorescence signal intensity distribution;

[0047] Calculate the gradients of the hand contour features in the horizontal and vertical directions to obtain the gradient magnitude and direction of each contour point;

[0048] If the gradient magnitude of any contour point exceeds the preset gradient threshold, the contour point is marked as a boundary inflection point, and the boundary sub-region of the contour is divided based on the boundary inflection point.

[0049] The fluorescence signal intensity distribution is mapped to the defined boundary sub-regions to generate an initial hand contamination distribution feature map.

[0050] Specifically, when processing visible light images in the hand multimodal data, the Canny edge detection algorithm is used to extract hand contour features. This algorithm sets the Gaussian filter kernel size to 5×5, the high threshold to 80, and the low threshold to 40. First, the visible light image is smoothed to reduce environmental noise interference. Then, edge pixels are identified by calculating the image gradient. Finally, non-maximum suppression and dual threshold filtering are applied to obtain continuous and complete hand edge lines, forming hand contour features. For microbial fluorescence signal data in the hand multimodal data, a linear intensity mapping method is used for quantization, mapping the original fluorescence signal intensity range [0,255] to the [0,1] interval, using the formula:

[0051]

[0052] Calculate the quantized fluorescence value of each pixel. ,in The original fluorescence signal intensity, This represents the minimum intensity value of the fluorescence signal. The maximum intensity value of the fluorescence signal is used to obtain the fluorescence signal intensity distribution through this quantification process, clarify the strength of microbial contamination signals at different pixels, and provide quantitative data for contamination distribution mapping.

[0053] To achieve fine segmentation of the hand contour, it is necessary to calculate the gradients of the hand contour features in the horizontal and vertical directions. The horizontal and vertical gradients are calculated using the Sobel operator. and The convolution operation yields, for example:

[0054]

[0055] Gradient magnitude at each contour point gradient direction .

[0056] A preset gradient threshold of 30 is used. When the gradient amplitude of any contour point exceeds this threshold, it indicates that the point is a boundary inflection point where the contour direction changes significantly. Using these boundary inflection points as segmentation nodes, the hand contour is divided into boundary sub-regions such as fingertips, finger pads, finger gaps, palms, backs of hands, and wrists. Each sub-region corresponds to a specific part of the hand's anatomical structure, ensuring that the region division matches the actual health risk assessment needs. When mapping the quantified fluorescence signal intensity distribution to the divided boundary sub-regions, a pixel coordinate matching mechanism is used to establish a spatial coordinate correspondence between fluorescence signal data and visible light images. All pixels within each boundary sub-region correspond to a unique quantified fluorescence value. By presenting the fluorescence signal intensity of each sub-region in grayscale form, an initial hand contamination distribution feature map is generated.

[0057] In this feature map, the gray levels of different boundary sub-regions directly reflect the intensity of microbial contamination signals in the corresponding regions, achieving a precise combination of hand contours and contamination signals. This provides a structured data foundation for subsequent calculation of contamination concentration gradients and graded assessments. At the same time, the fine contour division reduces the interference of hand wrinkles and shadows on fluorescence signal recognition, improving the accuracy of contamination distribution identification and solving the technical problem that simple fluorescence imaging is easily interfered with and cannot accurately locate contamination areas.

[0058] Step S103: Calculate the pollution concentration gradient based on the initial hand pollution distribution characteristic map, and classify and evaluate the pollution intensity of different hand areas in combination with the fluorescence signal intensity distribution to determine the pollution degree distribution results.

[0059] In one specific embodiment, step S103 includes the following steps:

[0060] Based on the initial hand contamination distribution feature map, the gradient values ​​of fluorescence signal intensity in the horizontal and vertical directions of each pixel are calculated respectively;

[0061] The pollution concentration gradient of each pixel is calculated based on the gradient values ​​in the horizontal and vertical directions, forming a gradient distribution matrix;

[0062] The hand is divided into different regions based on anatomical structure. Fluorescence signal intensity data of each region in the gradient distribution matrix are extracted. The mean and variance of fluorescence signal intensity in each region are calculated to obtain the pollution quantification index of each region.

[0063] The pollution quantification indicators of each region are compared with the preset pollution intensity classification standards. Combined with the health risk weight of different areas of the hands, the pollution intensity of each region is divided into three levels: low, medium and high.

[0064] By integrating the pollution intensity classification results of various regions, a pollution degree distribution result is generated.

[0065] Specifically, based on the initial hand contamination distribution feature map, the Sobel operator is used to calculate the gradient values ​​in the horizontal and vertical directions for the fluorescence signal intensity of each pixel in the map. For example, the Sobel operator in the horizontal direction is set to... The vertical Sobel operator is set as By convolving the operator with a 3×3 pixel neighborhood in the image, the gradient value of each pixel in the horizontal direction is obtained. and the gradient value in the vertical direction During convolution, the spatial coordinates of the pixels remain unchanged, ensuring that the gradient value corresponds precisely to the pixel position. The magnitude of the gradient value directly reflects the rate of change of the fluorescence signal intensity in that direction, providing basic data for subsequent calculation of pollution concentration gradient.

[0066] Combining the gradient values ​​in the horizontal and vertical directions, using the formula... Calculate the contamination concentration gradient of each pixel. This formula integrates two-dimensional gradient changes into a single quantized value through vector synthesis, intuitively reflecting the spatial variation of contamination concentration around a pixel. The contamination concentration gradients of all pixels are arranged sequentially according to their row and column coordinates in the initial hand contamination distribution feature map, forming a gradient distribution matrix. The number of rows and columns in the matrix matches the pixel resolution of the initial hand contamination distribution feature map. Each element in the matrix corresponds to the contamination concentration gradient of a pixel in the image, achieving structured storage and spatial localization of the contamination concentration gradient.

[0067] Based on the anatomical structure of the hand, the hand is divided into seven functional regions: fingertips, finger pads, finger gaps, palm, nail groove, back of hand, and wrist. The boundaries of each region are determined according to the distribution characteristics of human hand bones and muscles, ensuring that the region division conforms to the actual physiological structure and cleaning scenario requirements. A subset of pixels corresponding to each region is extracted from the gradient distribution matrix. Simultaneously, the raw fluorescence signal intensity data of the same pixel subset is extracted from the fluorescence signal intensity distribution obtained in step S102. The mean and variance of the fluorescence signal intensity data for each region are calculated. The mean reflects the overall pollution signal level of the region, and the variance reflects the uniformity of pollution distribution within the region. Together, they constitute the quantitative pollution index for each region, achieving a quantitative description of the pollution level.

[0068] A pre-defined pollution intensity grading standard is established, with low pollution levels corresponding to a mean range of [0, 0.4] and a variance range of [0, 0.08], medium pollution levels to [0.4, 0.8] and a variance range of [0.08, 0.2], and high pollution levels to [0.8, 1.0] and a variance range of [0.2, 1.0]. This standard is based on the hygiene and safety requirements of the medical and food processing industries, ensuring that the grading results conform to actual application scenarios. Simultaneously, the hygiene risk weights for different areas of the hand are incorporated: the weight for finger crevices and nail grooves is set at 0.9, the weight for fingertips and palms at 0.7, the weight for fingertips and wrists at 0.5, and the weight for the back of the hand at 0.3. The weight allocation is determined based on the probability of contact with contaminants and the difficulty of cleaning in each area. The pollution quantification indicators for each area are compared with the pre-defined grading standard using a formula... Calculate the comprehensive pollution score (in, The mean value of fluorescence signal intensity data for each region, The variance of fluorescence signal intensity data for each region, (As a regional health risk weight), based on the comprehensive pollution score, when When classified as low pollution intensity, The pollution level is divided into medium and high intensity. The time is divided into high pollution intensity to achieve precise classification based on regional risk differences.

[0069] The pollution intensity classification results of the seven regions were integrated according to their spatial location in the hand images. Coordinate mapping technology was used to mark the pollution level of each region onto the corresponding hand contour area, generating a pollution degree distribution result. This result is presented in image form, with regions of different pollution intensities identified by different colors. It also includes quantitative pollution indicators (mean, variance) and a comprehensive pollution score for each region, providing clear pollution levels and quantitative basis for subsequent thermal imaging data fusion and pollution correction. This addresses the problems of existing pollution assessment technologies lacking quantitative standards and ignoring regional risk differences, leading to assessment results that deviate from reality.

[0070] Step S104: Based on the pollution degree distribution results, perform spatial registration of thermal imaging data and calibration of fluorescence signal intensity, then perform superposition processing, and obtain a temperature-weighted hand pollution distribution map by calculating the temperature field distribution weight.

[0071] In one specific embodiment, performing step S104 includes the following steps:

[0072] Establish a spatial coordinate mapping relationship between thermal imaging data and pollution level distribution results, and perform spatial registration of thermal imaging data through spatial transformation to achieve spatial registration of the hand area in thermal imaging data and pollution level distribution results;

[0073] Based on the pollution intensity classification of each region in the pollution degree distribution results, the fluorescence signal intensity calibration coefficient of the corresponding region is determined, and the fluorescence signal intensity of the corresponding region is calibrated using the calibration coefficient;

[0074] The spatially registered thermal imaging data and the calibrated fluorescence signal intensity data are superimposed at the pixel level to generate a preliminary fused image.

[0075] Temperature field distribution information is extracted from spatially registered thermal imaging data, and temperature values ​​are converted into corresponding weight coefficients through a weight mapping function.

[0076] By multiplying the fluorescence signal intensity value of each pixel in the preliminary fused image with the corresponding weight coefficient, and integrating the calculation results of all pixels, a temperature-weighted hand contamination distribution map is obtained.

[0077] Specifically, using the pollution level distribution results as a spatial reference, which clearly defines the pollution intensity classification and spatial coordinate range of each area of ​​the hand, the thermal imaging data contains hand temperature distribution information, but the spatial coordinates may deviate from the pollution level distribution results. Therefore, a spatial coordinate mapping relationship needs to be established between the two. Spatial registration is achieved using affine transformation. By selecting five feature points (anatomical features such as fingertips, palm center, and wrist midpoint) from both the pollution level distribution results and the thermal imaging data, the coordinate information of these feature points in both sets of data is obtained. Based on these coordinates, the affine transformation matrix T is calculated, where:

[0078]

[0079] in, For scaling and rotation parameters, As translation parameters, the coordinate error of the transformed feature points is minimized using the least squares method, ensuring the error is less than one pixel. The coordinates of each pixel in the thermal imaging data are then... Substitute into the transformation formula Obtain the transformed coordinates This allows the hand region in the thermal imaging data to be perfectly aligned spatially with the hand region in the pollution level distribution results, achieving spatial registration between the thermal imaging data and the pollution level distribution results, and providing a spatial consistency basis for subsequent multi-data fusion.

[0080] Based on the pollution intensity classification (low, medium, high) of each region in the pollution degree distribution results, the corresponding fluorescence signal intensity calibration coefficients are determined. The calibration coefficient for low pollution intensity regions is set to 1.1, for medium pollution intensity regions to 1.0, and for high pollution intensity regions to 0.9. This coefficient setting is determined based on the attenuation characteristics of fluorescence signals and the degree of background interference under different pollution intensities. The higher the pollution intensity, the greater the interference of its own concentration and background noise on the fluorescence signal, requiring an appropriate reduction in the calibration coefficient to correct the deviation. Fluorescence signal intensity data for each region are extracted from the fluorescence signal intensity distribution obtained in step S102 and processed using the formula... Calculate the calibrated fluorescence signal intensity ,in The original fluorescence signal intensity, This serves as the calibration coefficient for the corresponding region, enabling precise calibration of fluorescence signal intensity in regions with different pollution levels, thereby reducing the impact of signal interference on subsequent fusion results.

[0081] The spatially registered thermal imaging data and the calibrated fluorescence signal intensity data are superimposed at the pixel level. The thermal imaging data represents temperature information in grayscale values ​​(grayscale range 0~255), and the calibrated fluorescence signal intensity data is also mapped to the 0~255 grayscale range. During the superposition process, a one-to-one correspondence between pixel coordinates is used to match the grayscale values ​​of the thermal imaging data at the same coordinate. Compared with the calibrated fluorescence signal intensity gray value Through formula Calculate the pixel grayscale values ​​after overlay The weighting is determined based on the importance of temperature information and pollution signals in the assessment, and a preliminary fusion image is generated. This image contains both temperature distribution and pollution signal information, thus achieving the initial integration of the two types of data.

[0082] Temperature field distribution information is extracted from spatially registered thermal imaging data, and the temperature value of each pixel is extracted. The temperature range is 20~40℃, and the result is obtained through a weighted mapping function. Convert temperature values ​​into corresponding weighting coefficients. ,in Lowest hand temperature The function, representing the highest hand temperature, establishes a positive correlation between temperature and weighting coefficients; higher temperatures result in larger weighting coefficients (range 0.2–1.0). This aligns with the physiological characteristics of higher temperature regions, where microbial activity is stronger and the risk of contamination is higher. The influence of temperature on contamination risk is quantified into weighting coefficients. The fluorescence signal intensity values ​​of each pixel in the initial fusion image are then used. With corresponding weight coefficients Through formula Calculate the weighted pixel values The process involves iterating through all pixels in the initial fusion image, integrating the weighted calculation results of all pixels, and forming a temperature-weighted hand contamination distribution map.

[0083] In the hand contamination distribution map, the gray value of each pixel contains both the calibrated fluorescence signal intensity information (reflecting the degree of contamination) and the weighting coefficient corresponding to temperature (reflecting the influence of temperature on contamination), realizing the deep fusion of contamination signal and temperature information. This solves the problem that existing technologies ignore the influence of temperature, causing the assessment results to deviate from the true risk, and makes the generated contamination distribution map more consistent with the actual contamination status and physiological characteristics of the hands.

[0084] Step S105: Determine whether the proportion of pollution in the high-temperature area in the temperature-weighted hand pollution distribution map exceeds the preset proportion threshold. If so, reduce the pollution intensity weight of the high-temperature area by using the temperature correction coefficient, and normalize the fluorescence signal intensity to determine the corrected pollution distribution feature map.

[0085] In one specific embodiment, step S105 includes the following steps:

[0086] Identify high-temperature regions in the temperature-weighted hand contamination distribution map and calculate the coverage ratio of contamination signals within these regions.

[0087] Determine whether the pollution coverage ratio exceeds the preset ratio threshold. If so, extract the temperature value of each pixel in the high temperature area and calculate the temperature difference with the average temperature of the hand. Construct a temperature gradient distribution based on the temperature difference.

[0088] The gradient interval is set according to the temperature gradient distribution, and a corresponding correction coefficient reference value is assigned to each gradient interval. The temperature correction coefficient of each pixel is calculated by linear interpolation.

[0089] Reduce the pollution intensity weight of high-temperature areas based on the temperature correction factor;

[0090] Fluorescence signal intensity data were extracted from the temperature-weighted hand contamination distribution map and normalized.

[0091] By combining the adjusted pollution intensity weights with the normalized fluorescence signal intensity data, the pollution distribution presentation is optimized, and the corrected pollution distribution feature map is determined.

[0092] Specifically, the temperature-weighted hand contamination distribution map integrates fluorescence signal intensity and temperature weighting information. First, the temperature values ​​of corresponding pixels are extracted from the spatially registered thermal imaging data. A high-temperature threshold of 37℃ is set, and all pixels in the temperature-weighted hand contamination distribution map with a temperature value reaching or exceeding 37℃ are identified. These pixels collectively constitute high-temperature regions, and the total number of pixels in these regions is counted. Simultaneously, based on the fluorescence signal intensity data in the temperature-weighted hand contamination distribution map, a fluorescence signal intensity threshold of 150 (corresponding to a grayscale range of 0-255) is set. The number of contaminated pixels with a fluorescence signal intensity reaching or exceeding 150 within the high-temperature regions is counted. The coverage ratio of contaminated signals within the high-temperature regions is calculated by the ratio of the number of contaminated pixels to the total number of pixels in the high-temperature regions. This ratio directly reflects the distribution density of contamination signals in the high-temperature regions, providing a quantitative basis for subsequent threshold determination.

[0093] The preset contamination ratio threshold is 0.6. This threshold is set based on the normal contamination distribution pattern in high-temperature areas of the hand in scenarios such as medical treatment and food processing. When the calculated contamination coverage ratio exceeds 0.6, it indicates that the contamination signal coverage in the high-temperature area is too wide, which may lead to abnormal enhancement of the fluorescence signal due to temperature rise, requiring correction processing. The temperature value of each pixel in the high-temperature area is extracted, and the temperature difference is obtained by subtracting the average hand temperature from the temperature value of each pixel. The average hand temperature is obtained by calculating the arithmetic mean of the temperature values ​​of all pixels in the hand. A temperature gradient distribution is constructed based on the temperature difference of all pixels. This distribution visually presents the degree of temperature deviation of each pixel in the high-temperature area, providing a basis for calculating the temperature correction coefficient.

[0094] Three gradient intervals were set based on the temperature gradient distribution: 0℃ to 1℃, 1℃ to 2℃, and 2℃ and above. A corresponding correction coefficient baseline value was assigned to each gradient interval, which was 0.9, 0.8, and 0.7 respectively. The larger the temperature difference, the higher the possibility of temperature interference with the fluorescence signal, and the lower the correction coefficient baseline value. A linear interpolation method was used to calculate the temperature correction coefficient for each pixel. For a temperature difference within a certain gradient interval, based on its specific position within the interval and the correction coefficient baseline values ​​corresponding to the upper and lower limits of the interval, a linear calculation was performed to obtain the temperature correction coefficient for that pixel. This ensured a continuous linear correlation between the temperature correction coefficient of each pixel and the temperature difference, achieving accurate correction.

[0095] The pollution intensity weight in high-temperature areas is reduced by a temperature correction coefficient. The original pollution intensity weight of each pixel in the temperature-weighted hand pollution distribution map has been calculated in step S104. The original pollution intensity weight is multiplied by the temperature correction coefficient of the corresponding pixel to obtain the adjusted pollution intensity weight. The pollution intensity weight of pixels in high-temperature areas decreases synchronously with the temperature correction coefficient, effectively offsetting the abnormal enhancement of pollution signals caused by high temperatures, making the pollution intensity weight more consistent with the actual pollution situation. Fluorescence signal intensity data in the temperature-weighted hand pollution distribution map is extracted. This data ranges from 0 to 255 and is normalized to map all fluorescence signal intensity values ​​to a unified range of 0 to 1, eliminating the magnitude difference in fluorescence signal intensity between different pixels and providing a unified data scale for subsequent fusion with weighted data. The normalization process is achieved by subtracting the minimum value from the original fluorescence signal intensity and then dividing by the difference between the maximum and minimum values. By combining the adjusted pollution intensity weights with the normalized fluorescence signal intensity, the two are multiplied to obtain the corrected pollution signal value for each pixel. The corrected pollution signal values ​​of all pixels are arranged according to their spatial coordinates, and the corrected pollution signal values ​​in the range of 0 to 1 are converted into gray values ​​of 0 to 255 using a gray-scale mapping method to generate a corrected pollution distribution feature map.

[0096] Step S106: Based on the corrected pollution distribution feature map and pollution degree distribution results, perform temperature-dependent diffusion simulation and concentration distribution spatial mapping to generate a comprehensive distribution map of hand pollution.

[0097] In one specific embodiment, step S106 includes the following steps:

[0098] The initial pollution concentration and corresponding temperature data of each region in the corrected pollution distribution feature map are extracted, and the diffusion path of pollutants with temperature changes is simulated using the finite difference method and thermodynamic principles.

[0099] Pollution weights are assigned to each region based on the pollution level distribution results, and the pollution contribution of each region in the diffusion simulation process is adjusted by weighted averaging.

[0100] Spatial mapping technology is used to map the diffusion simulation results to a two-dimensional hand image coordinate system. Interpolation is used to smooth the transition of the pollution concentration gradient, forming a continuous concentration distribution mapping result.

[0101] By integrating the diffusion data after regional pollution weight adjustment with the concentration distribution mapping results, and using color coding to quantify the pollution concentration level, a comprehensive distribution map of hand pollution is generated.

[0102] Specifically, the initial contamination concentration and corresponding temperature data for seven regions—fingertip, fingertip, finger gaps, palm, nail groove, back of hand, and wrist—are extracted from the corrected contamination distribution feature map. The initial contamination concentration is obtained by quantizing the fluorescence signal intensity of pixels in each region of the feature map. The initial contamination concentration for each region is taken as the average of the quantized fluorescence signal intensity values ​​of all pixels in that region, denoted as . ( For area code, The corresponding temperature data is extracted from the spatially registered thermal imaging data. The temperature data for each region is the average of the temperature values ​​of all pixels within that region, denoted as . The diffusion path of pollutants as a function of temperature is simulated using the finite difference method and thermodynamic principles. A diffusion equation is constructed based on Fick's diffusion law, and the diffusion coefficient is... It has a linear relationship with temperature, set Divide each region into

[0103] A discrete grid was used, with each grid cell having a side length of 0.1 mm. Diffusion simulation was performed at 10 time steps, each time step being 0.2 seconds. The diffusion equation was solved discretically using the finite difference method, and the discretization formula is as follows:

[0104] in For the grid cell at time step k pollution concentration, and The side length of the grid cell. Using the time step as the formula, the pollution concentration of each grid cell at each time step is calculated iteratively to obtain the diffusion path data of pollutants as temperature changes. This data reflects the dynamic changes and migration trends of pollution concentration in different regions under the influence of temperature.

[0105] Pollution weights were assigned based on the pollution intensity classification (low, medium, high) of each region in the pollution degree distribution results. The weight for low-pollution areas was set to 0.3, for medium-pollution areas to 0.6, and for high-pollution areas to 0.9. The weights for each region are denoted as follows: By adjusting the pollution contribution of each region in the diffusion simulation process using a weighted average, the pollution concentration data of each grid cell after the diffusion simulation are obtained. Combined with the weight of its region Calculate the adjusted pollution concentration This strengthens the pollution contribution of high-pollution-intensity areas in the diffusion simulation results, while weakening the pollution contribution of low-pollution-intensity areas, ensuring that the diffusion simulation results are more consistent with the actual pollution risk distribution.

[0106] Spatial mapping techniques were used to map the diffusion simulation results to a two-dimensional hand image coordinate system. This system was based on the image coordinates of the contamination distribution results, with an image resolution of 1920×1080 pixels. Each pixel corresponded to a 0.05mm×0.05mm area on the actual hand surface. The discrete mesh cells in the diffusion simulation results and the pixels in the two-dimensional hand image were established through coordinate transformation, using the following formula: ,in These are the pixel coordinates in a two-dimensional hand image. For the grid cell coordinates in the diffusion simulation, Let be the starting coordinates of the region in the 2D hand image. Continuous mapping of the diffusion simulation results is achieved through bilinear interpolation. For each pixel corresponding to a diffusion simulation grid cell, the contamination concentration value of each pixel is calculated based on the distance weight between the grid cell and the pixel. The interpolation formula is:

[0107] This interpolation process smooths the transition of the pollution concentration gradient, forming a continuous concentration distribution mapping result. This result fully presents the concentration distribution state after pollution diffusion in the two-dimensional hand image coordinate system.

[0108] The diffusion data and concentration distribution mapping results after regional pollution weight adjustment are integrated, and the adjusted diffusion data and concentration distribution mapping results are mapped one-to-one according to pixel coordinates, using the formula... The merged pollution concentration values ​​are calculated using a formula that balances the dynamic results of diffusion simulation with the static distribution of spatial mapping. A color coding method is used to quantify the pollution concentration levels, dividing the merged pollution concentration values ​​into five levels with corresponding concentration ranges of [0,0.2), [0.2,0.4), [0.4,0.6), [0.6,0.8), and [0.8,1.0], respectively. These levels are assigned five colors: blue, green, yellow, orange, and red, generating a comprehensive hand contamination distribution map.

[0109] By correlating temperature with diffusion coefficients, the diffusion trend of pollution in different temperature regions is accurately simulated, compensating for the shortcomings of single static assessment. Pollution weight allocation is based on regional pollution intensity classification, highlighting the pollution contribution of high-risk areas and solving the problem of one-sided assessment. Spatial mapping and interpolation processing realize the transformation of discrete data into continuous images, and combined with color coding, the pollution distribution is made intuitive and identifiable, improving the practicality of the assessment results. The final comprehensive distribution map of hand pollution fully integrates dynamic diffusion information and static concentration distribution, providing accurate data support for subsequent inter-regional pollution comparison analysis and the generation of targeted assessment reports.

[0110] Step S107: Based on the comprehensive distribution map of hand contamination, conduct a comparative analysis of contamination between regions to determine the sanitary status of specific areas of the hands and generate a targeted assessment report.

[0111] In one specific embodiment, step S107 includes the following steps:

[0112] Extract the quantitative indicators of pollution concentration for each region from the comprehensive distribution map of hand contamination, and calculate the difference in the quantitative indicators of pollution concentration between adjacent regions.

[0113] Based on the comparison results between the degree of difference and the preset difference threshold, combined with the pollution concentration level of each region, the hygiene status of specific areas of the hands, especially hidden areas such as between the fingers, is judged, and high-risk pollution areas are marked.

[0114] Summarize the results of the sanitation status assessment and information on high-risk polluted areas, focus on the pollution risk in hidden areas, and quantitatively analyze the pollution distribution characteristics and potential diffusion hazards.

[0115] Based on the quantitative analysis results, a targeted assessment report is generated, which clarifies the hygiene compliance status, pollution risk level, and targeted cleaning recommendations for each specific area.

[0116] Specifically, quantitative indicators of pollution concentration were extracted from seven regions—fingertips, fingertips, finger gaps, palms, nail grooves, backs of hands, and wrists—in the comprehensive distribution map of hand pollution. These indicators include the average pollution concentration of each region. ( For area code, ) and distribution density The average pollution concentration is obtained by averaging the pollution concentration values ​​of all pixels within the region. The distribution density is calculated as the ratio of the number of pixels with a pollution concentration value higher than 0.4 to the total number of pixels in the region. The difference in pollution concentration quantification indicators between adjacent regions is calculated, encompassing both the difference in average concentration and the difference in distribution density. Distribution density differences Overall Difference This formula balances the differences between concentration levels and distribution uniformity through weight allocation, making the calculation of the degree of difference more in line with the actual needs of pollution risk assessment.

[0117] Based on the comparison results between the difference degree and the preset difference threshold, the sanitary status is determined by combining the pollution concentration level of each area. The preset difference threshold is set to 0.3, and the pollution concentration level is divided according to the merged pollution concentration value: [0, 0.2) is level 1, [0.2, 0.4) is level 2, [0.4, 0.6) is level 3, [0.6, 0.8) is level 4, and [0.8, 1.0] is level 5. When the comprehensive difference degree of adjacent areas... A value exceeding 0.3 indicates significant differences in pollution risk between regions. Combined with pollution concentration levels, if any region reaches level four or higher, it is marked as a high-risk pollution area. For hidden areas such as between fingers and nail grooves, even if the overall difference does not exceed the threshold, if the pollution concentration level reaches level three or higher, it is also directly marked as a high-risk pollution area. This judgment logic specifically addresses the problem of insufficient identification of pollution risk in hidden areas by existing technologies, ensuring that no high-risk areas are overlooked by strengthening the assessment standards for hidden areas.

[0118] This study summarizes the results of sanitation status assessments and information on high-risk contaminated areas, focusing on a quantitative analysis of pollution risks in hidden areas. It extracts data such as average pollution concentration, distribution density, and overall variability in high-risk hidden areas, and combines this with temperature-dependent diffusion simulation results to analyze pollution distribution characteristics, including concentrated pollution areas, diffusion pathways, and potential diffusion ranges. A risk index is used to quantify potential diffusion risks. ,in The concealment coefficient is set to 1.6 for finger gaps and nail grooves, and 1.0 for other areas. The temperature influence coefficient (calculated based on temperature data for this region). , (Based on the regional average temperature), the risk index directly reflects the severity of pollution risks in hidden areas and the likelihood of their spread.

[0119] Based on the quantitative analysis results, a targeted assessment report is generated. The report clearly defines the hygiene compliance status of specific areas. The compliance standard is set at a pollution concentration level below level three and not marked as a high-risk pollution area. Areas that do not meet the standard must provide a detailed list of the quantitative indicators of pollution concentration, risk index, and reasons for high-risk marking. Pollution risk levels are divided according to the risk index: [0, 0.3) is low risk, [0.3, 0.6) is medium risk, [0.6, 0.9) is high risk, and [0.9, 1.0] is extremely high risk. Different cleaning recommendations correspond to different risk levels. For hidden areas such as between fingers and nail grooves, it is recommended to use dedicated cleaning tools for focused wiping, with a cleaning time of no less than 30 seconds. For areas with high pollution concentration levels and a high risk of spread, it is recommended to increase the frequency of cleaning and use cleaning products with stronger bactericidal effects. The report also includes a comprehensive hand contamination distribution chart, visually displaying the location of high-risk areas and the characteristics of contamination distribution, providing users with clear cleaning guidance.

[0120] The entire process addresses the issue of biased judgment caused by a single assessment indicator by extracting multi-dimensional quantitative indicators and calculating differences. It also strengthens assessment standards for hidden areas, compensating for the shortcomings of existing technologies in identifying hidden pollution risks. By combining diffusion simulation results with risk index quantitative analysis, it achieves accurate assessment of pollution risks and early warning of potential hazards. The resulting targeted assessment report provides a scientific basis for hygiene and effectively improves the effectiveness of hand hygiene management, especially suitable for scenarios with stringent hygiene standards, such as medical and food processing industries.

[0121] Please see Figure 2 , Figure 2 The performance comparison chart shows the quantitative comparison results between the proposed method and traditional methods in eight core performance dimensions: early warning capability, anti-interference capability, risk identification, comprehensiveness, detection sensitivity, practicality, stability, and accuracy. It demonstrates that the proposed method outperforms traditional methods in all performance dimensions, especially in early warning capability, risk identification, and comprehensiveness. It effectively solves the problems of traditional single-modal assessment being one-sided, having weak anti-interference capability, and lacking dynamic early warning, and verifies the technical improvement effect of the proposed method in terms of assessment comprehensiveness, accuracy, and risk prediction capability.

[0122] Please see Figure 3 , Figure 3 The bar charts show the percentage improvement in performance across eight capability indicators (detection sensitivity, risk identification, early warning capability, anti-interference capability, comprehensiveness, practicality, accuracy, and stability). Early warning capability improved by 70%, and risk identification by 69%, representing the two highest improvements. Detection sensitivity improved by 22%, a relatively lower improvement. This indicates that the method's optimization effect is most prominent in risk-related capabilities (early warning and identification), while also achieving varying degrees of performance enhancement in other dimensions such as comprehensiveness and stability. Overall, the method exhibits highly targeted and comprehensive improvements.

[0123] Please see Figure 4 The following describes the hand contamination assessment system fused with multimodal images in the embodiments of this application. The hand contamination assessment system fused with multimodal images includes:

[0124] The data acquisition module is used to simultaneously acquire visible light images, microbial fluorescence signals and thermal imaging data of the hand surface to form a raw multimodal dataset, which is then processed to obtain hand multimodal data.

[0125] The contour segmentation module is used to extract hand contour features and fluorescence signal intensity distribution from hand multimodal data, and to finely segment the hand contour features to obtain an initial hand contamination distribution feature map.

[0126] The pollution grading module is used to calculate the pollution concentration gradient based on the initial hand pollution distribution feature map, and to grade and assess the pollution intensity of different hand areas in combination with the fluorescence signal intensity distribution to determine the pollution degree distribution results.

[0127] The weighted fusion module is used to perform spatial registration of thermal imaging data and calibration of fluorescence signal intensity based on the pollution degree distribution results, and then perform superposition processing. By calculating the temperature field distribution weight, a temperature-weighted hand pollution distribution map is obtained.

[0128] The correction and optimization module is used to determine whether the proportion of pollution in the high-temperature area in the temperature-weighted hand pollution distribution map exceeds the preset proportion threshold. If so, the pollution intensity weight in the high-temperature area is reduced by the temperature correction coefficient, and the fluorescence signal intensity is normalized to determine the corrected pollution distribution feature map.

[0129] The visual presentation module is used to perform temperature-dependent diffusion simulation and concentration distribution spatial mapping based on the corrected pollution distribution feature map, and generate a comprehensive distribution map of hand pollution.

[0130] The report generation module is used to conduct comparative analysis of pollution between regions based on the comprehensive distribution map of hand contamination, determine the hygiene status of specific areas of the hands, and generate targeted assessment reports.

[0131] Through the collaborative efforts of the aforementioned components, this system constructs a multimodal, end-to-end intelligent assessment and dynamic risk early warning system for hand contamination. It achieves end-to-end closed-loop management, from synchronous acquisition of multi-source heterogeneous data, to refined analysis of hand contours and contamination features, multi-dimensional contamination intensity grading, temperature-weighted multimodal data fusion, dynamic correction of contamination weights in high-temperature areas, to temperature-dependent contamination diffusion simulation, visualized comprehensive distribution presentation, and generation of targeted assessment reports.

[0132] The data acquisition module, as the core foundation, simultaneously captures multi-dimensional data of the hand surface using visible light sensors, fluorescence imaging devices, and thermal imaging devices. After noise reduction processing including median filtering, threshold segmentation, and Gaussian filtering, it provides a high-quality, interference-free raw data source for subsequent analysis. The contour segmentation module extracts hand contour features using edge detection algorithms, quantifies the fluorescence signal intensity distribution, and divides boundary sub-regions using gradient calculation. It then maps the fluorescence distribution to these sub-regions to generate an initial pollution distribution feature map, transforming raw data into structured features. The pollution grading module calculates the pollution concentration gradient based on the initial feature map, divides regions using hand anatomical structures, compares quantitative indicators such as mean and variance with preset standards, and uses regional health risk weights to classify pollution intensity into low, medium, and high levels, outputting the pollution degree distribution results. The weighted fusion module establishes a spatial registration relationship between thermal imaging data and pollution distribution results, calibrates fluorescence signal intensity, performs pixel-level overlay, extracts the temperature field distribution, and converts it into weighting coefficients to generate a temperature-weighted pollution distribution map, achieving deep fusion and interoperability of multi-modal data. The supplementary and optimization module identifies high-temperature areas and calculates the pollution coverage ratio. If it exceeds a preset threshold, it constructs a gradient distribution based on temperature differences, generates a temperature correction coefficient to adjust the pollution intensity weight, and normalizes the fluorescence signal to optimize the pollution distribution presentation. The visual presentation module extracts the initial pollution concentration and temperature data from the corrected feature map, uses the finite difference method to simulate temperature-dependent pollution diffusion paths, and combines regional pollution weight adjustment and spatial mapping technology to generate an intuitive comprehensive distribution map of hand pollution through color coding. The report generation module extracts the pollution quantitative indicators of each area in the comprehensive distribution map, calculates the difference between adjacent areas, judges the sanitary status of specific areas, especially hidden areas, based on the pollution level, marks high-risk areas, and quantitatively analyzes the pollution distribution characteristics and potential diffusion hazards. Finally, it generates a standardized assessment report that includes compliance status, risk level, and targeted cleaning recommendations. Each module is interconnected and progressively builds upon the previous one, ensuring both the accuracy and comprehensiveness of data processing and the practicality and foresight of the assessment results, effectively solving the limitations of traditional single-modal assessments.

[0133] This application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the hand contamination assessment method fused with multimodal images.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the methods and systems described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0135] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for assessing hand contamination by fusing multimodal images, characterized in that, Includes the following steps: Step S101: Simultaneously acquire visible light images, microbial fluorescence signals, and thermal imaging data of the hand surface to form a raw multimodal dataset, and obtain hand multimodal data after denoising processing; Step S102: Extract hand contour features and fluorescence signal intensity distribution from the hand multimodal data, and refine the hand contour features through edge detection and gradient calculation to obtain an initial hand contamination distribution feature map; Step S103: Calculate the pollution concentration gradient based on the initial hand pollution distribution feature map, and evaluate the pollution intensity of different hand areas by combining the fluorescence signal intensity distribution to determine the pollution degree distribution result; Step S104: Establish the spatial coordinate mapping relationship between thermal imaging data and the pollution degree distribution result, and perform spatial registration on the thermal imaging data. Based on the pollution intensity classification of each region in the pollution degree distribution result, calibrate the fluorescence signal intensity of the corresponding region. Superimpose the spatially registered thermal imaging data and the calibrated fluorescence signal intensity data at the pixel level. Calculate the temperature field distribution weight to obtain a temperature-weighted hand pollution distribution map. Step S105: Mark the areas with temperatures above 37 degrees Celsius in the temperature-weighted hand contamination distribution map as high-temperature areas, determine whether the contamination ratio of the high-temperature areas exceeds the preset ratio threshold of 0.6, if so, construct a temperature gradient distribution based on the temperature difference between the temperature value of each pixel in the high-temperature area and the average temperature of the hand, calculate the temperature correction coefficient through linear interpolation, reduce the contamination intensity weight of the high-temperature areas through the temperature correction coefficient, and normalize the fluorescence signal intensity to determine the corrected contamination distribution feature map. Step S106: Based on the corrected pollution distribution characteristic map and the pollution degree distribution results, the diffusion coefficient formula is applied. Temperature-dependent diffusion simulations were performed, and the simulation results were processed using spatial mapping techniques and interpolation to generate a comprehensive distribution map of hand contamination. This indicates the corresponding temperature data for each region. Indicates the diffusion coefficient; Step S107: Based on the comprehensive distribution map of hand contamination, conduct a comparative analysis of contamination between regions to determine the hygiene status of specific areas of the hands and generate a targeted assessment report.

2. The method according to claim 1, characterized in that, Step S101 includes: Visible light images, microbial fluorescence signals, and thermal imaging data of the hand surface are acquired simultaneously using visible light sensors, fluorescence imaging devices, and thermal imaging devices. The acquired visible light images, microbial fluorescence signals, and thermal imaging data are time-stamped to generate the original multimodal dataset; For the original multimodal dataset, median filtering is performed on the visible light image, threshold segmentation is performed on the microbial fluorescence signal, and Gaussian filtering is performed on the thermal imaging data to obtain hand multimodal data.

3. The method according to claim 2, characterized in that, Step S102 includes: The visible light image in the hand multimodal data is processed by an edge detection algorithm to identify hand edge lines and form hand contour features; Intensity mapping is performed on the microbial fluorescence signal data in the hand multimodal data to quantify the fluorescence value distribution of each pixel and obtain the fluorescence signal intensity distribution; Calculate the gradient of the hand contour features in the horizontal and vertical directions to obtain the gradient magnitude and direction of each contour point; If the gradient magnitude of any contour point exceeds a preset gradient threshold, the contour point is marked as a boundary inflection point, and the boundary sub-region of the contour is divided based on the boundary inflection point. The fluorescence signal intensity distribution is mapped onto the defined boundary sub-regions to generate an initial hand contamination distribution feature map.

4. The method according to claim 1, characterized in that, Step S103 includes: Based on the initial hand contamination distribution feature map, the gradient values ​​of the fluorescence signal intensity of each pixel in the horizontal and vertical directions are calculated respectively; The pollution concentration gradient of each pixel is calculated based on the gradient values ​​in the horizontal and vertical directions, forming a gradient distribution matrix; The hand is divided into different regions based on anatomical structure. Fluorescence signal intensity data of each region are extracted from the gradient distribution matrix. The mean and variance of fluorescence signal intensity in each region are calculated to obtain the pollution quantification index of each region. The pollution quantification indicators of each region are compared with the preset pollution intensity classification standards. Combined with the health risk weight of different areas of the hands, the pollution intensity of each region is divided into three levels: low, medium and high. By integrating the pollution intensity classification results of various regions, a pollution degree distribution result is generated.

5. The method according to claim 1, characterized in that, Step S104 includes: Establish a spatial coordinate mapping relationship between thermal imaging data and the pollution degree distribution results, and perform spatial registration of thermal imaging data through spatial transformation to achieve spatial registration between thermal imaging data and the hand area in the pollution degree distribution results; Based on the pollution intensity classification of each region in the pollution degree distribution results, the fluorescence signal intensity calibration coefficient of the corresponding region is determined, and the fluorescence signal intensity of the corresponding region is calibrated using the calibration coefficient; The spatially registered thermal imaging data and the calibrated fluorescence signal intensity data are superimposed at the pixel level to generate a preliminary fused image. Temperature field distribution information is extracted from spatially registered thermal imaging data, and temperature values ​​are converted into corresponding weight coefficients through a weight mapping function. By multiplying the fluorescence signal intensity value of each pixel in the preliminary fused image with the corresponding weight coefficient, and integrating the calculation results of all pixels, a temperature-weighted hand contamination distribution map is obtained.

6. The method according to claim 1, characterized in that, Step S105 includes: Identify the high-temperature region in the temperature-weighted hand contamination distribution map and calculate the coverage ratio of the contamination signal within the high-temperature region; Determine whether the pollution coverage ratio exceeds a preset ratio threshold. If so, extract the temperature value of each pixel in the high-temperature area and calculate the temperature difference with the average hand temperature. Construct a temperature gradient distribution based on the temperature difference. Based on the temperature gradient distribution, a gradient interval is set, and a corresponding correction coefficient reference value is assigned to each gradient interval. The temperature correction coefficient of each pixel is calculated by linear interpolation. The pollution intensity weight of the high-temperature region is reduced according to the temperature correction coefficient. Extract the fluorescence signal intensity data from the temperature-weighted hand contamination distribution map and normalize it. By combining the adjusted pollution intensity weights with the normalized fluorescence signal intensity data, the pollution distribution presentation is optimized, and the corrected pollution distribution feature map is determined.

7. The method according to claim 1, characterized in that, Step S106 includes: The initial pollution concentration and corresponding temperature data of each region in the corrected pollution distribution feature map are extracted, and the diffusion path of pollutants with temperature changes is simulated using the finite difference method and thermodynamic principles. Pollution weights are assigned to each region based on the pollution level distribution results, and the pollution contribution of each region in the diffusion simulation process is adjusted by weighted averaging. Spatial mapping technology is used to map the diffusion simulation results to a two-dimensional hand image coordinate system. Interpolation is used to smooth the transition of the pollution concentration gradient, forming a continuous concentration distribution mapping result. By integrating the diffusion data after regional pollution weight adjustment with the concentration distribution mapping results, and using color coding to quantify the pollution concentration level, a comprehensive distribution map of hand pollution is generated.

8. The method according to claim 1, characterized in that, Step S107 includes: Extract the quantitative indicators of pollution concentration for each region in the comprehensive distribution map of hand contamination, and calculate the difference in the quantitative indicators of pollution concentration between adjacent regions. Based on the comparison results between the difference degree and the preset difference threshold, combined with the pollution concentration level of each region, the hygiene status of specific areas of the hand, especially hidden areas such as between the fingers, is determined, and high-risk pollution areas are marked. Summarize the results of the sanitary status assessment and information on high-risk contaminated areas, focus on the pollution risk in hidden areas, and quantitatively analyze the pollution distribution characteristics and potential diffusion hazards; Based on the quantitative analysis results, a targeted assessment report is generated, which clarifies the hygiene compliance status, pollution risk level, and targeted cleaning recommendations for each specific area.

9. A hand contamination assessment system fused with multimodal images, used to implement the hand contamination assessment method fused with multimodal images as described in any one of claims 1 to 8, characterized in that, The hand contamination assessment system that integrates multimodal images includes: The data acquisition module is used to simultaneously acquire visible light images, microbial fluorescence signals and thermal imaging data of the hand surface to form a raw multimodal dataset, which is then processed to obtain hand multimodal data. The contour segmentation module is used to extract hand contour features and fluorescence signal intensity distribution from the hand multimodal data, and to finely segment the hand contour features through edge detection and gradient calculation to obtain an initial hand contamination distribution feature map. The pollution grading module is used to calculate the pollution concentration gradient based on the initial hand pollution distribution feature map, and to grade and evaluate the pollution intensity of different hand areas in combination with the fluorescence signal intensity distribution to determine the pollution degree distribution result; The weighted fusion module is used to establish a spatial coordinate mapping relationship between thermal imaging data and the pollution degree distribution results and to perform spatial registration of the thermal imaging data. Based on the pollution intensity classification of each region in the pollution degree distribution results, the fluorescence signal intensity of the corresponding region is calibrated. The spatially registered thermal imaging data and the calibrated fluorescence signal intensity data are superimposed at the pixel level. Through temperature field distribution weight calculation, a temperature-weighted hand pollution distribution map is obtained. The correction and optimization module is used to mark areas with temperatures above 37 degrees Celsius in the temperature-weighted hand contamination distribution map as high-temperature areas, determine whether the contamination ratio of high-temperature areas exceeds a preset ratio threshold of 0.6, and if so, construct a temperature gradient distribution based on the temperature difference between the temperature value of each pixel in the high-temperature area and the average temperature of the hand, calculate the temperature correction coefficient through linear interpolation, reduce the contamination intensity weight of high-temperature areas through the temperature correction coefficient, and normalize the fluorescence signal intensity to determine the corrected contamination distribution feature map. The visual presentation module is used to apply the diffusion coefficient formula based on the corrected pollution distribution feature map and the pollution degree distribution results. Temperature-dependent diffusion simulations were performed, and the simulation results were processed using spatial mapping techniques and interpolation to generate a comprehensive distribution map of hand contamination. This indicates the corresponding temperature data for each region. Indicates the diffusion coefficient; The report generation module is used to conduct inter-regional pollution comparison analysis based on the comprehensive distribution map of hand contamination, determine the hygiene status of specific areas of the hands, and generate a targeted assessment report.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by a processor, the processor performs the hand contamination assessment method based on any one of claims 1 to 8.

Images