Non-contact intelligent nursing night ward-round system and method based on multi-mode infrared sensing

Abstract

The invention discloses a non-contact intelligent nursing night ward-round system and method based on multi-mode infrared sensing, and relates to the technical field of intelligent nursing night ward-round. The non-contact intelligent nursing night ward round method based on multi-mode infrared sensing comprises the following steps: continuously acquiring thermal imaging time sequence data and pressure distribution time sequence data of a patient to be examined at night in a set period; based on the thermal imaging time sequence data of the patient to be examined at night, analyzing a thermal cycle stability characteristic value; and on the basis of a pre-trained hot-pressing posture sensing model, in combination with the thermal imaging time sequence data and the pressure distribution time sequence data, analyzing a thermal posture steady-state characteristic value, and carrying out fusion processing on the thermal posture steady-state characteristic value and a thermal cycle stable characteristic value to obtain a thermal imbalance characteristic value of the patient to be examined at night. Therefore, cooperative monitoring of thermal circulation and posture change of the patient is achieved, the reliability of patient state recognition is improved, and then the precision of night ward-round nursing is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent nursing night rounds technology, specifically to a non-contact intelligent nursing night rounds system and method based on multimodal infrared sensing. Background Technology

[0002] With the rapid development of smart healthcare and intelligent nursing technologies, the automation and intelligence of hospital nighttime nursing has become an important research direction. The traditional nighttime ward round mode mainly relies on nursing staff to conduct regular manual rounds and observe changes in patients' vital signs and postures through visual inspection and palpation. This method is affected by factors such as human resources, perception threshold, and nighttime lighting conditions, making it difficult to achieve continuous and non-disruptive monitoring. In recent years, with the maturity of non-contact sensing technology and artificial intelligence algorithms, non-contact methods such as infrared sensing have been gradually used to dynamically monitor patients' nighttime conditions.

[0003] Infrared thermal imaging technology collects infrared radiation information from the human body surface. This technology has advantages such as being non-contact, radiation-free, and low-interference. It can continuously monitor the thermal field state of the patient without affecting their sleep. At the same time, distributed pressure sensing technology on the bed surface can reflect the contact state and support changes between the patient and the bed surface, and can be used to identify behavioral events such as falls. However, at present, it is mostly based on single-modal independent acquisition, which makes it difficult to achieve unified analysis of human body thermal distribution and posture in the same time sequence.

[0004] The limitations of existing technologies include at least the following problems: existing technologies cannot monitor changes in body surface heat distribution and posture in real time without disturbing the patient's sleep. As a result, when faced with different nighttime behaviors, it is difficult to reflect changes in the patient's thermal circulation and postural homeostasis in a timely manner, which can easily lead to delays in ward rounds or inaccurate risk identification. For example, when the patient's body surface heat circulation fluctuates but the posture remains still, if it is not distinguished as a local metabolic abnormality, potential risks may be overlooked. When the body surface heat field is stable but the supporting pressure changes abruptly, it may be difficult to judge whether it is a fall or abnormal turning over. In addition, when the heat field and posture changes are not perceived simultaneously, it is easy to lead to inaccurate results in the patient's overall homeostasis assessment, which in turn affects the response timeliness and safety reliability of night ward rounds. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a non-contact intelligent nursing night round system and method based on multimodal infrared sensing, which solves the problem that existing technologies are unable to simultaneously sense patients' thermal circulation and posture changes, leading to inaccurate identification during night rounds.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a non-contact intelligent nursing night round method based on multimodal infrared sensing, comprising the following steps: continuously acquiring thermal imaging time-series data and pressure distribution time-series data of the patient to be examined within a set period; analyzing the thermal cycling stability characteristic value of the patient based on the thermal imaging time-series data; analyzing the thermal posture steady-state characteristic value of the patient based on a pre-trained thermal pressure posture perception model, combined with the thermal imaging time-series data and pressure distribution time-series data, and fusing it with the thermal cycling stability characteristic value to obtain the thermal imbalance characteristic value of the patient; and performing ward round nursing care based on the thermal imbalance characteristic value.

[0007] Furthermore, the thermal imaging time-series data includes several time-sampled frame thermal imaging data, and each time-sampled frame thermal imaging data specifically includes the temperature value and two-dimensional coordinates of each pixel.

[0008] Further, the specific steps for analyzing the thermal circulation stability characteristics of patients awaiting night inspection are as follows: Based on the thermal imaging time series data of patients awaiting night inspection, analyze the thermal characteristic set of patients awaiting night inspection, including body surface heat distribution characteristics and respiratory heat flux characteristics; based on the thermal characteristic set of patients awaiting night inspection, analyze the thermal circulation stability characteristics of patients awaiting night inspection.

[0009] Further, the specific steps for analyzing the thermal feature set of the patients to be examined at night are as follows: preprocess the thermal imaging data of each time frame of the patients to be examined at night; divide the preprocessed thermal imaging data of each time frame of the patients to be examined at night into regions to obtain the pixel set of several thermal regions in the thermal imaging of each time frame of the patients to be examined at night; and deconstruct the pixel set of each thermal region in the thermal imaging of each time frame of the patients to be examined at night to obtain the surface heat distribution feature value and respiratory heat flux feature value of the patients to be examined at night.

[0010] Furthermore, the pressure distribution time series data specifically refers to the pressure value at each location in each time sampling frame, and the hot pressure attitude perception model includes an input layer, a feature extraction layer, a spatiotemporal convolutional layer, and an output layer.

[0011] Further, the specific steps for analyzing the thermal posture steady-state characteristic values ​​of patients awaiting night inspection are as follows: input the thermal imaging time-series data and pressure distribution time-series data of the patients awaiting night inspection into the pre-trained thermal pressure posture perception model, analyze the posture mapping feature set of the patients awaiting night inspection, including thermal posture coordination feature values, thermal posture migration feature values, and tipping risk feature values; based on the posture mapping feature set of the patients awaiting night inspection, analyze the thermal posture steady-state characteristic values ​​of the patients awaiting night inspection.

[0012] Further, the specific steps for analyzing the posture mapping feature set of patients awaiting night inspection are as follows: In the input layer of the thermo-pressure posture perception model, the thermal imaging time-series data and pressure distribution time-series data of the patients awaiting night inspection are received and preprocessed; in the semantic node recognition layer of the thermo-pressure posture perception model, the preprocessed thermal imaging time-series data and pressure distribution time-series data of the patients awaiting night inspection are recognized and processed to extract the thermal zone feature vector of each time sampling frame of the patients awaiting night inspection; in the spatiotemporal convolutional layer of the thermo-pressure posture perception model, the posture feature vector of the patients awaiting night inspection is extracted based on the thermal zone feature vector of each time sampling frame of the patients awaiting night inspection; in the output layer of the thermo-pressure posture perception model, the posture mapping feature set of the patients awaiting night inspection is output based on the posture feature vector of the patients awaiting night inspection.

[0013] Furthermore, the specific formula for calculating the thermal imbalance characteristic value of patients to be examined at night is as follows: ;in, For the thermal imbalance characteristic values ​​of patients awaiting night examination, For the thermal cycling stability characteristic values ​​of patients awaiting night examination, The thermal cycling adjustment coefficients are stored in the database. The thermal posture steady-state characteristic values ​​of patients awaiting night examination. These are the thermal attitude steady-state adjustment coefficients stored in the database. The difference adjustment coefficients are stored in the database. .

[0014] Furthermore, the specific steps for conducting ward rounds and nursing care for patients to be checked at night based on thermal imbalance characteristic values ​​are as follows: compare the thermal imbalance characteristic values ​​of the patients to be checked at night with the preset thermal imbalance characteristic threshold; and conduct ward rounds and nursing care for the patients to be checked at night based on the comparison results.

[0015] A non-contact intelligent nursing night round system based on multimodal infrared sensing includes: a data acquisition module for continuously acquiring thermal imaging time-series data and pressure distribution time-series data of patients to be examined within a set period; a thermal circulation analysis module for analyzing the thermal circulation stability characteristic values ​​of patients to be examined based on the thermal imaging time-series data; a thermal posture analysis module for analyzing the thermal posture steady-state characteristic values ​​of patients to be examined based on a pre-trained thermal pressure posture perception model, combined with the thermal imaging time-series data and pressure distribution time-series data; a comprehensive analysis module for fusing the thermal circulation stability characteristic values ​​and thermal posture steady-state characteristic values ​​of patients to be examined to obtain the thermal imbalance characteristic values; and a ward round nursing feedback module for providing ward round nursing care based on the thermal imbalance characteristic values.

[0016] The present invention has the following beneficial effects: (1) The non-contact intelligent nursing night round method based on multimodal infrared perception achieves coordinated dynamic perception of body surface thermal circulation and posture by synchronously collecting thermal imaging time-series data and pressure distribution time-series data within a set period. Based on the thermal imaging time-series data, thermal circulation stability feature values ​​that characterize changes in the patient's thermal circulation are extracted. At the same time, combined with pressure distribution time-series data, a posture mapping feature set that can reflect posture behavior is constructed, and thermal posture stability feature values ​​are generated. These feature values ​​are then fused and analyzed with thermal circulation stability feature values ​​to obtain thermal imbalance feature values. This allows for the joint identification of the patient's thermal imbalance level from both energy conduction and posture changes, thereby achieving coordinated monitoring of the patient's thermal circulation and posture changes and improving the reliability of patient status identification, thus effectively improving the accuracy of night round nursing.

[0017] (2) The non-contact intelligent nursing night round method based on multimodal infrared perception introduces a pre-trained thermal pressure posture perception model, thereby realizing the joint analysis of thermal imaging and pressure information. In the semantic node recognition layer, the main parts are used as node features by dividing the thermal domain region and extracting the centroid. In the spatiotemporal convolution layer, the corresponding features are extracted, so as to capture the correspondence between heat conduction and displacement dynamics between different parts, thereby accurately identifying abnormal body postures of patients and improving the proactive perception capability of intelligent nursing.

[0018] (3) This non-contact intelligent nursing night round method based on multimodal infrared perception extracts the surface heat distribution characteristic value and respiratory heat flux characteristic value by deeply analyzing the thermal imaging time series data. It comprehensively analyzes the temperature balance and heat conduction continuity and respiratory heat flux change law between different thermal areas, thereby reflecting the stability of heat energy transfer in the patient in the time and space domain. It can then dynamically identify the fluctuation trend of surface heat circulation. When the patient has local metabolic imbalance or respiratory rhythm disorder, the heat circulation stability characteristic value will change significantly, thereby realizing continuous monitoring of the patient's heat circulation status and improving the automation level of night rounds.

[0019] (4) The non-contact intelligent nursing night round system based on multimodal infrared perception realizes the automated response of night rounds through the hierarchical collaboration of modules. The thermal circulation analysis module and the thermal posture analysis module respectively process the thermal circulation state and posture steady state of the patient's body surface in parallel, and complete the feature fusion in the comprehensive analysis module. In the ward round nursing feedback module, abnormality judgment is performed, and ward round nursing is performed when abnormality occurs. Thus, the patient's night state can be automatically monitored and intelligently responded to throughout the night, thereby effectively improving the real-time and accuracy of nursing work and reducing the intensity of manual patrols, thereby ensuring the continuity of the night nursing process.

[0020] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0021] Figure 1 This is a flowchart of a non-contact intelligent nursing night round method based on multimodal infrared sensing according to the present invention.

[0022] Figure 2 This is a flowchart illustrating the specific steps involved in analyzing the thermal characteristic set of patients to be examined during a non-contact intelligent nursing ward round based on multimodal infrared sensing, as described in this invention.

[0023] Figure 3 This is a schematic diagram of the posture mapping temporal feature set data of the patient to be checked in a non-contact intelligent nursing ward round method based on multimodal infrared sensing according to the present invention.

[0024] Figure 4 This is a block diagram of a non-contact intelligent nursing night round system based on multimodal infrared sensing according to the present invention. Detailed Implementation

[0025] Please see Figure 1 This invention provides a technical solution: a non-contact intelligent nursing night round method based on multimodal infrared sensing, comprising the following steps: within a set period (it should be noted that, to ensure the alignment of thermal imaging time sampling frames and pressure data in the time dimension, the length of the time sampling frame sequence within the period is set to the least common multiple of the sampling frequencies of the two types of data), continuously acquiring thermal imaging time-series data and pressure distribution time-series data of the patient to be examined; based on the thermal imaging time-series data of the patient to be examined, analyzing the thermal circulation stability characteristic value of the patient to be examined; based on a pre-trained thermal pressure posture perception model, and combined with the thermal imaging time-series data and pressure distribution time-series data of the patient to be examined, analyzing the thermal posture steady-state characteristic value of the patient to be examined, and fusing it with the thermal circulation stability characteristic value to obtain the thermal imbalance characteristic value of the patient to be examined (used to characterize the comprehensive imbalance degree of the patient's body surface thermal circulation and posture within the set period); and performing ward round nursing care based on the thermal imbalance characteristic value of the patient to be examined.

[0026] The specific formula for calculating the thermal imbalance characteristic value of patients to be examined at night is as follows: ;in, For the thermal imbalance characteristic values ​​of patients awaiting night examination, For the thermal cycling stability characteristic values ​​of patients awaiting night examination, The thermal cycling adjustment coefficients are stored in the database. The thermal posture steady-state characteristic values ​​of patients awaiting night examination. These are the thermal attitude steady-state adjustment coefficients stored in the database. The difference adjustment coefficients are stored in the database. .

[0027] It needs to be explained that the thermal cycling regulation coefficients stored in the database Thermal attitude steady-state adjustment coefficient The acquisition steps are as follows: Obtain the thermal cycling stability feature values ​​and thermal attitude steady-state feature values ​​for several historical cycles. Extract the mean thermal cycling stability feature value and the thermal attitude steady-state feature value respectively (it should be noted that the mean thermal cycling stability feature value and the thermal attitude steady-state feature value are obtained by transforming the result using the reciprocal suppression mapping function f(x)=1 / (1+x), and then normalize them. Summate these results to obtain the imbalance sum value. Ratio the normalized mean thermal cycling stability feature value and the thermal attitude steady-state feature value with the imbalance sum value, and use the corresponding results as the thermal cycling adjustment coefficient. Thermal attitude steady-state adjustment coefficient .

[0028] Difference adjustment coefficients stored in the database The acquisition steps are as follows: Obtain the thermal cycling stability characteristic values ​​and thermal posture steady-state characteristic values ​​for several historical cycles, and extract the correlation value (absolute value) between the two based on the Pearson adjustment coefficient, which is then used as the difference adjustment coefficient. .

[0029] The specific steps for conducting ward rounds and nursing care for patients to be checked at night based on thermal imbalance characteristic values ​​are as follows: The thermal imbalance characteristic values ​​of the patients to be checked at night are compared with preset thermal imbalance characteristic thresholds. Based on the comparison results, ward rounds and nursing care are conducted on the patients to be checked at night. Specifically: if the thermal imbalance characteristic value of the patients to be checked at night is lower than or equal to the preset thermal imbalance characteristic threshold, no ward round is conducted, indicating that the patient's condition is safe, and therefore no nursing ward round operation is triggered to avoid unnecessary manual intervention; if the thermal imbalance characteristic value of the patients to be checked at night is higher than the preset thermal imbalance characteristic threshold, a ward round is conducted, indicating that there may be an abnormal situation, and a nursing ward round instruction is automatically triggered, sending a ward round nursing reminder message to the nursing terminal or nurse's workstation.

[0030] The thermal imaging time series data includes several time-sampled frames of thermal imaging data. Each time-sampled frame of thermal imaging data specifically contains the temperature value and two-dimensional coordinates of each pixel.

[0031] Specifically, the steps for analyzing the thermal circulation stability characteristic values ​​of patients awaiting night inspection are as follows: Based on the thermal imaging time-series data of the patients awaiting night inspection, analyze the thermal state characteristic set of the patients awaiting night inspection, including surface heat distribution characteristic values ​​and respiratory heat flux characteristic values; Based on the thermal state characteristic set of the patients awaiting night inspection, analyze the thermal circulation stability characteristic values ​​of the patients awaiting night inspection (used to characterize the degree of energy circulation coordination between the patient's nighttime surface heat potential distribution and respiratory heat flux transfer process, to reflect the overall stability of human body heat generation, conduction and dissipation), which specifically involves: normalizing the surface heat distribution characteristic values ​​and respiratory heat flux characteristic values ​​of the patients awaiting night inspection, and then weighting the normalized surface heat distribution characteristic values ​​and respiratory heat flux characteristic values ​​of the patients awaiting night inspection to extract the thermal circulation stability characteristic values ​​of the patients awaiting night inspection.

[0032] It should be noted that in this implementation example, the weighting coefficients of each parameter in the weighted processing can be obtained using sample entropy weighting. Taking the weighted processing process of obtaining the stable characteristic value of the thermal cycle as an example, the surface heat distribution characteristic value and respiratory heat flux characteristic value of several historical cycles are obtained, and their corresponding information entropy values ​​are extracted respectively. Then, their corresponding information entropy values ​​are transformed using the reciprocal suppression mapping function f(x)=1 / (1+x), such as 1 / (1+information entropy value of surface heat distribution characteristic value), and summed to obtain the information entropy sum value. The corresponding transformed information entropy values ​​are then compared with the information entropy sum value to obtain the weighting coefficients corresponding to each parameter.

[0033] like Figure 2 As shown, the specific steps for analyzing the thermal feature set of patients to be examined at night are as follows: preprocess the thermal imaging data of each time frame of the patients to be examined at night, that is, denoise and normalize the thermal imaging data of each time frame. The denoising can be done by moving average filtering, wavelet threshold denoising or Gaussian smoothing filtering to eliminate environmental infrared interference and random noise. The temperature value of each pixel is normalized by Z-score or min-max normalization to map the temperature value to the standard range between 0 and 1. The preprocessed thermal imaging data of patients to be examined at night is divided into regions to obtain a set of pixels representing several thermal regions in each time frame of the thermal imaging. Specifically, for each time frame of thermal imaging, a local neighborhood (e.g., four pixels above, below, left, and right) is constructed centered on each pixel. Based on the Sobel operator, the temperature gradient value of each pixel is extracted, and the mean and standard deviation of the temperature gradient are calculated. A temperature gradient threshold is set (e.g., a multiple of the mean and standard deviation of the temperature gradient, which can be 1.2 to 1.8). Pixels with temperature gradient values ​​higher than the temperature gradient threshold are marked as thermal region boundary points. Connectivity analysis is performed on all thermal region boundary points (an eight-neighborhood-based connected component labeling algorithm or a depth-first search algorithm can be used to identify interconnected thermal region boundary clusters. Mathematical morphological processing, including dilation, erosion, and closing operations, is performed on the connected component analysis results to repair boundary breaks and remove isolated noise points) to obtain several thermal regions. The temperature values ​​and two-dimensional coordinates of all pixels contained in the thermal regions are marked as a set of pixels. It should be noted that the temperature value of each pixel within each thermal region is averaged to extract the average temperature of each region. For each thermal region, the temperature difference between adjacent time-sampled frames (i.e., the difference in the average temperature between two adjacent time-sampled frames) is analyzed. Based on this, a Fast Fourier Transform (FFT) is performed to extract several frequency components and their corresponding amplitudes. The frequency component corresponding to the largest amplitude is counted as the dominant frequency. It is then determined whether the thermal region is within a preset periodic fluctuation frequency range (e.g., 0.15–0.4 Hz, i.e., 9–24 fluctuations per minute). If it is within the preset periodic fluctuation frequency range and has the highest average temperature, the thermal region is marked as a breathing region. Thermal regions with an average temperature lower than a preset background temperature threshold are designated as background regions. Furthermore, the pixel set of each thermal region in each time frame of the thermal imaging of the patient to be examined at night is deconstructed to obtain the surface thermal distribution characteristic value and respiratory heat flux characteristic value of the patient to be examined at night, specifically: Based on the temperature value of each pixel in each thermal region (excluding the respiratory and background regions) of the patient to be examined at each time frame of thermal imaging, the thermal temperature value of each thermal region in each time frame of thermal imaging is extracted. The thermal temperature variance of each thermal region in the thermal imaging time frame sequence is also extracted and averaged to extract the body surface thermal temperature variance. Furthermore, the thermal temperature difference is calculated in each time frame of thermal imaging (i.e., the average of the differences between the thermal temperature values ​​of different thermal regions in that time frame is extracted), and the thermal temperature difference is extracted. The maximum value and the mean thermal difference are combined and processed, i.e., |maximum thermal difference - mean thermal difference| / mean thermal difference, to extract the surface thermal fluctuation value. This value is then weighted with the surface thermal variance value. The weighted result is then transformed using the reciprocal suppression mapping function f(x)=1 / (1+x), i.e., 1 / (1+the weighted result), to extract the surface thermal distribution characteristic value. This value is used to characterize the surface thermal balance state of the non-respiratory area of ​​the patient to be examined at night within a set time period. The higher the value, the more balanced the surface thermal distribution and the more stable the heat conduction process. The system reads the pixel set of the respiratory region from each time-sampling frame of the thermal imaging of the patient to be examined at night, extracts the average respiratory temperature from each time-sampling frame, and calculates its time derivative. When the time derivative of the average respiratory temperature is positive, it is determined to be the expiratory phase; when the time derivative of the average respiratory temperature is negative, it is determined to be the inspiratory phase. The time-sampling frame where the average respiratory temperature reaches its maximum value corresponds to the end of expiration, and the time-sampling frame where the average respiratory temperature reaches its minimum value corresponds to the end of inspiration. Each pair of time-sampling frames corresponding to the end of inspiration constitutes a respiratory cycle. For each respiratory cycle, the expiratory heat flux energy during the exhalation phase is calculated (by subtracting the mean respiratory heat and temperature values ​​of two consecutive time-sampling frames during the respiratory phase, retaining only the portion greater than zero to characterize the heat transfer rate, and summing the values ​​throughout the entire exhalation phase to obtain the expiratory heat flux energy) and the inspiratory heat flux energy during the inhalation phase are calculated (by subtracting the mean respiratory heat and temperature values ​​of two consecutive time-sampling frames during the respiratory phase, retaining only the portion less than zero to characterize the heat transfer rate, and summing the absolute values ​​throughout the entire exhalation phase to obtain the inspiratory heat flux energy). These are then combined into a single value: |expiratory heat flux energy - inspiratory heat flux energy| / (expiratory heat flux energy + inspiratory heat flux energy) to extract the heat exchange offset value for each respiratory cycle. Simultaneously, the similarity values ​​of expiratory heat flow propagation during the exhalation phase are statistically analyzed (i.e., for each pixel in the respiratory region of the thermal imaging at each time sampling frame during the exhalation phase, the temperature value and two-dimensional coordinates are used to construct a covariance matrix, and feature decomposition is performed. The eigenvector corresponding to the largest eigenvalue is used to obtain the principal propagation direction angle in the thermal imaging at each time sampling frame, and the average propagation direction angle is extracted. Within the exhalation phase, the cosine similarity between the principal propagation direction angle and the average propagation direction angle of all time sampling frames is averaged to obtain the airflow direction consistency index) and the similarity values ​​of inspiratory heat flow propagation during the inhalation phase (the logic for obtaining the similarity values ​​of expiratory heat flow propagation is consistent with that of expiratory heat flow propagation). The similarity values ​​of propagation and inspiratory heat flow propagation are arithmetically averaged to extract the comprehensive propagation stability value of respiratory heat flow for each respiratory cycle. This value is then weighted and averaged with the symmetric value of heat exchange. In this process, the heat exchange offset value is transformed using the reciprocal suppression mapping function f(x)=1 / (1+x), i.e., 1 / (1+heat exchange offset value), to extract the characteristic value of respiratory heat flow. This characteristic value is used to characterize the overall coordination of the respiratory heat flow field of the patient to be examined at night within a single respiratory cycle. This characteristic value comprehensively reflects the balance of heat energy transfer and the stability of airflow propagation during respiration. The higher the value, the more coordinated the heat flow exchange and the more stable the respiration.

[0034] In this implementation plan, a comprehensive and dynamic assessment of the patient's thermal circulation process is achieved through refined analysis of thermal imaging sequences. Secondly, by dividing thermal regions and identifying respiratory regions, the raw thermal imaging data is transformed from a static temperature distribution into a dynamic thermal energy feature set. This not only reflects the heat conduction balance of different body parts but also reveals the energy change pattern of respiratory airflow in the thermal field. Finally, the joint analysis of heat transfer and respiratory flow characteristics forms a thermal circulation characterization index. This process can achieve non-contact detection of respiratory intensity, rhythm stability, and the degree of coordination of body surface circulation without external sensors. This allows for real-time capture of thermophysiological fluctuations in patients at rest, enabling continuous detection and dynamic trend analysis of the patient's thermal circulation status. Consequently, it allows for timely capture and feedback of circulatory abnormalities, thereby improving the reliability of nighttime monitoring.

[0035] Specifically, the pressure distribution time series data is the pressure value at each location in each time sampling frame (which can be obtained by deploying a distributed pressure sensor array in the key support area of ​​the bed surface to form a pressure distribution matrix that can reflect the local contact state, and the key support area of ​​the bed surface can include the head support area, back support area, hip support area and leg support area, etc., that is, each location has its own corresponding label, such as the head support area). The thermal pressure posture perception model includes an input layer, a feature extraction layer, a spatiotemporal convolutional layer and an output layer.

[0036] The specific steps for analyzing the thermal posture steady-state characteristic values ​​of patients awaiting night inspection are as follows: Input the thermal imaging time-series data and pressure distribution time-series data of the patients awaiting night inspection into the pre-trained thermal pressure posture perception model, and analyze the posture mapping feature set of the patients awaiting night inspection, including thermal posture coordination feature values, thermal posture migration feature values, and tipping risk feature values; Based on the posture mapping feature set of the patients awaiting night inspection, analyze the thermal posture steady-state characteristic values ​​of the patients awaiting night inspection (used to characterize the overall posture balance of the patients within a set period).

[0037] The specific formula for calculating the thermal posture steady-state characteristic value of patients to be examined at night is as follows: ;in, The thermal posture steady-state characteristic values ​​of patients awaiting night examination. The thermal posture coordination characteristic values ​​of patients awaiting night examination. These are the thermal attitude coordination adjustment coefficients stored in the database. For the thermal posture migration characteristics of patients awaiting night examination, These are the thermal attitude migration adjustment coefficients stored in the database. The dumping risk characteristic value for patients awaiting night inspection. The dumping risk adjustment coefficient is stored in the database. .

[0038] It needs to be explained that the thermal attitude coordination adjustment coefficients stored in the database Thermal attitude migration adjustment coefficient Dumping risk adjustment coefficient The acquisition steps are as follows: Obtain the thermal posture coordination feature values, thermal posture migration feature values, and tipping risk feature values ​​for several historical periods. Extract the mean values ​​of the thermal posture coordination feature, thermal posture migration feature, and tipping risk feature respectively (it should be noted that the mean values ​​of the thermal posture migration feature and tipping risk feature are obtained by transforming them using the reciprocal suppression mapping function f(x)=1 / (1+x)). Normalize these values ​​and sum them to obtain the steady-state thermal posture sum. Ratio the normalized mean values ​​of the thermal posture coordination feature, thermal posture migration feature, and tipping risk feature with the steady-state thermal posture sum, and use the corresponding results as the thermal posture coordination adjustment coefficients. Thermal attitude migration adjustment coefficient Dumping risk adjustment coefficient .

[0039] The following is a specific implementation example for calculating the thermal posture steady-state characteristic values ​​of patients awaiting night inspection. The available data includes thermal posture coordination characteristic values, thermal posture migration characteristic values, and tipping risk characteristic values ​​of patients awaiting night inspection for 5 periods (randomly selected), as detailed in Table 1 and... Figure 3 As shown: Table 1. Example of temporal feature set data of posture mapping of patients awaiting night inspection.

[0040] Thermal attitude coordination coefficients stored in the database Approximately 0.330; Thermal posture migration adjustment coefficients stored in the database Approximately 0.321; The dumping risk adjustment coefficient stored in the database Approximately 0.349; Substituting the data from Table 1 and the aforementioned coefficients into the specific formula for calculating the thermal posture steady-state characteristic value of the patients to be examined at night, we obtain: The thermal posture steady-state characteristic value of the patients awaiting night inspection in the first cycle = 0.330×0.874 + (0.321 / (1+0.235)) + 0.349×exp(-0.156) ≈ 0.847; The thermal posture steady-state characteristic value of the patients awaiting night inspection in the second cycle = 0.330×0.762+(0.321 / (1+0.368))+0.349×exp(-0.243)≈0.760; The thermal posture steady-state characteristic value of the patients awaiting night inspection in the third cycle = 0.330 × 0.927 + (0.321 / (1 + 0.189)) + 0.349 × exp(-0.097) ≈ 0.893; The thermal posture steady-state characteristic value of the patients awaiting night inspection in the fourth cycle = 0.330×0.645+(0.321 / (1+0.427))+0.349×exp(-0.294)≈0.698; The thermal posture steady-state characteristic value of the patients awaiting night inspection in the fifth cycle is 0.330×0.831+(0.321 / (1+0.306))+0.349×exp(-0.221)≈0.800.

[0041] The specific steps for analyzing the posture mapping feature set of patients to be examined at night are as follows: In the input layer of the thermal pressure posture perception model, the thermal imaging time series data and pressure distribution time series data of the patients to be examined at night are received, and data preprocessing is performed, such as noise suppression and background removal for thermal imaging, median filtering or wavelet threshold denoising method for random thermal noise, and normalization processing is performed on the pressure matrix at each time sampling frame to normalize the pressure value to the standard interval [0, 1] to eliminate the influence of differences in weight and bed surface hardness of different patients; In the semantic node recognition layer of the thermal pressure attitude perception model, the thermal imaging time-series data and pressure distribution time-series data of the patients to be examined at night after data preprocessing are identified and processed to extract the thermal zone feature vector of each time sampling frame of the patients to be examined at night. Specifically: For each time-sampled frame of thermal imaging data, the mean temperature and standard deviation of the temperature for that time-sampled frame are extracted, and a temperature threshold is set accordingly, such as the mean temperature + 0.5 times the standard deviation of the temperature. Only pixels with a temperature higher than this threshold are retained, and connected component analysis is performed on the retained high-temperature pixel regions to identify the sets of interconnected pixels, thereby obtaining several connected regions. The area of ​​the corresponding connected regions (i.e., the total number of pixels) is extracted, and connected regions with an area less than a preset threshold (e.g., 1% of the total number of pixels) are identified as non-human residue areas and removed. The retained connected regions are regarded as suspected human thermal regions, and the centroid coordinates (i.e., the mean of the two-dimensional coordinates of all pixels in the region), average temperature value, region area, and temperature variance of each suspected human thermal region are extracted respectively. Each suspected human body thermal region is sorted from smallest to largest according to its vertical coordinate value in the centroid coordinate system (i.e., regions with smaller vertical coordinate values ​​correspond to the top of the image, and regions with larger vertical coordinate values ​​correspond to the bottom of the image). When the vertical distance between two adjacent suspected human body thermal regions (i.e., the absolute value of the difference between the vertical coordinate values ​​in the centroid coordinate system of the two adjacent suspected human body thermal regions) is less than a preset distance threshold (e.g., 10% of the image height), they are determined to be continuous regions of the same body part and are merged. When the vertical distance exceeds the threshold, they are considered to belong to different body parts. In this way, several human body thermal regions are obtained, and the centroid coordinates, average temperature value, area, and temperature variance of the corresponding thermal regions are extracted. When the vertical coordinate value of the centroid of the thermal zone of the human body is within the first 20% of the image height, and the average temperature value of the thermal zone is more than 5% higher than the global temperature average (i.e., the average temperature value of all pixels in the thermal image of the sampling frame at that time), and the area of ​​the thermal zone is less than 15% of the total area (i.e., the total number of all pixels in the thermal image of the sampling frame at that time), it is determined to be the head region. When the vertical coordinate value of the centroid of the human body's thermal region is between 30% and 60% of the image height, and the average temperature value of the thermal region is close to or slightly higher than the global average, and the thermal region area is at its maximum, it is identified as the torso region; when the vertical coordinate value of the centroid of the human body's thermal region is below the centroid of the torso region, and the vertical distance is less than 20% of the image height, and the thermal region area is between 10% and 20% of the total area, it is identified as the buttock region; when the vertical coordinate value of the centroid of the human body's thermal region is above 70% of the image height, and the longitudinal length of the region (i.e., the difference between the maximum and minimum longitudinal coordinate values ​​of all pixels within the human body's thermal region) is greater than the transverse width (i.e., the difference between the maximum and minimum transverse coordinate values ​​of all pixels within the human body's thermal region), it is identified as the lower limb region. Based on the pressure value and corresponding label of each location in each time sampling frame, the average pressure value of each support area (such as head support area, back support area, hip support area and leg support area) is calculated and then concatenated with the centroid coordinates of the hot zone, the average temperature value of the hot zone, the area of ​​the hot zone and the temperature variance of the hot zone in the head area, trunk area, hip area and lower limb area to form the hot zone feature vector. In the spatiotemporal convolutional layer of the thermal pressure posture perception model, the posture feature vector of the patient to be examined at night is extracted based on the thermal feature vector of each time sampling frame of the patient. Specifically, the thermal feature vector sequence output by the semantic node recognition layer is constructed into a temporal graph structure. The nodes in the graph correspond to the thermal regions of each human body (such as head, torso, buttocks and lower limbs). The node features include the centroid coordinates of the thermal region, the average temperature value of the thermal region, the temperature variance of the thermal region, and the average pressure value (that is, the support area corresponding to the average pressure value is attached to the corresponding node). Fixed connection edges are established based on the natural structure sequence of the human body, that is, the head region is connected to the torso region, the torso region is connected to the buttock region, and the buttock region is connected to the lower limb region in sequence to form a thermal topology chain structure arranged along the longitudinal direction of the human body. The edge weight of the connection edge can be obtained by using the Pearson correlation coefficient of the average temperature values ​​of the thermal regions of the two connected regions as the edge weight. Spatiotemporal graph convolutional networks perform graph convolution operations in the spatial dimension to aggregate feature information from neighboring nodes, capturing the correlation between heat conduction and posture changes between different body parts. Simultaneously, they perform convolution operations in the temporal dimension, calculating the changes in node features across consecutive time-sampled frames to extract dynamic trends in body posture changes, thus obtaining posture feature vectors, such as: For two nodes that establish a connection edge (which can be regarded as a group), the change value of thermal zone temperature variance in each time sampling frame is extracted (e.g., for the two nodes of torso and hip, the difference of thermal zone temperature variance between the torso node and the hip node is divided by the thermal zone temperature variance of the hip node). This is then used for fitting (e.g., using the least squares method for fitting) to obtain the thermal zone temperature variance slope values ​​of several groups of nodes. These values ​​are then weighted (and in this process, the edge weight corresponding to each group of nodes is used as its corresponding weight coefficient). In this process, the weighted result is transformed using the reciprocal suppression mapping function f(x)=1 / (1+x), i.e., 1 / (1+the weighted result), to extract thermal posture coordination features, which are used to characterize the thermal fluctuation conduction coordination of the main thermal zones of the human body in the time dimension. When the human body posture is stable and the movement of each part is coordinated, the temperature variance change trend of adjacent thermal zones is consistent, the variance slope of each node pair is small, the weighted result is low, and the feature obtained after the reciprocal mapping is high. The centroid coordinates of the thermal zone of each node in continuous time sampling frames are tracked. The displacement change rate of each node (taking the torso node as an example, the difference of the centroid coordinates of the thermal zone between two adjacent time sampling frames can be obtained by using the Euclidean distance formula to obtain the displacement value between the two adjacent time sampling frames, and the ratio is processed with the time interval between the two adjacent time sampling frames and the average value is taken) and the average temperature change rate of the thermal zone are extracted. The root mean square processing is performed to obtain the root mean square value of displacement change and the root mean square value of average temperature change of the thermal zone. The weighted processing is then performed to extract thermal posture migration features, which are used to characterize the synchronicity of the position change and thermal energy change of the main thermal zones of the human body in the time dimension. When a node undergoes significant displacement and is accompanied by rapid temperature fluctuation, the thermal posture migration feature value increases significantly, indicating that the body part has undergone significant posture adjustment or thermal energy disturbance behavior. For each node (taking the torso node as an example), the centroid coordinates of the hot zone of that node are tracked in continuous time sampling frames. The change in longitudinal displacement of the centroid between two adjacent time sampling frames is calculated, and the change in longitudinal displacement is divided by the time interval to obtain several sets of longitudinal displacement change rates between two adjacent time sampling frames. The maximum value of the longitudinal displacement change is extracted. At the same time, the temperature variance between two adjacent time sampling frames of that node is processed by difference, and the temperature variance change amplitude value is extracted (this value is the absolute value of the smallest result among all negative results in the difference processing results). The average pressure value of each time sampling frame corresponding to that node is used to extract the pressure change amplitude value (with the same logic as the temperature variance change amplitude value extraction). This value is then weighted and averaged with the maximum longitudinal displacement change value and the temperature variance change amplitude value to extract the tipping risk feature, which is used to characterize the degree of fall risk. The thermal posture coordination feature, thermal posture migration feature, and tipping risk feature are then concatenated into an attitude feature vector. In the output layer of the thermal posture perception model, based on the posture feature vector of the patient to be examined at night, the posture mapping feature set of the patient to be examined at night is output. Specifically, the thermal posture coordination feature, thermal posture migration feature, and tipping risk feature in the posture feature vector are activated by the Sigmoid function to obtain thermal posture coordination feature value, thermal posture migration feature value, and tipping risk feature value between 0 and 1.

[0042] The pre-training steps of the hot-press attitude perception model are as follows: A labeled dataset was obtained, consisting of time-series thermal imaging data and time-series bed pressure distribution data collected from several subjects in a nighttime nursing setting. The data acquisition environment maintained constant room temperature and stable lighting conditions to avoid interference from external heat sources. The labeled data was manually annotated by nursing experts and posture monitoring personnel based on synchronously collected real posture images and body surface contact conditions. Each sample in the labeled dataset includes thermal imaging pixel matrices of the patient's head, trunk, buttocks, and lower limbs, corresponding pressure matrices, and posture labels (including categories such as supine, lateral, turning over, and falling) from multiple consecutive time sampling frames. At the same time, the patient's time-series state characteristics (such as the centroid coordinates of the thermal zone, average temperature value, and average pressure value) were recorded to form a complete training sample.

[0043] The labeled dataset is preprocessed, including: noise suppression and background removal on the thermal imaging data, normalization of temperature values; zero-point correction and standardization of the pressure matrix; one-hot encoding of the label data to adapt to the model output format. The preprocessed dataset is then divided into training, validation and test sets, for example, in a ratio of 80%, 10% and 10%, and the data is arranged in chronological order to ensure that the model learns the dynamic dependencies and attitude change patterns of the time series.

[0044] During pre-training, each layer of the model is trained sequentially: the input layer is responsible for the joint alignment of multimodal data; the semantic node recognition layer automatically learns the thermal region distribution features and stress pattern features through a convolutional neural network (CNN) and outputs semantic node feature vectors; the spatiotemporal convolutional layer achieves feature fusion of spatial and temporal information through a joint structure of graph convolutional network (GCN) and one-dimensional convolution (Conv1D) to capture the coordinated temperature changes and displacement trends of thermal regions. During training, the cross-entropy loss function and Adam optimization algorithm are used, and the network parameters are updated based on the backpropagation mechanism to minimize the error between the predicted pose and the true label.

[0045] During training, the learning rate and weight decay parameters are dynamically adjusted to prevent overfitting. The validation set is used to monitor the model's performance at different stages, and hyperparameters (such as kernel size, number of hidden layer nodes, and Dropout ratio) are optimized to improve the model's generalization ability and stability. After training, the model is evaluated using a test set to verify its pose recognition accuracy, fall detection sensitivity, and hot zone matching accuracy on unseen data. This ensures that the model can stably output pose mapping feature sets under diverse patient conditions. Finally, the trained model weights and parameters are saved for subsequent online inference stages to achieve real-time recognition and evaluation of the pose and thermal state of patients undergoing nighttime inspections.

[0046] In this implementation scheme, the collaborative identification of human posture and thermal energy dynamics is achieved through the fusion and spatiotemporal modeling of thermal imaging time-series data and pressure distribution time-series data. Secondly, the semantic node recognition layer is used to uniformly map thermal imaging and pressure into structured thermal zone feature vectors, realizing adaptive partitioning of major thermal zones. Next, the spatiotemporal convolutional layer constructs a human thermal topology chain structure, incorporating the temperature equations of thermal zones into the graph convolutional analysis framework, thereby capturing the dynamic correlation and conduction delay features between body parts. Thus, thermal posture coordination features reflect body balance, thermal posture migration features reveal the synchronicity of thermal energy and displacement, and the fall risk features further integrate longitudinal displacement, thermal disturbance, and pressure mutation to identify fall events. Finally, the thermal posture steady-state feature value, through multi-feature fusion, can reflect the overall posture stability of the patient within a set period, thereby achieving continuous tracking and stable assessment of the patient's nocturnal posture changes, thus improving the intelligent response capability and safety assurance level of night rounds.

[0047] Please see Figure 4 This invention provides a technical solution: a non-contact intelligent nursing night round system based on multimodal infrared sensing, comprising: a data acquisition module for continuously acquiring thermal imaging time-series data and pressure distribution time-series data of patients to be examined within a set period; a thermal circulation analysis module for analyzing the thermal circulation stability characteristic values ​​of patients to be examined based on the thermal imaging time-series data; a thermal posture analysis module for analyzing the thermal posture steady-state characteristic values ​​of patients to be examined based on a pre-trained thermal pressure posture perception model, combined with the thermal imaging time-series data and pressure distribution time-series data; a comprehensive analysis module for fusing the thermal circulation stability characteristic values ​​and thermal posture steady-state characteristic values ​​of patients to be examined to obtain the thermal imbalance characteristic values ​​of patients to be examined; and a ward round nursing feedback module for performing ward round nursing care based on the thermal imbalance characteristic values ​​of patients to be examined.

[0048] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0049] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A non-contact intelligent nursing night round method based on multi-modal infrared perception, characterized in that, The method comprises the following steps: acquiring thermal imaging time series data and pressure distribution time series data of the patient to be checked at night in a set period; analyzing thermal cycle stability characteristic values of the patient to be checked at night based on the thermal imaging time series data of the patient to be checked at night; analyzing thermal posture stability characteristic values of the patient to be checked at night based on the pre-trained thermal pressure posture perception model and in combination with the thermal imaging time series data and the pressure distribution time series data of the patient to be checked at night, and fusing the thermal cycle stability characteristic values to obtain thermal imbalance characteristic values of the patient to be checked at night; performing ward nursing treatment on the patient to be checked at night based on the thermal imbalance characteristic values.

2. The non-contact smart care night rounding method based on multi-modal infrared perception according to claim 1, characterized in that, The thermal imaging time series data comprises a plurality of time sampling frame thermal imaging data, and each time sampling frame thermal imaging data is specifically a temperature value and a two-dimensional coordinate of each pixel point.

3. The non-contact smart care night rounding method based on multi-modal infrared perception of claim 2, wherein, The specific steps of analyzing the thermal cycle stability characteristic values of the patient to be checked at night are as follows: analyzing thermal state characteristic sets of the patient to be checked at night based on the thermal imaging time series data of the patient to be checked at night, wherein the thermal state characteristic sets comprise body surface heat distribution characteristic values and respiratory heat flow characteristic values; analyzing thermal cycle stability characteristic values of the patient to be checked at night based on the thermal state characteristic sets of the patient to be checked at night.

4. The non-contact smart care night rounding method based on multi-modal infrared perception according to claim 3, characterized in that, The specific steps of analyzing the thermal state characteristic sets of the patient to be checked at night are as follows: preprocessing each time sampling frame thermal imaging data of the patient to be checked at night; performing regional division processing on the preprocessed each time sampling frame thermal imaging data of the patient to be checked at night to obtain pixel point sets of a plurality of thermal regions in each time sampling frame thermal imaging of the patient to be checked at night; and deconstructing each thermal region pixel point set in each time sampling frame thermal imaging of the patient to be checked at night to obtain body surface heat distribution characteristic values and respiratory heat flow characteristic values of the patient to be checked at night.

5. The non-contact smart care night rounding method based on multi-modal infrared perception of claim 1, wherein, The pressure distribution time series data is specifically a pressure value of each position of each time sampling frame, and the thermal pressure posture perception model comprises an input layer, a feature extraction layer, a space-time convolution layer and an output layer.

6. The non-contact smart care night rounding method based on multi-modal infrared perception according to claim 5, characterized in that, The specific steps of analyzing the thermal posture stability characteristic values of the patient to be checked at night are as follows: inputting the thermal imaging time series data and the pressure distribution time series data of the patient to be checked at night into the pre-trained thermal pressure posture perception model to analyze posture mapping characteristic sets of the patient to be checked at night, wherein the posture mapping characteristic sets comprise thermal posture coordination characteristic values, thermal posture migration characteristic values and toppling risk characteristic values; analyzing thermal posture stability characteristic values of the patient to be checked at night based on the posture mapping characteristic sets of the patient to be checked at night.

7. The non-contact smart care night rounding method based on multi-modal infrared perception according to claim 6, characterized in that, The specific steps of analyzing the posture mapping characteristic sets of the patient to be checked at night are as follows: in the input layer of the thermal pressure posture perception model, receiving the thermal imaging time series data and the pressure distribution time series data of the patient to be checked at night and performing data preprocessing; in the semantic node recognition layer of the thermal pressure posture perception model, performing recognition processing on the data preprocessed thermal imaging time series data and pressure distribution time series data of the patient to be checked at night to extract thermal region feature vectors of each time sampling frame of the patient to be checked at night; in the space-time convolution layer of the thermal pressure posture perception model, extracting posture feature vectors of the patient to be checked at night based on the thermal region feature vectors of each time sampling frame of the patient to be checked at night; in the output layer of the thermal pressure posture perception model, outputting posture mapping characteristic sets of the patient to be checked at night based on the posture feature vectors of the patient to be checked at night.

8. The multimodal infrared perception based non-contact smart nursing night round method according to claim 1, characterized in that, The specific formula for calculating the thermal imbalance characteristic values of the patient to be checked at night is as follows: ； wherein, a thermal imbalance characteristic value for the patient to be checked at night, a thermal cycle stability characteristic value for the patient to be checked at night, a thermal cycle adjustment coefficient stored in the database, a thermal posture stability characteristic value for the patient to be checked at night, a thermal posture stability adjustment coefficient stored in the database, a difference adjustment coefficient stored in the database, .

9. The multimodal infrared perception based non-contact smart nursing round of claim 1, wherein, The specific steps of the nursing treatment of the patient to be checked at night based on the heat imbalance characteristic value are as follows: The heat imbalance characteristic value of the patient to be checked at night is compared with the preset heat imbalance characteristic threshold value; Based on the comparison result, the patient to be checked at night is nursed.

10. A non-contact intelligent nursing night round system based on multi-modal infrared perception, applying the non-contact intelligent nursing night round method based on multi-modal infrared perception in any one of claims 1-9, characterized in that, Comprise: The data acquisition module is used for continuously acquiring the heat imaging time series data and the pressure distribution time series data of the patient to be checked at night within a set period; The heat cycle analysis module is used for analyzing the heat cycle stability characteristic value of the patient to be checked at night based on the heat imaging time series data of the patient to be checked at night; The heat posture analysis module is used for analyzing the heat posture stability characteristic value of the patient to be checked at night based on the pre-trained heat pressure posture perception model and in combination with the heat imaging time series data and the pressure distribution time series data of the patient to be checked at night; The comprehensive analysis module is used for fusing the heat cycle stability characteristic value and the heat posture stability characteristic value of the patient to be checked at night to obtain the heat imbalance characteristic value of the patient to be checked at night; The ward nursing feedback module is used for nursing the patient to be checked at night based on the heat imbalance characteristic value.

Images