Pressure damage monitoring method and device based on multi-modal data fusion

Abstract

The invention relates to the field of medical instrument detection, in particular to a pressure damage monitoring method and device based on multi-modal data fusion, and the method comprises the steps: carrying out the preprocessing of collected pressure, temperature and humidity data, and obtaining a pressure difference map, a pressure map entropy temperature feature and a humidity feature; taking the pressure difference map as input, and performing feature extraction and dimension reduction processing by using a convolutional neural network to obtain a one-dimensional feature vector; splicing the one-dimensional feature vector, the pressure map entropy, the temperature feature and the humidity feature in each time window to generate a local fusion feature vector; splicing all the local fusion feature vectors to obtain a global feature vector; taking the global feature vector as input, and processing by using an LSTM network to obtain a final fusion feature vector; and inputting the final fusion feature vector into a classifier for classification to obtain a stress damage staging result. The method solves the ubiquitous problem that in the prior art, judgment is only based on static pressure, and the tissue state cannot be comprehensively reflected.

Description

Technical Field

[0001] The present invention relates to the field of medical device detection, and in particular to a pressure injury monitoring method and device based on multimodal data fusion. Background Art

[0002] Pressure injuries (commonly known as bedsores) are common and serious complications in clinical nursing. Their pathophysiological mechanism is essentially a synergistic effect of pressure, temperature, and humidity. In bedridden patients (such as the elderly, those with neurological impairments, or those in the ICU), when local tissue is subjected to continuous vertical pressure, capillary blood flow is obstructed, leading to ischemia and hypoxia. Simultaneously, the accumulation of metabolic waste in the pressured area triggers an inflammatory response, causing an abnormal increase in local temperature. High humidity accelerates stratum corneum hydration. These three factors together lead to a breakdown of the skin's barrier function, ultimately causing irreversible tissue damage.

[0003] In the prior art, patent publication number CN221489942U proposes a pressure injury monitoring device that implements monitoring functions by placing a pressure sensor in the interlayer of a blanket and combining it with a controller. However, its functionality is relatively simple, lacking comprehensive assessment of other key factors, and its low level of intelligence makes it difficult to meet the clinical needs for precise pressure ulcer prevention. Patent publication number CN119655712A collects pressure data through a pressure sensor, extracts eigenvalues ​​after preprocessing, and constructs a multi-level feature matrix. Ultimately, a model is used to determine the status of the pressure ulcer. However, this method, based solely on pressure data, fails to fully reflect the various factors that cause pressure ulcers and is difficult to accurately predict the occurrence of pressure ulcers in complex situations. Patent publication number CN112370271A proposes a real-time dynamic prevention and detection system based on pressure sensors. This system uses pressure sensors and temperature and humidity detection modules on the mattress to acquire data and performs predictive analysis using dynamically converted detection rules. However, the system's dynamic conversion method is relatively simple, lacking sufficient flexibility and generalization, making it difficult to adapt to different patients and complex and diverse clinical application scenarios. Summary of the Invention

[0004] To overcome the above-mentioned defects of current pressure injury monitoring equipment, the present invention provides a pressure injury monitoring method and device based on multimodal data fusion. It takes pressure, temperature and humidity sensor data as input, adopts a multimodal deep learning algorithm to obtain pressure distribution, temperature and humidity change information, and realizes real-time monitoring and early warning of pressure injuries, so as to help nursing staff take preventive measures in time.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A pressure injury monitoring method based on multimodal data fusion includes the following steps:

[0007] S1. Obtaining pressure, temperature and humidity data of the patient's compressed skin location;

[0008] S2. Preprocess the pressure, temperature and humidity data respectively to obtain the pressure difference map ΔP, pressure map entropy S, temperature characteristics T norm and humidity characteristics H norm ;

[0009] S3, using the pressure difference map ΔP as input, using the convolutional neural network to extract the features of the pressure difference map ΔP and reduce the dimension to obtain a one-dimensional feature vector P d ;

[0010] S4. For each one-dimensional feature vector P in the time window d , pressure map entropy S, temperature characteristics T norm and humidity characteristics H norm Splicing is performed to generate a local fusion feature vector; then all the local fusion feature vectors are spliced ​​end to end in sequence to obtain the feature vector within the set time period, that is, the global feature vector; the global feature vector is used as input and processed using the LSTM network to obtain the final fusion feature vector;

[0011] S5. Input the final fused feature vector into the classifier for classification to obtain the pressure injury staging result.

[0012] Furthermore, the step S2 pre-processes the pressure, temperature and humidity data respectively to obtain the pressure difference graph ΔP, the temperature characteristic T norm and humidity characteristics H norm ; Its implementation process includes the following steps:

[0013] S2.1. Using temperature data or humidity data as a benchmark, use the missing high-frequency data method to make the number of temperature, humidity, and pressure data the same;

[0014] S2.2. Divide the data obtained in step S2.1 into multiple time windows within a set time period, each containing pressure, temperature, and humidity data. Calculate the pressure data of adjacent time windows for the pressure data within consecutive time windows to obtain a pressure difference graph ΔP, also known as a pressure data change graph.

[0015] S2.3. Calculate the average values ​​of two adjacent time windows for the temperature and humidity data respectively, and adjust the data sequence so that it is aligned with the pressure difference graph ΔP in the time dimension; then normalize the temperature data to obtain the temperature feature T norm , normalize the humidity data to obtain the humidity feature H norm .

[0016] Furthermore, the convolutional neural network, LSMT neural network and classifier are optimized using a loss function, which consists of two parts: cross entropy loss and dynamic pressure consistency loss.

[0017] The cross entropy loss is:

[0018]

[0019] Where N is the total number of pressure injury staging categories, y i is the true label, is the probability distribution predicted by the model;

[0020] The dynamic pressure consistency loss is:

[0021] in, is the square of the vector norm, P(t) refers to the eigenvector extracted from the t-th pressure difference map, and P(t+1) refers to the eigenvector extracted from the t+1-th pressure difference map.

[0022] The total loss function is:

[0023] Loss=λ1Loss c +λ2Loss d ;

[0024] Among them, λ1 and λ2 are weight coefficients.

[0025] Furthermore, the above-mentioned pressure injury monitoring method based on multimodal data fusion also includes:

[0026] Based on the pressure injury staging result obtained in step S5, personalized nursing recommendations are generated to reduce the risk of pressure injuries.

[0027] Furthermore, the classifier consists of a first fully connected layer, a first ReLU activation function layer, a second fully connected layer, a second ReLU activation function layer, a third fully connected layer and a Softmax function layer, which are connected in sequence.

[0028] A pressure injury monitoring device based on multimodal data fusion includes a data acquisition module, a data preprocessing module, a feature extraction and fusion module, and a staging module. The above modules are used to implement a pressure injury monitoring method based on multimodal data fusion.

[0029] This invention, based on the pathophysiological mechanisms of pressure injuries, addresses the common problem of existing technologies that rely solely on static pressure assessments and fail to fully reflect tissue status. It proposes a monitoring method and device that integrates multimodal data from pressure, temperature, and humidity. This method uses dynamic change capabilities as the basis for health assessment, models the temporal response of tissues through pressure difference graph sequences, and introduces a multimodal feature fusion mechanism to improve the accuracy of identifying high-risk areas. The details are as follows:

[0030] In terms of data acquisition and preprocessing, this paper uses multimodal sensors to synchronously collect pressure, temperature, and humidity data. During the preprocessing phase (step S2), a cross-modal time alignment algorithm is used to eliminate differences in data sampling frequencies between different sensors. A sliding window approach is then used to generate temporally continuous feature data segments. This process ensures consistency in the temporal dimension for subsequent feature extraction and fusion.

[0031] In terms of feature modeling, this paper proposes, for the first time, using a sequence of pressure difference maps (ΔP) as input, combining a convolutional neural network (CNN) and a recurrent neural network (LSTM) to extract dynamic temporal features that reflect the temporal trends in tissue compression. Simultaneously extracted temperature and humidity features reflect tissue metabolism and environmental influences. These three features are concatenated at a local fusion layer to form a joint representation vector, which is then used by a subsequent classifier to determine the risk level of pressure injuries.

[0032] The present invention also designs a dynamic consistency regularized loss function to optimize the pressure feature extraction process. This loss function, constructed based on the second-order difference of the pressure feature vector sequence, aims to characterize the dynamic response of the skin region per unit time. By encouraging the model to capture areas with significant fluctuations while suppressing misjudgments of areas with slow dynamic changes, the model improves its sensitivity to early signs of pressure ulcers and its predictive accuracy.

[0033] In summary, the present invention achieves a more comprehensive and dynamic assessment of skin tissue health status through improvements in sensor data organization, feature extraction path, and loss function structure, and is suitable for complex and changeable clinical care scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a pressure injury monitoring method based on multimodal data fusion according to an embodiment. DETAILED DESCRIPTION

[0035] During the formation of ischemic injuries, concentrated pressure leads to capillary occlusion (mechanical factors), local metabolic heat production causes temperature to rise (physiological reaction), and a high humidity environment accelerates the softening of the stratum corneum (environmental factors). Traditional monitoring equipment only focuses on static pressure distribution, ignoring that the essence of pressure injuries is the result of the dynamic coupling of three factors: pressure, temperature, and humidity. This makes it difficult to make a comprehensive assessment of the pressure ulcer situation without accurate monitoring results. To this end, this embodiment proposes a pressure injury monitoring method and device based on multimodal data fusion based on the pathophysiology of pressure injuries.

[0036] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0037] like Figure 1 As shown, this embodiment provides a pressure injury monitoring method based on multimodal data fusion, comprising the following steps:

[0038] S1. Obtaining pressure, temperature, and humidity data at the patient's skin pressure location. This embodiment uses a 32×32 dot matrix pressure sensor to collect pressure data. Temperature and humidity sensors are distributed within the pressure sensor array. This arrangement enables simultaneous acquisition of pressure, temperature, and humidity data.

[0039] S2. Preprocess the pressure, temperature and humidity data respectively to obtain the pressure difference graph ΔP and the pressure graph entropy S mean , temperature characteristics T norm and humidity characteristics H norm This embodiment uses a cross-modal time alignment algorithm to solve the synchronization problem of multi-source heterogeneous data, ensuring the consistency of subsequent fusion features in the time dimension and laying the foundation for dynamic analysis. The specific steps are as follows:

[0040] S2.1. Using temperature data or humidity data as a benchmark, use the missing high-frequency data method to align the temperature, humidity, and pressure data to the same timestamp so that the three data have the same number.

[0041] S2.2. Divide the data obtained in step S2.1 into 10 time windows within a set time period of 1 second, with each time window containing pressure, temperature, and humidity data. The pressure data in consecutive time windows are calculated for the pressure data in adjacent time windows to obtain a pressure difference graph ΔP, which is also called a pressure data change graph. The calculation formula for the pressure difference graph ΔP is:

[0042] ΔP=P(t+1)-P(t);

[0043] Wherein, P(t+1) represents the pressure data of the t+1 time window.

[0044] S2.3. Introduce the definition of Shannon entropy and define the entropy of the pressure map in each time window as:

[0045]

[0046] Among them, P i,j Represents the pressure data of the i-th row and j-th column of the pressure map. In order to align with other feature data, it is necessary to take the average value S of the pressure map entropy calculated in two consecutive time windows. mean There is no need to normalize the entropy here because the calculated entropy value is already in the range of (0,1).

[0047] S2.4. Calculate the average values ​​of two adjacent time windows for the temperature and humidity data, and adjust the data sequence so that it is aligned with the pressure difference graph ΔP in the time dimension. Then normalize the temperature data to obtain the temperature feature T norm , normalize the humidity data to obtain the humidity feature H mean ; The formula for normalizing temperature data is:

[0048]

[0049] Among them, T mean represents the average temperature of two time windows, T max Indicates the maximum body temperature that may occur in the pressure injury area of ​​the human body. Based on nursing experience data, it is set to 45°C in this example; T min This value represents the minimum body temperature that may occur in the pressure injury area of ​​the human body. It is derived from nursing experience data and is set to 35°C in this example.

[0050] The formula for normalizing humidity data is:

[0051]

[0052] Among them, H mean Represents the average humidity of two time windows, H max Indicates the maximum humidity that may occur in the pressure injury area of ​​the human body. Based on nursing experience data, this example is set to 85%; H min Indicates the minimum humidity that may occur in the pressure injury area of ​​the human body. It is obtained based on nursing experience data and is set to 25% in this example.

[0053] S3, taking the pressure difference map ΔP as input, using convolutional neural network to extract the pressure difference map ΔP through CNN feature extraction and dimensionality reduction to obtain a one-dimensional feature vector P d , where the range of each dimension of the eigenvector is (0,1).

[0054] S4. For each one-dimensional feature vector P in the time window d , pressure map entropy S mean , temperature characteristics T norm 、Humidity characteristics H mean And splicing to generate local fusion feature vector F w (t)=[P d ,S mean ,T norm ,H norm ]; Then all the local fusion feature vectors are spliced ​​together in sequence to obtain the feature vector within the set time period, that is, the global feature vector; then the global feature vector is used as input and processed using the LSTM network to obtain the final fusion feature vector F within the time period fused =LSTM([F w (1),F w (2),...,F w (9)]).

[0055] S5. Input the final fusion feature vector into the classifier for classification to obtain the pressure injury staging result. The classifier is composed of a first fully connected layer, a first ReLU activation function layer, a second fully connected layer, a second ReLU activation function layer, a third fully connected layer and a Softmax function layer connected in sequence. The first fully connected layer receives the final fusion feature vector F fused After dimensionality reduction and feature extraction in the second and third fully connected layers, the Softmax function converts the output into a probability distribution. Pressure injury staging is then derived based on this distribution probability. Reluctant Unit (ReLU) activation function layers are introduced between the first and second fully connected layers, and between the second and third fully connected layers, primarily to enhance the model's expressiveness.

[0056] In the above-described embodiment, steps S3 and S4 are the core steps. The pressure difference map ΔP is extracted through CNN to capture the local pressure gradient changes in the pressure area. The entropy S of the pressure map reflects whether the pressure injury area is concentrated. The temperature T norm and humidity H norm The normalized features of reflect the metabolic state of tissue in the pressure injury area and the influence of environmental humidity. The local fusion feature vector F w By integrating all key indicators, the LSTM network further explores temporal dependencies, enabling the model to identify risk accumulation patterns such as "a slowdown in pressure signal fluctuation frequency due to pressure injury."

[0057] In order to improve the accuracy of the model, this embodiment proposes a new loss function to optimize the convolutional neural network, LSTM neural network and classifier; the loss function proposed in this embodiment is composed of cross entropy loss and dynamic pressure consistency loss Loss d Composition, of which:

[0058] Cross entropy loss is a standard loss function that measures the difference between the model prediction results and the true labels. Its calculation formula is as follows:

[0059]

[0060] Where N is the total number of pressure injury staging categories, y i is the true label, is the probability distribution predicted by the model;

[0061] The dynamic pressure consistency loss explicitly captures the dynamic changes in pressure data in the temporal dimension, such as fluctuations caused by heartbeats and breathing, by calculating the pressure difference map between adjacent time windows. This approach not only enhances the model's sensitivity to physiological signals but also suppresses noise interference, thereby improving the performance and robustness of classification tasks. The calculation formula for the dynamic pressure consistency loss is as follows:

[0062]

[0063] in, is the square of the two-norm, P(t) refers to the eigenvector extracted from the t-th pressure difference map, and P(t+1) refers to the eigenvector extracted from the t+1-th pressure difference map.

[0064] Finally, the total damage function is obtained

[0065] Loss=λ1Loss c +λ2Loss d ;

[0066] Among them, λ1 and λ2 are weight coefficients.

[0067] This example uses a graded assessment of pressure injury risk to categorize the results into three levels: low risk, medium risk, and high risk. Specific nursing recommendations are provided for each level:

[0068] Low risk: All data are within normal range. It is recommended to regularly check the patient's pressure points, keep the skin clean and dry, and adjust the body position every 2 hours to prevent potential risks.

[0069] Medium risk: Local pressure exceeding the standard value is detected, accompanied by a slight increase in temperature and humidity. It is recommended to increase the frequency of body position changes to once every hour, use a pressure relief mat to relieve pressure, and strengthen local skin care to ensure dry skin.

[0070] High risk: Low entropy values ​​on the pressure map indicate concentrated pressure in specific areas, along with significant increases in temperature and humidity. Immediate intervention is recommended, such as the use of an air mattress or dynamic pressure relief device, and notification of healthcare professionals for further evaluation to check for inflammation or infection.

[0071] In addition, during the entire staging process, this implementation can also provide corresponding suggestions based on the real-time collected data:

[0072] When the collected pressure data is higher than the standard value, it is recommended to adjust the patient's position and use a pressure relief pad to reduce local pressure;

[0073] When the collected temperature data is higher than the standard value: it is recommended to reduce the temperature locally and check for inflammation; if the temperature is low: it is recommended to increase warming measures and improve blood circulation;

[0074] When the collected humidity data is higher than the standard value: it is recommended to keep the skin dry and use a moisture-absorbing dressing; when the humidity is too low: it is recommended to use a moisturizing cream to prevent the skin from drying and cracking, and to increase the ambient humidity;

[0075] When the collected pressure, temperature and humidity data are all higher than the standard values, it means that the comprehensive indicators are abnormal. At this time, medical staff need to be notified.

[0076] In summary, this embodiment provides accurate and dynamic nursing advice through real-time monitoring and multimodal data fusion, reduces the risk of pressure injuries, and improves nursing efficiency.

[0077] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A pressure injury monitoring method based on multimodal data fusion, characterized in that: The following steps are involved: S1. Obtaining pressure, temperature and humidity data of the patient's compressed skin location; S2. Preprocess the pressure, temperature and humidity data respectively to obtain the pressure difference map ΔP, pressure map entropy S, temperature characteristics T norm and humidity characteristics H norm ; S3, using the pressure difference map ΔP as input, using the convolutional neural network to extract the features of the pressure difference map ΔP and reduce the dimension to obtain a one-dimensional feature vector P d ; S4. For each one-dimensional feature vector P in the time window d , pressure map entropy S, temperature characteristics T norm and humidity characteristics H norm Splicing is performed to generate a local fusion feature vector; then all the local fusion feature vectors are spliced ​​end to end in sequence to obtain the feature vector within the set time period, that is, the global feature vector; the global feature vector is used as input and processed using the LSTM network to obtain the final fusion feature vector; S5. Input the final fused feature vector into the classifier for classification to obtain the pressure injury staging result.

2. A pressure injury monitoring method based on multimodal data fusion according to claim 1, characterized in that: The step S2 pre-processes the pressure, temperature and humidity data respectively to obtain the pressure difference graph ΔP, the temperature characteristic T norm and humidity characteristics H norm ; Its implementation process includes the following steps: S2.

1. Using temperature data or humidity data as a benchmark, use the missing high-frequency data method to make the number of temperature, humidity, and pressure data the same; S2.

2. Divide the data obtained in step S2.1 into multiple time windows within a set time period, each containing pressure, temperature, and humidity data. Calculate the pressure data of adjacent time windows for the pressure data within consecutive time windows to obtain a pressure difference graph ΔP, also known as a pressure data change graph. S2.

3. Calculate the average values ​​of two adjacent time windows for the temperature and humidity data respectively, and adjust the data sequence so that it is aligned with the pressure difference graph ΔP in the time dimension; then normalize the temperature data to obtain the temperature feature T norm , normalize the humidity data to obtain the humidity feature H norm .

3. The pressure injury monitoring method based on multimodal data fusion according to claim 2, characterized in that: The convolutional neural network, LSMT neural network and classifier are optimized using a loss function. The loss function consists of two parts: cross entropy loss and dynamic pressure consistency loss. The cross entropy loss is: Where N is the total number of pressure injury staging categories, y i is the true label, is the probability distribution predicted by the model; The dynamic pressure consistency loss is: in, is the square of the vector norm, P(t) refers to the eigenvector extracted from the t-th pressure difference map, and P(t+1) refers to the eigenvector extracted from the t+1-th pressure difference map. The total loss function is: Loss=λ1Loss c +λ2Loss d ; Among them, λ1 and λ2 are weight coefficients.

4. The pressure injury monitoring method based on multimodal data fusion according to claim 1, characterized in that: The pressure injury monitoring method based on multimodal data fusion also includes: Based on the pressure injury staging result obtained in step S5, personalized nursing recommendations are generated to reduce the risk of pressure injuries.

5. The pressure injury monitoring method based on multimodal data fusion according to claim 1, characterized in that: The classifier consists of a first fully connected layer, a first ReLU activation function layer, a second fully connected layer, a second ReLU activation function layer, a third fully connected layer and a Softmax function layer connected in sequence.

6. A pressure injury monitoring device based on multimodal data fusion, comprising a data acquisition module, a data preprocessing module, a feature extraction and fusion module, and a staging module, wherein the above modules are used to implement a pressure injury monitoring method based on multimodal data fusion as described in any one of claims 1 to 5.

Images