Early pressure injury warning method based on multi-modal hemodynamic sensing and analysis

Abstract

The present application relates to the technical field of medical auxiliary diagnosis and wearable / non-contact health monitoring, and particularly relates to an early pressure injury early warning method based on multi-modal hemodynamics sensing and analysis, which is characterized by designing a data acquisition device in a video stream acquisition process, and constructing a polarized acquisition light path by using a linear polarizer and a polarizer in the acquisition device, so as to reduce light path noise and environmental interference, and improve data acquisition accuracy. In the physiological feature extraction process, the physiological feature extraction is completed by calculating the relative perfusion rate and the signal complexity, the anti-interference ability and the robustness of the model in the extreme strong noise environment are greatly improved, the diagnosis failure problem frequently occurring in the traditional pure visual model is avoided, and the decision weight can be dynamically and smoothly migrated to the subcutaneous hemodynamics branch which is not affected by light changes and body surface stain interference. The method of the present application provides a valuable early intervention time window and visual clinical decision support for medical personnel before the lesion causes irreversible tissue ulceration.

Description

Technical Field

[0001] This invention relates to the fields of medical auxiliary diagnosis and wearable / non-contact health monitoring technology, specifically to an early warning method for pressure injury based on multimodal hemodynamic sensing and analysis. Background Technology

[0002] Pressure injuries (PIs) are a major challenge in aged care worldwide, placing a heavy burden on healthcare systems. Early detection of injuries, such as stage I pressure ulcers, is crucial for timely intervention and can increase healing rates to over 80%. In clinical practice, nurses typically rely on visual inspection and palpation to assess skin integrity and reactive hyperemia, but this method is time-consuming, labor-intensive, and susceptible to subjective experience, leading to a high risk of misdiagnosis. While high-frequency ultrasound (HFUS) can identify deep tissue damage beneath the epidermis, its high equipment cost and reliance on skilled operators make it unsuitable for routine high-frequency monitoring in home care or aged care facilities.

[0003] In recent years, deep learning-based computer-aided diagnostic techniques have made some progress in the automated diagnosis of pressure ulcers. However, existing research still has some significant shortcomings: most methods only use RGB single-modal clinical images, simplifying pressure ulcer diagnosis to a purely visual texture recognition problem. This method, which relies excessively on surface features, still has inherent limitations when distinguishing morphologically similar lesions, such as reactive hyperemia from stage I pressure ulcers. Specifically:

[0004] 1. Disconnect between internal and external pathology: Both reactive hyperemia (RH) and stage I pressure ulcers (S1) present as erythema on the skin surface. The pathological difference between the two lies in the microcirculatory hemodynamic characteristics, such as whether there is whitening and fading. Static RGB images cannot capture the dynamic response of subcutaneous blood flow, so the detection accuracy needs to be improved.

[0005] 2. The insidious nature of deep tissue pressure injury (DTPI): Deep tissue pressure injury (DTPI) often presents as an intact skin surface, but with ischemic necrosis of the subcutaneous soft tissue. Models that rely solely on surface visual information are prone to overfitting superficial noise artifacts and neglecting deep features that reflect tissue activity, thus leading to ambiguous diagnostic criteria and insufficient accuracy.

[0006] While traditional ultrasound equipment can detect subcutaneous tissue, it is expensive and requires professional personnel, making it unsuitable for routine high-frequency monitoring in homes or nursing homes.

[0007] Therefore, there is an urgent need for a non-contact, low-cost, multimodal auxiliary diagnostic system that can combine surface morphology and subcutaneous hemodynamic information. Summary of the Invention

[0008] The purpose of this invention is to provide an early warning method for pressure injury based on multimodal hemodynamic sensing and analysis. By introducing remote photoplethysmography (rPPG) as a deep sensing marker of subcutaneous tissue vitality, a physiological guidance network is constructed to achieve accurate identification of early pressure ulcers, reactive hyperemia, and deep tissue damage.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] An early warning method for pressure injury based on multimodal hemodynamic sensing and analysis includes the following steps:

[0011] Step 1. Video stream acquisition: Use a data acquisition device to acquire the raw video stream of the subject's pressure ulcer-prone areas;

[0012] Step 2. Dual-branch feature extraction: The dual-branch approach consists of a visual feature branch and a physiological feature branch. The visual feature branch extracts visual features, while the physiological feature branch extracts physiological features. The feature extraction process for each branch is as follows:

[0013] a. Visual Branch: The RGB image data of the original video stream is used as input. Spatial visual representations are extracted by truncating the ResNet-18 backbone network. The feature map output by the last convolutional stage of the ResNet-18 backbone network is obtained and global pooled to generate visual features.

[0014] b. The physiological branch extracts physiological features through the following sub-steps:

[0015] b1. Preprocessing the video stream: Based on an adaptive threshold, the region of interest (ROI) of the lesion and the surrounding normal skin ROI are delineated in the initial frame of the video; motion estimation is performed on subsequent frames of the video; based on the estimated motion information, frame-by-frame affine transformation is performed on the boundary between the delineated ROI and the surrounding normal skin ROI to realize the dynamic update of the spatial position of the image data of the ROI and the surrounding normal skin ROI in each frame of the video.

[0016] b2. Based on step b1, rPPG-BVP (blood flow volume pulse wave) signals are extracted from the image data of the lesion region of interest and the surrounding normal skin region of interest after the dynamic update in step 2, respectively, to obtain the original rPPG-BVP signal and perform filtering processing. Then, physiological features x_phy are extracted from the filtered rPPG-BVP signal; the physiological features are multi-dimensional physiological features including time domain features, frequency domain features and nonlinear dynamic features.

[0017] Step 3. Using the visual and physiological features extracted in Step 2 as input, a physiologically guided multimodal network is used for classification and prediction to obtain a pressure ulcer risk warning classification.

[0018] Furthermore, the data acquisition device is a handheld portable structure, including an imaging unit and a handheld clamp;

[0019] The imaging unit includes a large-aperture fixed-focus lens, a broadband white LED light source, a linear polarizer, an analyzer, and an industrial-grade color CMOS camera. The incident end of the large-aperture fixed-focus lens faces the pressure ulcer-prone area of ​​the subject, and the emitting end is coaxially connected to the photosensitive surface of the industrial-grade color CMOS camera via a mechanical interface. The white LED light source is a ring light source, surrounding the outer periphery of the large-aperture fixed-focus lens, and its emitting surface faces the pressure ulcer-prone area of ​​the subject. The linear polarizer is attached to or covers the outer side of the emitting surface of the broadband white LED light source (103). The analyzer is attached to or mounted on the outer side of the incident end of the large-aperture fixed-focus lens, and its polarization direction is 90° orthogonal to the polarization direction of the linear polarizer in the optical path. The industrial-grade color CMOS camera, in conjunction with the large-aperture fixed-focus lens, is configured to disable automatic gain control and lossy compression, and to acquire lossless video data at a sampling rate of 79fps.

[0020] The top of the handheld clamp is fixed to the imaging unit, and the bottom is the gripping part, forming a handheld portable structure.

[0021] Furthermore, the large-aperture fixed-focus lens is a fixed-focus lens with a 6mm focal length and an F1.4 aperture; the broadband white LED light source is a DC-driven 8W LED light source.

[0022] Furthermore, the handheld clamp is configured such that when handheld operation is performed via the grip at its bottom end, the distance between the incident end of the large aperture fixed-focus lens and the pressure ulcer-prone area of ​​the subject is stably maintained at 15-20 cm, and the field of view of the large aperture fixed-focus lens completely covers the pressure ulcer-prone area of ​​the subject and the adjacent normal skin area used to provide a dynamic control benchmark.

[0023] Furthermore, in step b1, the Shi-Tomasi corner detection algorithm is used to extract the corner feature points of the video frame, and the KLT optical flow algorithm is used to track the corner feature points of the frame frame by frame. The motion displacement vector between adjacent frames is estimated, and the boundary position between the region of interest of the lesion and the region of interest of the surrounding normal skin is dynamically updated according to the motion displacement vector, so as to correct the positional offset caused by the shaking of the human body plane in real time.

[0024] Furthermore, in step b2, the process of obtaining the original rPPG-BVP signal includes:

[0025] Obtain the pixel mean values ​​of the red, green, and blue channels, calculate their power spectral density, and extract the channel-specific signal-to-noise ratio (SNR). k :

[0026] ;

[0027] in, Physiological frequency band For noise frequency band, For integration variables, It is a very small constant used to prevent the denominator from being zero;

[0028] Adaptive channel weights are calculated based on specific signal-to-noise ratio. :

[0029] ;

[0030] Substituting the adaptive channel weights mentioned above into the orthogonal projection formula, the weight contribution of color channels that are contaminated by noise or have weak physiological signals is dynamically reduced to suppress environmental interference and generate the original projection signal:

[0031] ;

[0032] ;

[0033] in, This represents the first original projection signal generated after weighted orthogonal projection. This represents the second original projection signal generated after weighted orthogonal projection; t represents time. , , These represent the adaptive channel weights of the three color channels, red (R), green (G), and blue (B), respectively, derived from the calculation. , , These represent the normalized alternating current (AC) signals for the red, green, and blue color channels, respectively.

[0034] based on and The standard deviation is used to determine the weights, and the two are weighted and fused to generate the original rPPG-BVP signal.

[0035] Furthermore, in step b2, filtering is performed, and then physiological features are extracted from the filtered rPPG-BVP signal. The process includes:

[0036] The original rPPG-BVP signal is subjected to a first bandpass filter with a passband frequency range of 0.6Hz to 4.0Hz to obtain the first filtered signal;

[0037] A second bandpass filter with a passband frequency range of 0.5Hz to 10.0Hz is performed on the original rPPG-BVP to obtain the second filtered signal;

[0038] The relative perfusion rate (RPR) is calculated based on the first filtered signal.

[0039] Calculate the Hjorth complexity ratio (Ratio_Comp) based on the second filtered signal.

[0040] Physiological characteristics were obtained by combining the relative perfusion rate and the Hjorth complexity ratio. :

[0041] .

[0042] in, This represents a higher-dimensional physiological semantic feature vector generated after mapping by the physiological hemodynamic encoder; , , These represent the first, second, and third fully connected network blocks cascaded in the physiological hemodynamic encoder, respectively.

[0043] Furthermore, in step 3, the process of using the visual and physiological features extracted in step 2 as inputs and performing classification prediction using a physiologically guided multimodal network includes:

[0044] Step 31. Process the input physiological features using a hierarchical embedding network to generate physiological semantic features; the hierarchical embedding network consists of sequentially connected linear transforms, batch normalization, and ReLU activation functions;

[0045] Step 32. Input both visual features and physiological semantic features into the gating unit; the gating unit performs channel-level reweighting on the visual features based on the physiological semantic features and outputs enhanced features;

[0046] Step 33. Establish a visual auxiliary classification head, a physiological auxiliary classification head, and a main classification head; input visual features into the visual auxiliary classification head to obtain the first output; input physiological semantic features into the physiological classifier to obtain the second output; the physiological auxiliary classification head takes physiological semantic features as input, and the main classification head takes enhancement features as input. The enhancement features are input into the main classification head to obtain the third output.

[0047] Step 34. The first output, the second output, and the third output are weighted and fused according to a predetermined weight to obtain the final classification result; when the final classification result exceeds a preset threshold, the smart terminal issues an early warning.

[0048] Furthermore, the gating unit includes a stitching layer, an SE (Squeeze-and-Excitation) attention module, and a weighting layer; the stitching layer is used to stitch visual features and physiological semantic features along the channel dimension to obtain stitched features; the SE attention module is used to calculate channel-level gating weights based on the stitched features; the weighting layer is used to multiply the gating weights with the visual features channel by channel to obtain enhanced feature output.

[0049] In real-world home or clinical care settings, we often face suboptimal imaging conditions (i.e., environmental degradation) such as insufficient lighting, motion blur caused by the patient's involuntary movement, or sensor hardware noise. When these adverse factors, such as Gaussian noise, cause severe damage, distortion, or even complete unreliability to the visual texture features of the lesion surface, the results can be significant.

[0050] Based on this, the early warning method for pressure injuries provided by this invention employs a specific data acquisition device, including a large-aperture fixed-focus lens, a broadband white LED light source, a linear polarizer, an analyzer, and an industrial-grade color CMOS camera. The components are designed with a unique connection method specific to this application (i.e., the linear polarizer at the front end of the linear source and the analyzer at the front end of the lens are orthogonally configured at 90 degrees). This orthogonal polarization optical gating mechanism physically blocks surface specular reflections that retain the incident polarization state (such as highlight artifacts generated by the stratum corneum and wound exudate), allowing only diffusely reflected photons carrying deep microcirculation hemodynamic information to penetrate and enter the camera after multiple scattering and depolarization of photons deep into subcutaneous tissue. This significantly improves the accuracy of the original image for extracting weak physiological signals at the source of data acquisition. Subsequently, during the physiological feature extraction process, physiological feature extraction is completed by calculating the relative perfusion rate and signal complexity, greatly enhancing the model's anti-interference ability and robustness in extremely noisy environments and effectively avoiding the diagnostic failure problem that easily occurs in traditional pure visual models. Simultaneously, it achieves dynamic and smooth transfer of decision weights to the subcutaneous hemodynamic branch, which is unaffected by external light and surface dirt. Finally, in the prediction and classification stage, artifacts are further removed through gating units, and visual and physiological classifiers are combined as auxiliary classifiers to participate in the main classifier's decision-making. This overcomes the limitations of superficial morphological features and outputs extremely accurate pathological staging results for pressure injuries. In particular, it can accurately distinguish between highly deceptive visual morphology-related reactive hyperemia (RH), stage I pressure ulcers (S1), and occult deep tissue injuries (DTPI) without surface rupture.

[0051] Compared with existing technologies, the method of the present invention provides medical staff with a valuable early intervention window and visualized clinical decision support before irreversible tissue ulceration occurs in the lesion.

[0052] Furthermore, the method of the present invention can also summarize the final pathological state classification results, local blood flow abnormality indicators and other data, and intuitively generate corresponding disease risk scores and graded early warning prompts on intelligent terminal systems (such as nursing staff's mobile devices or monitoring screens). Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A schematic diagram of the data acquisition device structure provided in the embodiment;

[0055] Figure 2 The overall processing flow framework of the early warning method provided in the embodiments Figure 1 Its focus is on showcasing optical flow tracing, weighted POS signal extraction, and visual preprocessing.

[0056] Figure 3 A schematic flowchart of the frequency domain signal-to-noise ratio weighted hemodynamic signal extraction (Weighted POS) algorithm provided for the embodiment;

[0057] Figure 4 Provides a warning method processing flow for the proposed embodiments. Figure 2 The focus is on demonstrating the principle of the dual-stream multimodal architecture based on the Physiological Guidance Network (PGNet);

[0058] Figure 5 A schematic diagram illustrating the internal structure and data processing flow of the gated unit (PVGU) provided for this embodiment;

[0059] Figure reference numerals: 100 is a data acquisition device, 101 is an industrial-grade color CMOS camera, 102 is a handheld fixture, 103 is a broadband white LED light source, 104 is a linear polarizer, 105 is an analyzer, and 106 is a large-aperture fixed-focus lens. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0062] This embodiment provides an early warning method for pressure injury based on multimodal hemodynamic sensing and analysis, specifically designed to differentiate morphologically similar reactive hyperemia (RH), stage 1 pressure ulcers (S1), and deep tissue injury (DTPI).

[0063] like Figure 2 and Figure 4 As shown, the early warning method for pressure injury includes the following steps:

[0064] Step 1. Video stream acquisition: Use a data acquisition device to acquire the raw video stream of the subject's pressure ulcer-prone areas.

[0065] The data acquisition device in this embodiment is a high-fidelity cross-polarization data acquisition device, specifically designed to overcome the technical challenge of sparse capillaries and severe attenuation of subcutaneous hemodynamic signals in areas prone to pressure ulcers. The data acquisition device is as follows... Figure 1 As shown, it includes an imaging unit and a handheld gripper. The specific structure and connection relationship of each component are as follows:

[0066] The imaging unit includes a large-aperture fixed-focus lens, a broadband white LED light source, a linear polarizer, an analyzer, and an industrial-grade color CMOS camera; wherein:

[0067] The incident end of the large-aperture fixed-focus lens faces the pressure ulcer-prone area of ​​the subject, while the exit end is coaxially connected to the photosensitive surface of the industrial-grade color CMOS camera via a mechanical interface. A ring-shaped white LED light source surrounds the outer periphery of the large-aperture fixed-focus lens, with its emitting surface facing the pressure ulcer-prone area of ​​the subject. A linear polarizer is attached to or covers the outer side of the emitting surface of the broadband white LED light source. An analyzer is attached to or mounted on the outer side of the incident end of the large-aperture fixed-focus lens, its polarization direction being 90° orthogonal to the polarization direction of the linear polarizer in the optical path. The industrial-grade color CMOS camera, in conjunction with the large-aperture fixed-focus lens, is configured to disable automatic gain control and lossy compression, and performs lossless video data acquisition at a sampling rate of 79fps.

[0068] In this embodiment, an industrial-grade color CMOS camera, coupled with a large-aperture fixed-focus lens with a focal length of 6mm and an aperture of F1.4, is configured to disable automatic gain control and lossy compression, and to perform lossless video data acquisition at a sampling rate of 79fps. While achieving high-frequency oversampling of the blood flow volume pulse wave (BVP) signal, the acquisition device is ensured to retain the minute morphological features of the systolic and diastolic blood flow peaks even when faced with high-frequency noise interference, thereby preserving the fine texture details of the lesion area.

[0069] In real clinical settings, the exudate and thick stratum corneum on the surface of pressure ulcer tissue produce severe specular artifacts. To suppress this interference physically, the broadband white LED light source is a DC-driven 8W LED light source. By placing a linear polarizer on the light-emitting surface of the LED light source and installing an analyzer at the incident end of a large-aperture fixed-focus lens, this optical gating design ensures that polarized light illuminating the skin surface retains its initial polarization state during specular reflection, thus being completely blocked by the orthogonally configured analyzer. Meanwhile, polarized light transmitted into the dermis and subcutaneous tissue undergoes depolarization after multiple scatterings. These diffusely reflected photons carrying deep microcirculatory hemodynamic information can then successfully penetrate the analyzer and be captured by an industrial-grade color CMOS camera. This mechanism achieves physical decoupling of surface specular artifacts from deep target physiological signals at the source stage of hardware acquisition.

[0070] To improve portability, the top of the handheld clamp is fixed to the imaging unit, while the bottom serves as the grip, forming a one-handed portable structure. During implementation, when operating the device handheld through the grip at the bottom, the distance between the incident end of the large-aperture fixed-focus lens and the pressure ulcer-prone area of ​​the subject is consistently maintained at 15-20 cm. The field of view of the large-aperture fixed-focus lens completely covers the pressure ulcer-prone area and its surrounding normal skin tissue, providing a solid hardware foundation for subsequent high-precision feature tracking and algorithm analysis. It should be noted that the surrounding area refers to the normal skin area adjacent to the pressure ulcer-prone area, used to provide a dynamic control baseline.

[0071] Step 2. Dual-branch feature extraction: The dual-branch approach consists of a visual feature branch and a physiological feature branch. The visual feature branch extracts visual features, while the physiological feature branch extracts physiological features. The feature extraction process for each branch is as follows:

[0072] a. Visual Branch: Taking image data from the original video stream as input, after extracting volumetric features using a semantic encoder, global reddening is applied to the feature map of the last convolution to generate visual features. Using the RGB image data of the original video stream as input, a truncated ResNet-18 backbone network is used to extract spatial visual representations. The feature map output from the last convolutional stage of the ResNet-18 backbone network is then obtained and globally pooled to generate visual features. ;

[0073] ;

[0074] Here, GAP stands for Global Average Pooling, which is used to compress spatial dimensions; This represents the high-level feature map output by the last convolutional stage of the ResNet-18 backbone network; This indicates that the generated visual feature vector v_vis belongs to a 512-dimensional real vector space.

[0075] b. The physiological branch extracts physiological features through the following sub-steps:

[0076] b1. Preprocessing the video stream: Based on an adaptive threshold, the region of interest (ROI) of the lesion and the surrounding normal skin ROI are delineated in the initial frame of the video. Motion estimation is performed on subsequent frames of the video. Based on the estimated motion information, a frame-by-frame affine transformation is performed on the boundaries of the delineated ROI and the surrounding normal skin ROI to dynamically update the spatial positions of the image data of the ROI and the surrounding normal skin ROI in each frame of the video. In this embodiment, the Shi-Tomasi corner detection algorithm is used to extract the corner feature points of the video frames, and the KLT optical flow algorithm is combined to track the frame corner feature points frame by frame. The motion displacement vector between adjacent frames is estimated, and the boundary position of the ROI and the surrounding normal skin ROI is dynamically updated according to the motion displacement vector to correct the positional offset caused by the shaking of the human body plane in real time.

[0077] b2. Based on step b1, rPPG-BVP (blood flow volume pulse wave) signals are extracted from the image data of the dynamically updated region of interest of the lesion and the region of interest of the surrounding normal skin in step 2, respectively. The original rPPG-BVP signals are obtained and filtered. Then, physiological features are extracted from the filtered rPPG-BVP signals. The physiological characteristics mentioned are multi-dimensional physiological characteristics, including time-domain characteristics, frequency-domain characteristics, and nonlinear dynamic characteristics.

[0078] Because pressure ulcer sites are characterized by low capillary density and weak signal strength, it is necessary to specifically extract rPPG-BVP signals and their characteristics. Unlike the standard POS algorithm, which assumes equivalent information across channels, this embodiment introduces a frequency domain signal-to-noise ratio weighted strategy (Weighted POS), quantifying the quality of the normalized AC signal through power spectral density. For example... Figure 3 As shown, the details are as follows:

[0079] Obtain the pixel mean values ​​of the red, green, and blue channels, and calculate their power spectral density. And extract channel-specific signal-to-noise ratio :

[0080] ;

[0081] in, Physiological frequency band For noise frequency band, For integration variables, It is a very small constant used to prevent the denominator from being zero;

[0082] Adaptive channel weights are calculated based on specific signal-to-noise ratio. :

[0083] ;

[0084] Substituting the adaptive channel weights mentioned above into the orthogonal projection formula, the weight contribution of color channels that are contaminated by noise or have weak physiological signals is dynamically reduced to suppress environmental interference and generate the original projection signal:

[0085] ;

[0086] ;

[0087] in, This represents the first original projection signal generated after weighted orthogonal projection. This represents the second original projection signal generated after weighted orthogonal projection; t represents time. , , These represent the adaptive channel weights of the three color channels, red (R), green (G), and blue (B), respectively, derived from the calculation. , , These represent the normalized alternating current (AC) signals for the red, green, and blue color channels, respectively.

[0088] based on and The standard deviation is used to determine the weights, and the two are weighted and fused to generate the original rPPG-BVP signal.

[0089] The original rPPG-BVP signal is subjected to a first passband filter to obtain a first filtered signal; the frequency range of the first passband filter is 0.6-4.0 Hz, which is used to estimate the relative pulse rate (RPR).

[0090] A second bandpass filter is applied to the original rPPG-BVP to obtain a second filtered signal. The frequency range of the second bandpass filter is 0.5Hz to 10.0Hz, used to preserve high-frequency harmonic characteristics.

[0091] Subsequently, time-domain, frequency-domain, and nonlinear dynamic features are extracted through feature engineering, and the ratio of lesion to normal area is calculated. In this embodiment, the Hjorth complexity ratio and relative perfusion rate (RPR), which have the highest discriminative power, are selected as the physiological prior vectors input to the network. Specifically:

[0092] The relative perfusion rate (RPR) is calculated based on the first filtered signal.

[0093] Calculate the Hjorth complexity ratio (Ratio_Comp) based on the second filtered signal.

[0094] The physiological feature vector is obtained by combining the relative perfusion rate and the Hjorth complexity ratio. :

[0095] .

[0096] in, This represents a higher-dimensional physiological semantic feature vector generated after mapping by the physiological hemodynamic encoder; , , These represent the first, second, and third fully connected network blocks cascaded in the physiological hemodynamic encoder, respectively.

[0097] Step 3. Using the visual and physiological features extracted in Step 2 as input, a physiologically guided multimodal network, PGNet, is used for classification and prediction to obtain a pressure ulcer risk warning classification. This step includes the following sub-steps:

[0098] Step 31. Use a hierarchical embedding network to process the input physiological features. Processing is performed to generate physiological semantic features. The hierarchical embedded network consists of sequentially connected linear transformations, batch normalization, and ReLU activation functions.

[0099] Step 32. Combine visual features and physiological semantic features. The input is fed into the gating unit; the gating unit then selects the input based on physiological semantic features. The visual features are reweighted at the channel level to output enhanced features. The gating unit in this embodiment includes a stitching layer, a Squeeze-and-Excitation (SE) attention module, and a weighting layer. Wherein:

[0100] The stitching layer is used to combine visual features with physiological and semantic features. By splicing along the channel dimension, the splicing feature is obtained.

[0101] The SE attention module is used to calculate channel-level gating weights based on splicing features. In this embodiment, the gating unit introduces a squeeze-and-excitation attention module to avoid the problem of high-dimensional visual noise suppressing weak physiological signals caused by direct splicing. The implementation process of the SE attention module is expressed by the following expression:

[0102] ;

[0103] in, This represents the Sigmoid activation function, used to normalize the output value to the interval [0, 1]. This represents the weight matrix of the second fully connected layer, used to restore the feature dimension to the original concatenation dimension; Represents the ReLU nonlinear activation function; `GAP()` represents the weight matrix of the first fully connected layer, used for channel-level compression and dimensionality reduction; `U` represents the cross-modal joint feature descriptor, derived from the visual feature vector `v`. vis and physiological semantic features phy It is composed of multiple layers. The weighted layer is used to multiply the gate weights with the visual features channel by channel to obtain the enhanced feature output; its expression is:

[0104] ;

[0105] in, ○ indicates the enhanced fusion feature output after recalibration; ○ indicates channel-by-channel multiplication operation.

[0106] Physiological priors are used to guide the entire network to focus on visual channels associated with microcirculatory dysfunction. This allows hemodynamic prior information to functionally highlight the corresponding visual channels and dynamically filter out surface background artifacts unrelated to tissue perfusion.

[0107] In this way, physiological priors are used to guide the entire network to focus on visual channels related to microcirculatory dysfunction. This allows hemodynamic prior information to functionally highlight the corresponding visual channels and dynamically filter out surface background artifacts unrelated to tissue perfusion.

[0108] Step 33. Model Training Phase: To prevent the visual branch from dominating in multimodal joint training and causing underfitting of the physiological branch, this embodiment employs a deep supervision mechanism. That is, based on the traditional single-modal model, visual and physiological auxiliary classifiers are added; in other words, this embodiment establishes a visual auxiliary classifier head, a physiological auxiliary classifier head, and a main classifier head. A joint objective loss function for the three classifiers is then constructed to train and optimize the model; the expression for the joint objective loss function is:

[0109] ;

[0110] in, This represents the classification loss between the predicted value output by the main classifier head and the true label. This represents the auxiliary classification loss between the predicted value output by the visual auxiliary classification head and the true label. This represents the auxiliary classification loss between the predicted value output by the physiological auxiliary classification head and the true label. and These represent the loss weights corresponding to visual-assisted classification loss and physiological-assisted classification loss, respectively, which are used to adjust the gradient flow direction during training to ensure that each modality branch captures independent pathological semantic features.

[0111] Step 34. Model Inference Stage: To effectively avoid the risk of failure in single-modal branches, especially the visual branch, this embodiment adopts a weighted integration inference mechanism based on multi-head logic values. Specifically:

[0112] Visual features are input into a visual auxiliary classification head to obtain the first output; physiological semantic features are then input into the classification head. The input is fed into the physiological classifier to obtain the second output; the physiological auxiliary classification head uses physiological semantic features. The main classifier takes the augmented features as input and outputs them to obtain the third output.

[0113] The first, second, and third outputs are weighted and fused according to predetermined weights to obtain the final classification result; when the final classification result exceeds a preset threshold, the smart terminal issues an early warning.

[0114] The process of obtaining the final classification result through weighted fusion in this embodiment can be represented as follows:

[0115] ;

[0116] in, This represents the decision operator that takes the index corresponding to the maximum value. Represents the preset integration weights corresponding to each branch; , and These represent the original logical values ​​output by the main classifier, the visual auxiliary classifier, and the physiological auxiliary classifier, respectively. , and These represent the preset ensemble weights for the main classification branch, the visual auxiliary branch, and the physiological auxiliary branch in the ensemble decision-making process.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An early warning method for pressure injury based on multimodal hemodynamic sensing and analysis, comprising the following steps: Step 1. Video stream acquisition: Use a data acquisition device to acquire the raw video stream of the subject's pressure ulcer-prone areas; Step 2. Dual-branch feature extraction: The dual-branch approach consists of a visual feature branch and a physiological feature branch. The visual feature branch extracts visual features, while the physiological feature branch extracts physiological features. The feature extraction process for each branch is as follows: a. Visual Branch: The RGB image data of the original video stream is used as input. Spatial visual representations are extracted by truncating the ResNet-18 backbone network. The feature map output by the last convolutional stage of the ResNet-18 backbone network is then subjected to global pooling to generate visual features. b. The physiological branch extracts physiological features through the following sub-steps: b1. Preprocessing the video stream: Based on an adaptive threshold, the region of interest (ROI) of the lesion and the surrounding normal skin ROI are delineated in the initial frame of the video; motion estimation is performed on subsequent frames of the video; based on the estimated motion information, frame-by-frame affine transformation is performed on the boundary between the delineated ROI and the surrounding normal skin ROI to realize the dynamic update of the spatial position of the image data of the ROI and the surrounding normal skin ROI in each frame of the video. b2. Based on step b1, rPPG-BVP signals are extracted from the image data of the lesion region of interest and the surrounding normal skin region of interest after the dynamic update in step 2, respectively, to obtain the original rPPG-BVP signals and perform filtering processing. Then, physiological features are extracted from the filtered rPPG-BVP signals. The physiological features are multi-dimensional physiological features including time-domain features, frequency-domain features, and nonlinear dynamic features. Step 3. Using the visual and physiological features extracted in Step 2 as input, a physiologically guided multimodal network is used for classification and prediction to obtain a pressure ulcer risk warning classification.

2. The early-stage pressure injury warning method based on multi-modal hemodynamic sensing and analysis of claim 1, wherein, The data acquisition device is a handheld portable structure, including an imaging unit and a handheld clamp; The imaging unit includes a large-aperture fixed-focus lens, a broadband white LED light source, a linear polarizer, an analyzer, and an industrial-grade color CMOS camera; wherein: The incident end of the large aperture fixed-focus lens is directed towards the area of ​​the subject prone to pressure sores, and the exit end is coaxially connected to the photosensitive surface of an industrial-grade color CMOS camera via a mechanical interface. The white LED light source is a ring light source, which is placed around the outer periphery of the large aperture fixed-focus lens, and its light-emitting surface faces the area of ​​the subject prone to pressure sores. A linear polarizer is attached to or covers the outer side of the light-emitting surface of a broadband white LED light source; an analyzer is attached to or mounted on the outer side of the incident end of a large aperture fixed-focus lens, and its polarization direction is 90° orthogonal to the polarization direction of the linear polarizer in the optical path. An industrial-grade color CMOS camera, paired with a large-aperture fixed-focus lens, is configured to disable automatic gain control and lossy compression, and to acquire lossless video data at a sampling rate of 79fps. The top of the handheld clamp is fixed to the imaging unit, and the bottom is the gripping part, forming a handheld portable structure.

3. The early-stage pressure injury warning method based on multi-modal hemodynamic sensing and analysis of claim 1, wherein, The large-aperture fixed-focus lens is a 6mm focal length, F1.4 aperture fixed-focus lens; the broadband white LED light source is a DC-driven 8W LED light source.

4. The early pressure injury warning method based on multimodal hemodynamic sensing and analysis according to claim 3, characterized in that, The handheld gripper is configured as follows: When operated by hand using the grip at the bottom, the distance between the incident end of the large aperture fixed-focus lens and the pressure ulcer-prone area of ​​the subject is kept stable at 15-20 cm, and the field of view of the large aperture fixed-focus lens completely covers the pressure ulcer-prone area of ​​the subject and the adjacent normal skin area used to provide a dynamic control baseline.

5. The early warning method for pressure injury based on multimodal hemodynamic sensing and analysis according to claim 1, characterized in that, In step b1, the Shi-Tomasi corner detection algorithm is used to extract the corner feature points of the video frame, and the KLT optical flow algorithm is used to track the corner feature points of the frame frame by frame. The motion displacement vector between adjacent frames is estimated, and the boundary position between the region of interest of the lesion and the region of interest of the surrounding normal skin is dynamically updated according to the motion displacement vector, so as to correct the positional offset caused by the shaking of the human body plane in real time.

6. The early warning method for pressure injury based on multimodal hemodynamic sensing and analysis according to claim 1, characterized in that, In step b2, the process of obtaining the original rPPG-BVP signal includes: Obtain the mean value of the pixels of the red, green and blue channels, calculate the power spectral density thereof, and extract channel-specific signal-to-noise ratio SNR k : Adaptive channel weights are calculated based on specific signal-to-noise ratio. : By substituting the aforementioned adaptive channel weights into the orthogonal projection formula, the weight contribution of color channels affected by noise or weak physiological signals is dynamically reduced to suppress environmental interference and generate the original projection signal. and : based on and The standard deviation is used to determine the weights, and the two are weighted and fused to generate the original rPPG-BVP signal.

7. The early warning method for pressure injury based on multimodal hemodynamic sensing and analysis according to claim 1, characterized in that, In step b2, filtering is performed, and then physiological features are extracted from the filtered rPPG-BVP signal. The process includes: The original rPPG-BVP signal is subjected to a first bandpass filter with a passband frequency range of 0.6Hz to 4.0Hz to obtain the first filtered signal; A second bandpass filter with a passband frequency range of 0.5Hz to 10.0Hz is performed on the original rPPG-BVP to obtain the second filtered signal; The relative perfusion rate is calculated based on the first filtered signal; Calculate the Hjorth complexity ratio based on the second filtered signal; Physiological characteristics were obtained by combining the relative perfusion rate and the Hjorth complexity ratio.

8. The early warning method for pressure injury based on multimodal hemodynamic sensing and analysis according to claim 1, characterized in that, In step 3, the process of using the visual and physiological features extracted in step 2 as inputs and performing classification prediction using a physiologically guided multimodal network includes: Step 31. Process the input physiological features using a hierarchical embedding network to generate physiological semantic features; the hierarchical embedding network consists of sequentially connected linear transforms, batch normalization, and ReLU activation functions; Step 32. Input both visual features and physiological semantic features into the gating unit; the gating unit performs channel-level reweighting on the visual features based on the physiological semantic features and outputs enhanced features; Step 33. Establish a visual auxiliary classification head, a physiological auxiliary classification head, and a main classification head; input visual features into the visual auxiliary classification head to obtain the first output; input physiological semantic features into the physiological classifier to obtain the second output; the physiological auxiliary classification head takes physiological semantic features as input, and the main classification head takes enhancement features as input. The enhancement features are input into the main classification head to obtain the third output. Step 34. The first output, the second output, and the third output are weighted and fused according to a predetermined weight to obtain the final classification result; when the final classification result exceeds a preset threshold, the smart terminal issues an early warning.

9. The early warning method for pressure injury based on multimodal hemodynamic sensing and analysis according to claim 8, characterized in that, The gating unit includes a stitching layer, an SE attention module, and a weighting layer. The stitching layer is used to stitch visual features and physiological semantic features along the channel dimension to obtain stitched features. The SE attention module is used to calculate channel-level gating weights based on the stitched features. The weighting layer is used to multiply the gating weights with the visual features channel by channel to obtain enhanced feature outputs.

Images