Real-time monitoring and early warning system for circulatory state image of cardiovascular surgery patient

Abstract

The present application relates to the technical field of medical image processing, in particular to a real-time monitoring and early warning system for the circulation state image of a cardiovascular surgery patient, comprising a video acquisition module, an image preprocessing module, a microcirculation evaluation module, a multi-site perfusion fusion module based on differential manifold, a trend prediction and early warning module, and a clinical interaction and data integration module; the video of the patient's fingernail bed and lips is simultaneously acquired by a high-frame-rate camera, and after preprocessing, the local perfusion index and pulsatility stability index are extracted; innovatively, the multi-site microcirculation signals are mapped onto the differential manifold for representation; through the time-frequency dual-channel fusion driven by Riemann geometry and the attention weighting mechanism, the whole-body perfusion comprehensive index is integrated; and based on the dynamic system modeling on the manifold, early warning can be provided before the conventional blood pressure significantly decreases, valuable time is won for clinical intervention, the non-invasive monitoring method avoids invasive complications, and whole-process continuous monitoring can be realized.

Description

Technical Field

[0001] This invention relates to the field of medical monitoring technology, specifically to a real-time monitoring and early warning system for the circulatory status of cardiovascular surgical patients, and particularly to a multi-site microcirculation image fusion and prediction early warning system based on the theory of differential manifolds. Background Technology

[0002] During cardiovascular surgery, continuous monitoring of the patient's circulatory status is crucial for ensuring surgical safety. Traditional monitoring methods mainly rely on macroscopic vital signs such as invasive blood pressure, heart rate, and blood oxygenation. These indicators often lag behind actual changes in circulatory status and are difficult to provide early warnings. With the development of medical technology, microcirculation monitoring has been gradually applied in clinical practice, which can reflect changes in tissue perfusion at an earlier stage.

[0003] Currently, microcirculation monitoring mainly employs technologies such as laser Doppler and near-infrared spectroscopy. However, these methods suffer from problems such as complex equipment, high costs, and cumbersome operation. Remote photoplethysmography (rPPG) based on visible light video offers a new approach for non-invasive microcirculation monitoring; however, existing rPPG technologies primarily target single-site monitoring, making them susceptible to local factors and resulting in unstable monitoring results. Furthermore, existing early warning systems often rely on simple threshold judgments, lacking the ability to predict circulatory trends and thus failing to provide true early warning.

[0004] Therefore, there is an urgent need for a system that can integrate microcirculation information from multiple sites, predict changes in circulatory status, and provide early warnings to improve perioperative safety for cardiovascular surgery patients. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time monitoring and early warning system for the circulatory status of cardiovascular surgery patients. This system can simultaneously monitor the microcirculation of the fingernail bed and lips of patients, integrate information from multiple sites into a comprehensive perfusion index through the theory of differential manifolds, and predict the trend of changes in circulatory status. It provides early warning before significant changes in traditional vital signs, thus gaining valuable intervention time for the medical team.

[0006] This invention proposes a real-time monitoring and early warning system for the circulatory status of cardiovascular surgical patients, comprising:

[0007] The video acquisition module is used to simultaneously or alternately acquire high frame rate visible light video of the patient's fingernail bed and lips;

[0008] The image preprocessing module is communicatively connected to the video acquisition module and is used to perform region segmentation, motion compensation and illumination normalization on the visible light video to obtain a stable tissue reflectance intensity sequence.

[0009] The microcirculation assessment module is communicatively connected to the image preprocessing module and is used to calculate the local perfusion index and pulsation stability index of various parts of the fingernail bed and lips based on the stable tissue reflection intensity sequence.

[0010] The multi-site perfusion fusion module based on differential manifolds is communicatively connected to the microcirculation evaluation module and is used for:

[0011] The local perfusion index and pulsatility index are mapped onto a differential manifold for characterization.

[0012] Perform time-frequency dual-channel fusion based on Riemannian geometry on the differential manifold;

[0013] By integrating information from various parts through an attention-weighted mechanism, a comprehensive perfusion index for the whole body is generated;

[0014] The trend prediction and early warning module is communicatively connected to the multi-site perfusion fusion module based on the differential manifold. It is used to construct a dynamic system model on the manifold based on the whole-body perfusion comprehensive index, predict the trend of circulatory state changes within the future time window, and generate early warning information when the risk exceeds the threshold.

[0015] The clinical interaction and data integration module is communicatively connected to the trend prediction and early warning module, and is used to display the change curve of the systemic perfusion comprehensive index, early warning information, and exchange data with the hospital monitoring system.

[0016] Preferably, the video acquisition module has a frame rate of not less than 120fps and has adaptive exposure and polarization anti-reflection functions to reduce saturation artifacts caused by highlights on the wet surfaces of the nail bed and lips.

[0017] Preferably, the image preprocessing module includes:

[0018] The region segmentation unit is used to perform fine segmentation of the nail bed and lip regions based on YCrCb and HSV dual-color gamut skin color screening technology combined with morphological and texture feature priors.

[0019] The motion compensation unit is used to perform non-rigid motion compensation based on optical flow and feature point tracking technology, and employs sub-pixel level registration to constrain vascular stripe stabilization.

[0020] The illumination normalization unit is used to stabilize video signals under varying illumination conditions by employing Retinex logarithmic domain correction and inter-frame luminance drift linear regression elimination techniques.

[0021] Preferably, the microcirculation assessment module is used for:

[0022] The green channel-dominated remote photovolume plot base signal and the RGB-PCA fused signal are calculated within the segmentation mask;

[0023] Perform bandpass filtering and phase alignment with the cardiac cycle to estimate the beat amplitude and the coefficient of variation of the cycle;

[0024] A local perfusion index was constructed by comparing pulsation amplitude with tissue color reflectance, and corrections were made for micro-amplitude arrival time and hue swing amplitude.

[0025] A pulsatility stability index was constructed using the period variation coefficient, cycle phase jitter, and spectral peak narrowness.

[0026] Preferably, the multi-site perfusion fusion module based on differential manifolds includes:

[0027] Manifold characterization unit is used to map the local perfusion index and pulsatility stability index of each part onto the differential manifold, construct the local geometric structure, and design part-specific manifold geometric models;

[0028] The time-frequency fusion unit is used to perform geodesic state estimation and frequency domain coherence alignment on the differential manifold to achieve collaborative fusion of time-domain and frequency-domain information;

[0029] The attention-weighted unit is used to calculate dynamic attention weights based on signal quality scores of each site, clinical context information, and historical predicted performance, and to integrate information from multiple sites to generate a comprehensive whole-body perfusion index.

[0030] Preferably, the manifold characterization unit is used for:

[0031] Extract an 8-dimensional feature vector that includes amplitude features, time-domain features, and frequency-domain features;

[0032] An initial manifold structure was constructed and individualized based on patient baseline data;

[0033] Construct a local coordinate system on the manifold, where the first principal direction is aligned with the axis of change in pulsation amplitude and the second principal direction is aligned with the axis of change in pulsation frequency;

[0034] Calculate local curvature, local volume, and local entropy to characterize the geometric properties of the current perfusion state.

[0035] Preferably, the time-frequency fusion unit is used for:

[0036] In the time domain path, a geodesic state estimation algorithm is executed, including state prediction and updating along the manifold geodesic, and dynamic adjustment of gain based on signal quality score and prediction error;

[0037] In the frequency domain path, adaptive wavelet transform is performed on the signals of each part to extract the energy distribution spectrum and phase synchronization characteristics, and feature alignment and fusion are performed on the spectral manifold;

[0038] By integrating the fusion results of the time-domain path and the frequency-domain path, a comprehensive perfusion state characterization is generated.

[0039] Preferably, the trend prediction and early warning module is used for:

[0040] The temporal evolution of the total perfusion index is modeled as a nonlinear dynamic system on a manifold;

[0041] Learn vector fields on manifolds based on historical data to identify stable and unstable equilibrium points on manifolds;

[0042] Calculate early warning indicators such as geodesic deviation, autocorrelation function decay rate, and manifold entropy growth rate;

[0043] The design incorporates a three-tiered early warning triggering mechanism: short-term (5-15 minutes), medium-term (15-30 minutes), and long-term (30-60 minutes).

[0044] Generate visualized early warning results that include risk heatmaps, status trajectory diagrams, and contribution analysis diagrams.

[0045] As a preferred option, it also includes:

[0046] The environmental adaptation module is used to monitor changes in ambient light conditions and perform compensation adjustments to ensure stable operation of the system under fluctuating intraoperative lighting conditions;

[0047] The quality control engine is used to evaluate video quality and signal reliability in real time. When interference factors such as motion artifacts, dry lips, and nail bed coating are detected, it automatically adjusts the processing strategy and provides quality prompts.

[0048] Preferably, the clinical interaction and data integration module is used for:

[0049] Displays perfusion curves at the site and systemic levels, and multi-scale risk heat maps;

[0050] Provide early warning interpretation information, including an analysis of the contribution of each part to the early warning and a prediction of the risk development trend;

[0051] It enables bidirectional data exchange with anesthesia monitors, extracorporeal circulation systems, and electronic medical record systems;

[0052] Record medical interventions and their effects, and update individualized early warning thresholds based on intervention-response data.

[0053] The present invention has the following beneficial effects:

[0054] 1. It realizes the coordinated monitoring of microcirculation in multiple parts, overcomes the limitation of single-point monitoring being easily affected by local factors, and improves the stability and reliability of monitoring;

[0055] 2. The innovative introduction of differential manifold theory for multi-site information fusion accurately captures the nonlinear geometric characteristics of physiological signals, achieving optimal information integration;

[0056] 3. Based on dynamic system modeling on manifolds, the prediction of circulatory state change trends was realized, providing early warning 20-25 minutes before a significant drop in blood pressure, thus gaining valuable time for clinical intervention;

[0057] 4. The system adopts a non-invasive monitoring method, avoiding the risk of complications associated with invasive monitoring, and is suitable for continuous monitoring throughout the entire process;

[0058] 5. It provides individualized monitoring strategies that can adapt to patient baseline differences and surgical type characteristics, improving the accuracy of early warning. Attached Figure Description

[0059] Figure 1 This is a block diagram of the overall structure of the real-time monitoring and early warning system for the circulatory status of cardiovascular surgical patients according to the present invention;

[0060] Figure 2 This is a schematic diagram of the video acquisition module of the present invention;

[0061] Figure 3 This is a flowchart of the image preprocessing module of the present invention;

[0062] Figure 4 This is a flowchart of the microcirculation evaluation module of the present invention;

[0063] Figure 5 This is a structural block diagram of the multi-site perfusion fusion module based on differential manifolds of the present invention;

[0064] Figure 6 This is a schematic diagram of the manifold characterization process in this invention;

[0065] Figure 7 This is a flowchart of the time-frequency dual-channel fusion process in this invention;

[0066] Figure 8 This is a flowchart of the trend prediction and early warning module of the present invention. Detailed Implementation

[0067] Please refer to Figures 1-8 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0068] like Figure 1As shown, the real-time monitoring and early warning system for the circulatory status of cardiovascular surgical patients provided by this invention includes a video acquisition module 1, an image preprocessing module 2, a microcirculation assessment module 3, a multi-site perfusion fusion module based on differential manifolds 4, a trend prediction and early warning module 5, and a clinical interaction and data integration module 6. The system may also include an environmental adaptation module 7 and a quality control engine 8.

[0069] Reference Figure 2 The video acquisition module 1 is used to simultaneously or alternately acquire high frame rate visible light video of the patient's fingernail bed and lips. Preferably, the video acquisition module 1 includes at least two high frame rate industrial cameras, respectively mounted on adjustable brackets and aimed at the patient's fingernail bed and lip area.

[0070] In one embodiment of the present invention, the frame rate of the video acquisition module 1 is not less than 120fps, and is typically set in the range of 120-200fps to ensure that subtle changes in microcirculation pulsation can be captured. Too low a frame rate may lead to the loss of high-frequency pulsation information, affecting the accuracy of subsequent analysis. Furthermore, the module also has adaptive exposure and polarization anti-reflection functions to reduce saturation artifacts caused by highlights on the moist surfaces of the nail bed and lips.

[0071] Specifically, the adaptive exposure function dynamically adjusts exposure parameters by analyzing the brightness distribution of the image in real time, ensuring that the image is neither overexposed nor underexposed. The polarization anti-reflection function effectively suppresses specular reflection and enhances the visibility of changes in blood flow within tissues by installing a polarizing filter in front of the lens, which is set to cross the polarization direction of the light source.

[0072] Reference Figure 3 The image preprocessing module 2 is connected to the video acquisition module 1 and is used to perform region segmentation, motion compensation and illumination normalization on the acquired visible light video to obtain a stable tissue reflectance intensity sequence.

[0073] Preferably, the image preprocessing module 2 includes a region segmentation unit 21, a motion compensation unit 22, and an illumination normalization unit 23.

[0074] Region segmentation unit 21 is used to perform fine segmentation of the nail bed and lip regions based on YCrCb and HSV dual-color gamut skin color screening technology combined with morphological and texture feature priors. In this invention, region segmentation adopts a two-stage strategy: first, coarse segmentation is performed based on color features, and then fine segmentation is performed using morphological and texture features.

[0075] Specifically, the color feature segmentation process is as follows:

[0076] (1) Convert the RGB image to the YCrCb color space and the HSV color space;

[0077] (2) In the YCrCb space, the range of chromaticity components Cr and Cb is set according to the characteristics of human skin color. Usually, the range of Cr is [133, 173] and the range of Cb is [77, 127].

[0078] (3) In the HSV space, set the range of hue H, saturation S and lightness V. For Asian skin tones, H usually takes the range of [0,20] and [340,360], S takes the range of [0.2,0.6], and V takes the range of [0.3,1.0].

[0079] (4) Perform a logical AND operation on the segmentation results of the two color spaces to obtain a more accurate skin color region.

[0080] In the fine segmentation stage, different processing strategies are adopted for the special morphological features of the fingernail bed and the lips: for the fingernail bed, fine segmentation is performed by utilizing the linear features of the nail bed edge and the color contrast between the nail bed and the periungual skin;

[0081] First, edge detection is performed within the skin-colored region obtained from coarse segmentation. The Canny operator is used for edge detection. The process involves first smoothing the image with a Gaussian filter to reduce noise, then calculating the image gradient intensity and direction, and finally extracting edges using non-maximum suppression and double thresholding. The formula for calculating the edge gradient intensity is:

[0082] ,

[0083] in This represents the gradient intensity, expressed in grayscale values ​​per pixel. The gradient component is in the horizontal direction, expressed in grayscale values ​​per pixel. This represents the vertical gradient component, expressed in grayscale values ​​per pixel. This represents the square root operation.

[0084] Secondly, linear features of the nail bed edge are extracted. The proximal edge of the nail bed (the junction with the periungual skin) usually presents an approximately straight line shape. Hough transform is used to detect linear edges, and for each detected edge segment, its parameterized representation is calculated:

[0085] ,

[0086] in The perpendicular distance from the origin to the line is expressed in pixels. The angle between the perpendicular line and the horizontal axis, expressed in radians, has a range of [missing information]. , The x-coordinate of the edge point is in pixels. The ordinate of the edge point is in pixels. For angle cosine value, For angle The sine value. (Through the parameter space) Through cumulative voting, straight segments were detected, and those with a length greater than 50% of the width of the nail bed and a nearly horizontal direction were selected as candidates for the proximal edge of the nail bed.

[0087] Next, the color contrast between the nail bed region and the periungual skin is calculated. Color information is sampled on both sides of the candidate nail bed edge. The sampling area on the nail bed side is set to a range of 3 to 10 pixels below the edge line, and the sampling area on the periungual skin side is set to a range of 3 to 10 pixels above the edge line. For each sampling area, the average value of the RGB three channels is calculated to obtain the color vector. and ,in This represents the average color vector of the nail bed region. , , These represent the average values ​​of the red, green, and blue channels for the nail bed region, expressed in grayscale values, with a range of [missing value]. The average color vector of the skin region around the nail. These represent the average values ​​of the red, green, and blue channels for the skin area around the nail, expressed in grayscale values, ranging from [0, 255]. (Superscript) This represents the vector transpose. Color contrast is defined as:

[0088] ,

[0089] in For color contrast, dimensionless, range is The Euclidean distance between two color vectors represents the degree of color difference. Let be the Euclidean norm of the nail bed color vector. Let be the Euclidean norm of the skin color vector around the nail. Represents the calculation of the Euclidean norm of a vector.

[0090] Finally, the optimal nail bed boundary is determined by combining linear features and color contrast. For each candidate edge segment, its comprehensive score is calculated:

[0091] ,

[0092] in The overall score for the edge line segment is dimensionless and ranges from [0,1]. The normalized line segment length is calculated by dividing the line segment length by the image width. It is dimensionless and ranges from [0,1]. The normalized Hough cumulative vote count is calculated by dividing the total number of votes by the maximum number of votes. It is dimensionless and ranges from [0,1]. For color contrast, , , For the weighting coefficients, take respectively , , ,satisfy The dimensionless line segment is selected as the boundary of the nail bed. Combined with the image edges and the boundaries on both sides of the nail bed, a complete nail bed segmentation mask is constructed. Morphological closing operations are performed on the segmentation results to fill internal holes, ensuring the continuity and integrity of the segmented region.

[0093] For the lips, fine segmentation is performed using the arched edge features of the lips and the textural differences between the lips and facial skin tone. The specific segmentation process includes the following steps:

[0094] First, the arched edge features of the lips are detected. The upper and lower edges of the lips exhibit typical arched or parabolic shapes, especially the arched contour of the upper lip. An active contour model based on shape prior is used for arched edge extraction. The initial contour curve is the circumscribed ellipse of the coarsely segmented region, parameterized as follows:

[0095] ,

[0096] in For the parameters on the contour The coordinate vector of the point, This represents the x-coordinate of the point, in pixels. The vertical coordinate of the point is in pixels, with superscript. This represents the transpose of a vector. Here are the curve parameters, with a range of... Let be the coordinate vector of the center of the ellipse. and The x and y coordinates of the center are respectively, in pixels. The radius of the major axis, in pixels. The minor axis radius is expressed in pixels. and Let be the unit vector along the principal axis of the ellipse. and Parameters The cosine and sine values.

[0097] The profile curve evolves under the drive of an energy function, which includes internal and external energy:

[0098] ,

[0099] in For total energy, This is internal energy used to maintain the smoothness of the curve and the bow-shaped constraint. External energy is used to drive the curve towards the image edge. Internal energy is defined as:

[0100] ,

[0101] in The elastic modulus is usually taken as... Dimensionless, used to control the tension of the curve. The stiffness coefficient is usually taken as... Dimensionless, used to control the curvature of a curve. This is the shape constraint coefficient, usually taken as... Dimensionless, used to maintain the arc-shaped feature. Let be the first derivative of the curve, and represent the direction of the tangent. Let be the second derivative of the curve, and represent the curvature. The square of the vector norm. The shape constraint term is defined as the deviation of the curve from the standard bow-shaped template. Indicates the parameter along the curve from 0 to 2. The integral of . External energy is defined as:

[0102] ,

[0103] in For contour points The grayscale value of the image at that location, For image gradient, This represents the square of the gradient strength; the negative sign indicates that the contour moves towards the edge with the larger gradient. This is achieved by minimizing the total energy. The contour position is iteratively updated using the gradient descent method until it converges to the edge of the lips.

[0104] Secondly, texture features of the lip region and facial skin tone region are extracted. The local binary mode operator is used to calculate the texture features. For each pixel in the image, its gray value is compared with the gray values ​​of its eight neighboring pixels. An 8-bit binary code is then generated.

[0105] ,

[0106] middle Center pixel The local binary pattern value ranges from [0, 255]. and The coordinates of the center pixel are in pixels. The grayscale value of the center pixel. For the first The grayscale value of each neighboring pixel. The value is the neighborhood pixel index. For symbolic functions, defined as when hour, when hour, 2 Power of 1 This represents the summation over 8 neighboring pixels. Histogram distributions of local binary patterns are calculated for the lip region and the facial skin color region, respectively. and ,in The value is a local binary pattern value, ranging from The local binary pattern value in the lip region is The number of pixels, The local binary pattern value in the facial skin color region is The number of pixels. Next, calculate the texture difference between the lips and facial skin tone. Use the chi-square distance to measure the difference between the two histograms:

[0107] ,

[0108] in This is a dimensionless measure of texture difference. The local binary pattern histogram of the lip region at the 1st Frequency of each box The local binary pattern histogram of the facial skin color region at the th Frequency of each box As a smoothing factor, it is usually taken as... Used to avoid division by zero errors. This represents the summation of all 256 possible local binary pattern values. The larger the texture difference value, the higher the texture distinction between the lips and facial skin tone.

[0109] Finally, fine segmentation is performed by integrating the bow-shaped edge features and texture differences. Based on the initial segmentation obtained after the active contour model converges, the boundary is further optimized according to texture differences. For pixels near the boundary, the similarity between their local binary patterns and the lip region and facial skin color region is calculated:

[0110] ,

[0111] ,

[0112] in For pixels Similarity to lip texture, ranging from [0,1]. For pixels Similarity to facial skin texture, ranging from [0,1]. The local binary mode value of this pixel. This represents the set of mode patterns in the local binary pattern histogram of the lip region. This represents the set of mode patterns in the local binary pattern histogram of facial skin color regions. This represents the minimum distance from the set of mode patterns. For temperature parameters, it is usually taken as... Dimensionless, used to control the steepness of the similarity function. It is an exponential function. If... If a pixel matches the texture similarity, it is classified as part of the lip region; otherwise, it is classified as part of the facial skin color region. This pixel reclassification based on texture similarity allows for fine-tuning of the segmentation boundaries, resulting in an accurate lip segmentation mask.

[0113] The motion compensation unit 22 is used to perform non-rigid motion compensation based on optical flow and feature point tracking technology, and employs sub-pixel level registration to constrain vascular stripe stability. During surgery, factors such as the patient's minute movements, breathing, and heartbeat can cause displacement and deformation in the monitoring area, affecting signal quality.

[0114] In this invention, motion compensation employs the following steps:

[0115] (1) The improved Lucas-Kanade algorithm was used to calculate the optical flow field between consecutive frames;

[0116] (2) Distribute feature points evenly within the ROI region and track these feature points using the KLT (Kanade-Lucas-Tomasi) feature tracking algorithm;

[0117] (3) Based on the displacement field of feature points, construct a non-rigid transformation model, including affine transformation and local deformation;

[0118] (3) Based on the displacement field of the feature points, a non-rigid transformation model is constructed, including affine transformation and local deformation. Specifically, the global affine transformation matrix A is first estimated from the feature point set using the RANSAC algorithm, and its form is:

[0119] ,

[0120] Where (x, y) are the original coordinates, , () represents the transformed coordinates. , , For deformation parameters, and The translation parameters are defined. Then, the residual between the actual displacement of each feature point and the predicted displacement of the global affine transformation is calculated to construct a local deformation model. Local deformation employs Moving Least Squares (MLS), where the transformed position of any point v in the image is determined. The calculation is as follows:

[0121] ,

[0122] in, The original position of the i-th feature point Let w_i(v) be the actual tracking position, and w_i(v) be the weight function based on spatial distance.

[0123] ,

[0124] It is a small constant (usually taken as ). To prevent division by zero. The final non-rigid transformation is a combination of a global affine transformation and a local deformation:

[0125] ,

[0126] (4) Apply the transformation model to perform geometric correction on the current frame to align it with the reference frame;

[0127] (5) Apply sub-pixel-level fine registration in the vascular stripe area to ensure the stability of the vascular texture.

[0128] The illumination normalization unit 23 is used to stabilize the video signal under varying illumination conditions by employing Retinex logarithmic domain correction and inter-frame luminance drift linear regression elimination techniques. Illumination conditions in operating rooms are typically unstable; even minor changes in light source intensity and direction can affect the quality of the video signal.

[0129] This invention employs the multi-scale Retinex algorithm for illumination normalization, and the processing procedure is as follows:

[0130] (1) Convert the original image to the logarithmic domain:

[0131] ,

[0132] in, The original image, with a size of A matrix representing an RGB three-channel image; For logarithmic field images, also for The matrix.

[0133] (2) Convolve the logarithmic domain image using Gaussian filters of different scales to obtain the estimated illumination components:

[0134] ,

[0135] in, For the first Illumination component estimation at each scale, for comparison with Matrices of the same dimension; For the first A Gaussian kernel of a certain scale is a A matrix of size, where The size of the Gaussian kernel; This indicates a convolution operation.

[0136] (3) For each scale, calculate the reflection component:

[0137] ,

[0138] in, For the first The reflection component at each scale is, in relation to Matrices of the same dimension.

[0139] (4) Weighted fusion of reflection components at different scales:

[0140] ,

[0141] in, For the final reflection component, for... Matrices of the same dimension; For the weighting coefficients, satisfying , usually take The scales are 15, 80, and 250 pixels, respectively, and the corresponding weights are usually set to... , , These weight settings are based on the following criteria:

[0142] (1) The multi-scale Retinex algorithm needs to balance the effects of detail preservation and illumination removal at different spatial scales;

[0143] (2) Small scale (15 pixels) is mainly used to preserve local fine blood vessel texture details, medium scale (80 pixels) is used to preserve medium structural features, and large scale (250 pixels) is used to remove large-scale lighting changes.

[0144] (3) Experimental verification shows that the mesoscale weight is slightly higher. This method can effectively remove the effects of lighting changes while preserving microcirculation signals, making it most suitable for medical video processing. Small-scale and large-scale data are weighted equally. It can achieve a balance between detail preservation and light removal;

[0145] (4) In a validation experiment with 10 subjects, this weighting configuration improved the signal-to-noise ratio of the extracted pulse signal by 15-20% and reduced the error rate by about 25% compared with other weighting configurations (such as uniform weighting or biased weighting).

[0146] (5) Linearly stretch the reflection component R to obtain the normalized image.

[0147] Specifically, linear stretching normalization employs the following steps: a) Calculate the reflection component. Statistical properties, including minimum value Maximum value mean and standard deviation b) Remove the influence of outliers and determine the effective range. ,in

[0148] ,

[0149] ,

[0150] c) Perform linear stretching mapping, Each element in Mapped to the range [0,1]:

[0151] ,

[0152] d) For elements that are out of range, truncate them:

[0153] ,

[0154] e) Finally, the normalized reflection component Convert back to RGB space to obtain a normalized image. An exponential mapping is used during the conversion to maintain consistency with the logarithmic domain conversion.

[0155] ,

[0156] This linear stretching and normalization method can enhance image contrast, making vascular structures more clearly visible, while maintaining the rationality of the overall brightness distribution of the image.

[0157] To eliminate inter-frame brightness drift, this invention also employs linear regression technology. Specifically, for continuous... Frame image (usually) (Approximately 0.25 seconds) Calculate the average brightness value per frame. Fitting a linear drift trend:

[0158] ,

[0159] in, For the first The average brightness value of the frame; The rate of change of brightness; This is the initial brightness value; This is due to random error. The parameters are estimated using the least squares method. and Then, brightness correction is performed on each frame based on the fitting results:

[0160] ,

[0161] in, For the original image, Both are brightness-corrected images. The matrix.

[0162] Reference Figure 4 The microcirculation assessment module 3 is connected to the image preprocessing module 2 and is used to calculate the local perfusion index (LPI) and pulsation stability index (PSI) of the fingernail bed and various parts of the lips based on a stable tissue reflection intensity sequence.

[0163] In a preferred embodiment of the present invention, the processing flow of the microcirculation assessment module 3 includes the following steps:

[0164] (1) The remote photoplethysmography (rPPG) base signal dominated by the green channel and the RGB-PCA fused signal were calculated within the segmentation mask. The green channel is most sensitive to changes in blood absorption and is the main source of the rPPG signal. At the same time, a stronger pulse signal can be extracted by fusing the RGB three-channel signals through principal component analysis (PCA).

[0165] Specifically, for each pixel, the signal sequence of its RGB three channels changing over time is extracted to form a time-color matrix. Where T is the length of the time window, typically taken as... (Corresponding to 2.5 seconds at 120fps). Then perform PCA decomposition on the matrix:

[0166] ,

[0167] in, It is a left singular matrix. It is a singular value diagonal matrix. Let be a right singular matrix, with the superscript T denoteing matrix transpose. Take... The first column is taken as the rPPG signal of that pixel. Finally, the rPPG signals of all pixels in the segmented region are averaged to obtain the rPPG base signal of that region.

[0168] (2) Perform bandpass filtering and phase alignment with the cardiac cycle to estimate the amplitude of the beat and the coefficient of variation of the cycle.

[0169] The bandpass filter used was a Butterworth filter with a passband range of 0.7–4 Hz (corresponding to a heart rate of 42–240 beats / minute) and an order of 4. After filtering, a peak detection algorithm was used to identify the peak value of each cardiac cycle and calculate the beat amplitude. (Peak-Valve Difference) and Periodic Variation Coefficient (Standard deviation of cardiac cycles divided by mean):

[0170] ,

[0171] in, This represents the duration of the i-th cardiac cycle. The standard deviation of the cardiac cycle duration. This represents the average duration of the cardiac cycle.

[0172] To accurately assess pulsation characteristics, this invention employs phase alignment technology to align waveforms from different cardiac cycles, thereby eliminating the influence of heart rate variations. Specifically, this is achieved by obtaining the instantaneous phase of the signal through Hilbert transform, and then resampling and aligning the waveform based on the phase information.

[0173] The formula for calculating the Hilbert transform is:

[0174] ,

[0175] in, The original signal, Let PV represent the Cauchy principal value integral, after the Hilbert transform. Combining the original signal and the transformed signal, we form the analytic signal:

[0176] ,

[0177] in The imaginary unit, The instantaneous amplitude of the signal. This represents the instantaneous phase of the signal.

[0178] (3) The local perfusion index (LPI) was constructed by the ratio of pulsation amplitude to tissue color reflectance, and micro-amplitude arrival time and hue swing amplitude were corrected.

[0179] The basic formula for calculating LPI is:

[0180] ,

[0181] in, The normalization coefficient is usually taken as... This resulted in a mean LPI of approximately 1.0 for healthy individuals. This represents the amplitude of the pulse, in any unit (au). This is the change in reflectance of the red-green channel, which is dimensionless. The modulation function is the arrival time of the micro-amplitude, with values ​​ranging from [0.7, 1]. is the modulation function for hue swing, with a value range of [0, 1.2].

[0182] Micro-amplitude arrival time (M-AT) refers to the time delay from the onset of the cardiac cycle to the detection of a significant amplitude change in the microcirculation region. It is usually related to vascular compliance and peripheral resistance, and is measured in seconds (s). Hue swing amplitude (H-Span) represents the range of hue component changes in the HSV color space during the cardiac cycle, reflecting the fluctuation of blood oxygen saturation, and is measured in degrees (°).

[0183] Specifically, the modulation function Adopt the following form:

[0184] ,

[0185] in, The modulation depth parameter is dimensionless. This is the slope parameter, in units of... ; For reference only, the unit is seconds (s); This is the hyperbolic tangent function. These parameter values ​​were determined through analysis of data from healthy volunteers. As M-AT increases, A decrease in the value indicates weakened perfusion.

[0186] Modulation function Adopt the following form:

[0187] ,

[0188] in, The maximum gain parameter is dimensionless. This is a sensitivity parameter, in degrees. ; The unit is degrees ( These parameters were determined through clinical validation. The higher the H-Span value, the better. The closer This indicates that the better the infusion condition.

[0189] (4) The pulsation stability index (PSI) is constructed using the period variation coefficient, the cycle phase jitter and the spectral peak narrowness.

[0190] The formula for calculating PSI is:

[0191] ,

[0192] in, is the periodic variation coefficient, dimensionless, and ranges from [0,1]. is a phase jitter index, dimensionless, with a range of [0,1]; is the spectral peak narrowing factor, dimensionless, ranging from [0,1]. The PSI value ranges from [0,1], with higher values ​​indicating more stable pulsations.

[0193] Phase jitter index It is obtained by calculating the standard deviation of the phase difference between adjacent cardiac cycles:

[0194] ,

[0195] in, Indicates the first The and the first Phase difference between cardiac cycles The standard deviation of the phase difference. The threshold is usually taken as... .

[0196] Spectral peak narrowness factor Then, by calculating the relative height and width of the dominant frequency peak in the signal power spectrum, we can obtain:

[0197] ,

[0198] in, This is the main peak value of the power spectrum. This represents the average power spectrum over the entire frequency band. For reference bandwidth, it is usually taken as , The -3dB bandwidth of the main peak is expressed in Hertz (Hz).

[0199] Reference Figure 5 The multi-site perfusion fusion module 4 based on the differential manifold is connected to the microcirculation assessment module 3 and is the core innovative module of this invention. It is used to map the local perfusion index and pulsatility index onto the differential manifold for characterization, perform time-frequency dual-channel fusion based on Riemannian geometry on the manifold, and integrate the information of each site through the attention weighting mechanism to generate the Global Perfusion Composite Index (GPCI).

[0200] Preferably, the multi-site perfusion fusion module 4 based on differential manifolds includes a manifold representation unit 41, a time-frequency fusion unit 42, and an attention weighting unit 43.

[0201] The manifold characterization unit 41 is used to map the local perfusion index and pulsation stability index of each part onto the differential manifold, construct the local geometric structure, and design the part-specific manifold geometric model.

[0202] Traditional methods simply treat perfusion signals as one-dimensional time series, ignoring their inherent complex geometric structure. This invention innovatively maps microcirculation signals from multiple sites onto a low-dimensional manifold for characterization, capturing their intrinsic geometric properties.

[0203] Specifically, the manifold representation unit 41 first extracts an 8-dimensional feature vector from the original LPI and PSI signals, which includes the following features:

[0204] Amplitude characteristics: peak amplitude, root mean square amplitude, amplitude variation coefficient;

[0205] Time-domain characteristics: rise time, half-peak width, and pulsation period stability;

[0206] Frequency domain characteristics: main frequency energy ratio, high frequency component ratio.

[0207] The extracted feature vectors are Z-score standardized to eliminate dimensional differences:

[0208] ,

[0209] in, This is the original feature vector, representing the 8 extracted features; This is a mean vector, representing the mean of each feature; This is the standard deviation vector, representing the standard deviation of each feature; This is the standardized feature vector.

[0210] Next, the initial manifold structure is constructed and individualized based on patient baseline data. The manifold adaptation process consists of three phases:

[0211] Rapid adaptation period (0-10 minutes): Constructing the initial manifold based on historical data from healthy individuals;

[0212] Individualized adjustment period (10–30 minutes): Fine-tuning manifold parameters based on patient baseline data;

[0213] Stable operation period (after 30 minutes): The manifold structure is updated every 30 minutes to adapt to long-term changes in the patient's condition.

[0214] A local coordinate system is constructed on the manifold, with the first principal direction aligned with the axis of change in pulsation amplitude and the second principal direction aligned with the axis of change in pulsation frequency. This design allows the coordinate system to intuitively reflect the main changing characteristics of physiological signals. Other directions are determined through a Schmitt orthogonalization process, forming a complete local coordinate basis.

[0215] The Schmidt orthogonalization process is represented as:

[0216] ,

[0217] in, For the original vector, These are the orthogonalized basis vectors. This indicates the inner product operation.

[0218] In addition, the manifold characterization unit also calculates the following geometric quantities to characterize the local geometric properties of the current infusion state:

[0219] 1. Local curvature: Reflects the degree of nonlinearity in changes in perfusion state; the calculation formula is:

[0220] ,

[0221] in, and These are the first and second basic form matrices, respectively, determined by the metric tensor in the local coordinate system. yes A matrix representing the metric relationship between parameterized coordinates; Too The matrix represents the degree of curvature of the manifold; Represents the determinant of a matrix; is the Gaussian curvature, a scalar measure of the degree of curvature of a manifold.

[0222] 2. Local volume: Reflects the diversity of perfusion states, obtained by calculating the square root of the determinant of the local metric tensor:

[0223] ,

[0224] in, For the local Riemannian metric tensor, it is A positive definite matrix; It is a local volume element and is a scalar.

[0225] 3. Local entropy: The amount of information carried by a quantized perfusion signal, calculated using the following formula:

[0226] ,

[0227] in, For the first in the local state space The probability distribution of each state is obtained by kernel density estimation of historical trajectories; To determine the discretization resolution of the state space, we typically take... ; Information entropy is a scalar quantity, and its unit is bit.

[0228] Finally, considering the different physiological characteristics of the fingernail bed and lips, the manifold characterization unit designed a site-specific manifold subspace:

[0229] The manifold subspace of the fingernail bed reflects the characteristics of sympathetic nerve regulation and is more sensitive to low-frequency changes (0.04–0.15 Hz).

[0230] The lip manifold subspace reflects the central circulatory state and has a more significant response to the heart rate frequency band (0.7–2 Hz).

[0231] A mapping relationship is established between the two subspaces to capture the physiological connections between different parts, providing a geometric basis for subsequent fusion. Specifically, the mapping relationship is established using a parallel transport method based on Riemannian geometry, including the following steps:

[0232] (1) First, for each part eigenvectors Calculate its value in its respective manifold The above represents ;

[0233] (2) In each manifold Build tangent space And calculate the local orthogonal basis. ,in The intrinsic dimension of a manifold (usually taken as...) );

[0234] (3) Based on historical physiological data, calculate the Riemann correlation matrix between manifolds in different parts. Defined as:

[0235] ,

[0236] in, Represents the inner product under the Riemannian metric. The number of samples;

[0237] (4) For the correlation matrix Perform singular value decomposition:

[0238] ,

[0239] in, and It is an orthogonal matrix. It is a diagonal matrix containing singular values;

[0240] (5) Constructing the mapping operator :

[0241] ,

[0242] The mapping operator will part Vectors in the tangent space Mapped to part The tangent space is used to establish the correspondence between physiological signal changes in two locations. Mapping operator It can capture the physiological connections between different parts of the body and achieve the synergistic fusion of information.

[0243] The time-frequency fusion unit 42 is used to perform geodesic state estimation and frequency domain coherence alignment on the differential manifold, thereby achieving the collaborative fusion of time-domain and frequency-domain information.

[0244] Traditional fusion methods directly average in Euclidean space, neglecting the nonlinear characteristics of physiological signals. This invention innovatively achieves synergistic fusion of the time and frequency domains on a Riemannian manifold, preserving the geometric properties of the signal.

[0245] In the time-domain fusion path, the time-frequency fusion unit 42 executes the geodesic state estimation algorithm, including state prediction and updating along the manifold geodesic. The specific process is as follows:

[0246] (1) State prediction steps: Predict the current state's position on the manifold based on historical trajectories. The prediction process uses geodesic extrapolation, rather than simple linear extrapolation:

[0247] ,

[0248] in, The predicted state represents the predicted feature vector; The state at the previous time step represents the feature vector at the previous time step. Let be the velocity vector at the previous moment, representing the rate of change of the eigenvector; The time interval is expressed in seconds (s). For the point Along vector The exponential mapping of directions represents the direction from point The operation of proceeding along the geodesic line.

[0249] (2) Measurement update steps: Adjust the state estimate based on the new observations. The update process considers distance metrics on the manifold:

[0250] ,

[0251] in, The updated state represents the updated feature vector; For new observations, this represents a new feature vector obtained from the sensor; This is the Kalman gain matrix, which controls the weights of the predicted and observed values. From point Time The logarithmic mapping, representing from arrive The geodesic tangent vector.

[0252] (3) Adaptive gain adjustment: dynamically adjust the Kalman gain based on signal quality score and prediction error history. .

[0253] ,

[0254] in, The prediction error covariance matrix represents the prediction uncertainty. Let be the observation error covariance matrix, and represent the observation noise. This is the Kalman gain matrix.

[0255] Observation error covariance Related to Signal Quality Index (SQI):

[0256] ,

[0257] in, This is an adjustment coefficient, usually taken as 1.5, and is dimensionless. The signal quality score is given, ranging from [0,1], and is dimensionless. The baseline covariance matrix is ​​determined through offline analysis. Thus, when signal quality is good, the weight of observations increases; when signal quality is poor, the weight of predictions increases.

[0258] In the frequency domain fusion path, the time-frequency fusion unit 42 performs adaptive wavelet transform on the signals of each part, extracts the energy distribution spectrum and phase synchronization characteristics, and performs feature alignment and fusion on the spectral manifold.

[0259] Specifically, the continuous wavelet transform (CWT) is first used to perform time-frequency analysis on the signal:

[0260] ,

[0261] in, The original signal is a time series. The wavelet mother function (Morlet wavelet is used in this invention); is a scale parameter, inversely proportional to frequency, and its unit is seconds (s). This is the translation parameter, representing the time position, in seconds (s); The wavelet coefficients are complex numbers. Indicates complex conjugation; It represents the integral from negative infinity to positive infinity.

[0262] Wavelet transform provides a time-frequency representation of the signal, allowing for the extraction of spectral features, including energy distribution and phase information across frequency bands. For energy distribution, the relative power of each frequency band is calculated.

[0263] ,

[0264] in, For frequency The relative power at a given point is dimensionless and ranges from [0,1]. For frequency wavelet coefficients at This represents the squared modulus of the wavelet coefficients, i.e., the power. This represents summing over all frequency points.

[0265] For phase synchronization characteristics, calculate the phase locking value (PLV) between signals at different locations:

[0266] ,

[0267] in, and The two signals are at different frequencies. ,time The phase at a given point, expressed in radians (rad); Number of time points; The imaginary unit; Represents the phase in complex form; For frequency The phase lock value at the location ranges from [0,1]. A larger value indicates higher phase synchronization.

[0268] Based on energy distribution and phase synchronization information, features of various parts are aligned and fused on the spectral manifold. The spectral manifold is a Riemannian manifold defined on the power spectral density space, and its metric reflects the similarity of the spectral distribution. The fusion process includes the following steps:

[0269] First, construct the geometric structure of the spectral manifold. Then, calculate the power spectral density of the signal at each location. Consider them as points on the manifold, where For the first Each part at frequency The power spectral density at that point is expressed in terms of energy per hertz. For the part index, the present invention takes Indicates the nail bed, Indicates lips, Let be the frequency variable, with units of Hertz. A Riemannian metric is defined on the spectral manifold, using Fisher information metric, and its differential form is:

[0270] ,

[0271] in Square the distance element on the spectral manifold. For discrete frequency points, For frequency point index, For frequency Power spectral density at that point The derivative of the power spectral density, As a weighting factor, it makes the frequency components with lower power have greater relative importance. This represents the summation over all discrete frequency points. Two spectra. and The geodesic distance between them is:

[0272] ,

[0273] in Power spectrum and The geodesic distance between them, expressed as the square root of energy per hertz. For the first power spectrum at frequency Power spectral density at that point For the second power spectrum at frequency Power spectral density at that point and These are the square roots of the power spectral density, This represents summing all frequency points and then taking the square root. Next, spectral alignment is performed to eliminate frequency shifts between different locations. Due to physiological differences and measurement conditions, the dominant frequency peaks in different locations may have slight shifts. A cross-correlation-based spectral alignment method is used for the power spectra of the fingernail bed and lips. and Calculate the frequency domain cross-correlation function:

[0274] ,

[0275] in For frequency offset Cross-correlation value at time, This is the frequency offset, measured in Hertz. Frequency point For the fingernail bed at frequency Power spectral density at that point For the lips at frequency Power spectral density at that point This represents summation over all frequency points. The goal is to find the frequency offset that maximizes the cross-correlation value.

[0276] ,

[0277] in This is the optimal frequency offset, measured in Hertz. The independent variable that makes the function reach its maximum value. Then, frequency shift correction was performed on the power spectrum of the lips:

[0278] ,

[0279] in The aligned lip power spectrum, in units of energy per hertz. This indicates that the original power spectrum is shifted along the frequency axis. After alignment, the dominant frequency peaks of each component are positioned in the same location, facilitating subsequent fusion. Next, the fusion weights for each component are calculated on the spectral manifold. These fusion weights are determined based on two factors: the spectral quality score and the coherence of the frequency components. Spectral quality score Defined as the ratio of the main peak energy to the total energy:

[0280] ,

[0281] in For the first The spectral quality score for each region is dimensionless and ranges from [0,1]. This represents the maximum value of the power spectrum, i.e., the main peak energy. The total energy of the power spectrum is obtained by summing the power at all frequency points. The coherence of the frequency components is determined by the phase-locking value. Quantization, the value of which has been defined previously. At each frequency point. The fusion weights for each part are:

[0282] ,

[0283] in For the first Each part at frequency The fusion weights at each point are dimensionless and range from [0,1], satisfying the following conditions: , For the first Spectral quality score for each part For the first Each part at frequency Phase lock value at that point, Indicates all parts Summation. The weighting calculation adaptively considers signal quality and phase synchronization characteristics, assigning greater weight to components with higher quality and coherence. Finally, weighted fusion is performed on the spectral manifold. The fused power spectrum. The calculation is as follows:

[0284] ,

[0285] in For the fused power spectrum at frequency The power spectral density at that point is expressed in terms of energy per hertz. For the first Each part at frequency The fusion weight at the point, For the first The power spectrum after alignment of each part is at frequency Power spectral density at that point The square root of the power spectral density is used to ensure that the fusion is performed in the amplitude domain rather than the power domain to maintain linearity. This represents summation over all parts, with the outermost layer squared. The amplitude-domain fusion result is transformed back into the power domain. This fusion method, performed within the geometric framework of the spectral manifold, better preserves the statistical properties and physiological significance of the spectrum compared to simple linear combination. The fused spectrum, along with the time-domain fusion result, is used to generate the final whole-body perfusion index.

[0286] Finally, the time-frequency fusion unit 42 integrates the fusion results of the time-domain path and the frequency-domain path to generate a comprehensive perfusion state characterization. The weights of the two paths are dynamically adjusted according to the signal characteristics, typically with a time-domain path weight of 0.6-0.8 and a frequency-domain path weight of 0.2-0.4.

[0287] The attention weighting unit 43 is used to calculate dynamic attention weights based on signal quality scores of each site, clinical context information and historical predicted performance, integrate information from multiple sites, and generate the General Perfusion Index (GPCI).

[0288] The calculation of dynamic attention weights considers the following three aspects:

[0289] (1) Base weight: determined by the Signal Quality Index (SQI) of each part. The SQI is calculated by evaluating the signal-to-noise ratio, stability, and physiological relevance of the signal, and ranges from [0,1]. The specific calculation formula is as follows:

[0290] ,

[0291] in, is the normalized signal-to-noise ratio, ranging from [0,1], dimensionless; Stability is the signal stability index, ranging from [0,1], dimensionless; Physio_relevance is the physiological relevance index, ranging from [0,1], dimensionless. , and For the weighting coefficients, satisfying Typically, these values ​​are 0.4, 0.3, and 0.3 respectively; SQI is the signal quality score, ranging from [0,1], and is dimensionless.

[0292] (2) Context Modulation: The weights are adjusted according to the surgical stage and physiological state. For example, during the initiation of cardiopulmonary bypass, the weight of the lip area should be appropriately increased because peripheral circulatory changes lag behind central circulatory changes at this time. The context modulation factor Context_factor is determined based on the surgical stage, body temperature, and medication use, and ranges from [0.7, 1.3], and is dimensionless.

[0293] (3) Historical performance weighting: The weights are adjusted based on the historical prediction accuracy. For parts with higher prediction accuracy, their weights are increased accordingly. The historical performance factor Hist_factor is determined by calculating the error between the predicted value and the actual value over a past period (usually 30 minutes), and its range is [0.8, 1.2], dimensionless.

[0294] Taking all the above factors into account, location attention weights The calculation formula is:

[0295] ,

[0296] in,, For part The attention weights, ranging from [0,1], are dimensionless and satisfy the following conditions: ; For part Signal quality score; For part Context modulation factor; For part Historical performance factor; Indicates all parts Sum.

[0297] Based on attention weights, information from multiple body sites is integrated to generate the General Perfusion Index (GPCI):

[0298] ,

[0299] in, The overall perfusion index is dimensionless. For part The local perfusion index is dimensionless. For part The pulsatility stability index is dimensionless; and Let LPI and PSI be the relative weights, satisfying Typically, these values ​​are 0.7 and 0.3 respectively. Indicates all parts Sum.

[0300] To ensure the stability of GPCI, this invention also designs a stability guarantee mechanism to limit the rate of change of GPCI at adjacent time points:

[0301] ,

[0302] in, The original calculated GPCI value is dimensionless. The GPCI value at the previous moment is dimensionless. This is the final value after limiting the rate of change; it is dimensionless. The maximum allowable change is typically set to 5% of the previous GPCI value. Indicates will Limited to Operations within the range, i.e. .

[0303] Reference Figure 8 The trend prediction and early warning module 5 is connected to the multi-site perfusion fusion module 4 based on the differential manifold. It is used to construct a dynamic system model on the manifold based on the whole body perfusion comprehensive index, predict the trend of circulatory state changes within the future time window, and generate early warning information when the risk exceeds the threshold.

[0304] In a preferred embodiment of the present invention, the trend prediction and early warning module 5 includes the following functions:

[0305] (1) The time evolution of the total perfusion index is modeled as a nonlinear dynamic system on the manifold.

[0306] Traditional early warning systems typically rely on simple threshold judgments and lack a deep understanding of system dynamics. This invention innovatively models the cyclic system as a nonlinear dynamic system on a manifold to capture its complex dynamic behavior.

[0307] The system state representation consists of three parts: the current state (the position of the GPCI and its first derivative on the manifold), the historical trajectory (the path of the system on the manifold over the past 60 minutes), and the steady-state position (the steady-state attractor determined based on the patient's baseline).

[0308] Based on historical data, a vector field is learned on the manifold, reflecting the system's evolutionary trend and rate. The vector field learning employs a combination of local linear and global nonlinear models to balance computational efficiency and model expressive power. The vector field is updated every 15 minutes to adapt to changes in the patient's condition.

[0309] By analyzing the vector field, stable and unstable equilibrium points on the manifold are identified. Stable equilibrium points correspond to states that the system tends to maintain (such as a healthy cyclic state), while unstable equilibrium points correspond to states that the system may rapidly deviate from (such as a critical point of cyclic function). The geodesic distances from the current state to each equilibrium point are calculated to assess the overall stability of the system.

[0310] (2) Calculate early warning indicators such as geodesic deviation, autocorrelation function decay rate and manifold entropy growth rate.

[0311] This invention designs a series of early warning indicators to capture early signs that a system is approaching a critical state. These indicators are divided into two categories: first-order indicators and second-order indicators.

[0312] First-order indices are based on direct state measurements and include:

[0313] Geodesic deviation: The degree of deviation between the actual trajectory and the predicted trajectory, calculated using the following formula:

[0314] ,

[0315] in, This is the geodesic deviation, dimensionless; The actual state represents the feature vector actually observed. The predicted state represents the feature vector predicted by the model. Let be the geodesic distance on the manifold, representing the distance from point . Time The shortest path length, dimensionless.

[0316] Rate of change of velocity: the time derivative of the rate of change of the system state, calculated by the following formula:

[0317] ,

[0318] in, Let be the velocity vector of state change, representing the rate of change of the eigenvector; The norm of the velocity vector represents the magnitude of the rate of change. The rate of change of velocity, i.e., the magnitude of acceleration, is expressed in units of per square second ( ). ) It represents the derivative with respect to time.

[0319] Acceleration direction: The angle between the system's state acceleration vector and the steady-state direction, calculated using the following formula:

[0320] ,

[0321] in, Let be the acceleration vector, and let represent the rate of change of the velocity vector. The direction vector from the current state to the nearest stable equilibrium point; Indicates the inner product operation; and Representing vectors respectively and The norm; Represents the inverse cosine function; The angle is expressed in radians (rad) or degrees (°).

[0322] Second-order indices are based on dynamic characteristics and include:

[0323] Autocorrelation function decay rate: reflects the system's memory effect, and is calculated using the following formula:

[0324] ,

[0325] in, The autocorrelation function represents the time delay between the signal and itself. The correlation after that is dimensionless and ranges from [-1, 1]. This is a time delay, measured in seconds (s). The characteristic time delay is typically set to 0.1 times the characteristic time scale, with units of seconds (s). Represent the natural logarithm function; Indicates to The derivative; The autocorrelation function decay rate, in units of per second ( The decrease in the decay rate of the autocorrelation function is an important sign that the system is approaching a critical transition.

[0326] State recovery rate: The rate at which a system returns to equilibrium after being disturbed, calculated using the following formula:

[0327] ,

[0328] in, This is the velocity vector of the state change; The direction vector from the current state to the nearest stable equilibrium point; Indicates the inner product operation; Representing vectors The square of the norm; State recovery rate, in units of per second ( A positive value indicates that the system is moving away from a steady state, while a negative value indicates that the system is approaching a steady state. The larger the absolute value, the faster the speed. A decrease in the state recovery rate indicates a weakening of system stability and is a precursor to a critical transition.

[0329] Fluctuation amplitude: The magnitude of state fluctuation caused by random disturbances, obtained by calculating the standard deviation of the state within a short time window;

[0330] ,

[0331] in, is the total perfusion index at time i; W is the length of the time window, usually 60 seconds; The mean of GPCI within the time window; Let be the standard deviation of the GPCI, representing the fluctuation range, which is dimensionless. An increase in fluctuation range is also a typical characteristic of a system approaching its critical point.

[0332] The logic for merging indicators varies depending on the time scale of the warning: short-term warnings focus on first-order indicators, while medium- and long-term warnings consider both first- and second-order indicators. The weights of each indicator are dynamically adjusted based on the historical prediction accuracy to improve the precision of the warnings.

[0333] (3) Design a three-level early warning triggering mechanism for short-term (5-15 minutes), medium-term (15-30 minutes) and long-term (30-60 minutes).

[0334] This invention designs a multi-scale early warning triggering mechanism to address the risk prediction needs at different time scales.

[0335] The triggering conditions for a short-term warning (5-15 minutes) are: the geodesic deviation exceeds the threshold and the acceleration direction is towards the unstable region. Specifically, a short-term warning is triggered when the following conditions are met:

[0336] ,

[0337] in, This refers to geodesic deviation; It is a threshold multiple, usually taken as 1.5, and is dimensionless; The mean geodesic deviation of the patient at baseline is usually taken as the average value of a 10-minute window during the stable period; The angle between the direction of acceleration and the steady-state direction; This is the angle threshold, usually set to 110 degrees.

[0338] The modulation factors for short-term warnings include the current signal quality and the system noise level. During surgical procedures, to reduce false positives, the trigger threshold is typically increased. Increased to 2.0.

[0339] The trigger condition for a mid-term alert (15-30 minutes) is an abnormally low autocorrelation function decay rate and a decreased state recovery rate. Specifically, a mid-term alert is triggered when the following conditions are met:

[0340] ,

[0341] in, The decay rate of the autocorrelation function; It is a threshold multiple, usually taken as 0.7, and is dimensionless; The mean autocorrelation function decay rate at the patient's baseline state; State recovery rate; It is a threshold multiple, usually taken as 0.6, and is dimensionless; This represents the mean recovery rate of patients at baseline.

[0342] The confidence assessment of the medium-term alert is based on statistical analysis of historical similar trajectories, and the alert level is graded according to the severity of the deviation from the trend. Once a medium-term alert is triggered, the system will continue to monitor short-term indicators. If the short-term indicators also begin to deteriorate, the alert level will be upgraded.

[0343] The trigger condition for a long-term warning (30–60 minutes) is that the gradual change characteristics of the manifold geometry conform to a critical deceleration mode. Specifically, a long-term warning is triggered when a combination of the following characteristics is observed:

[0344] The autocorrelation function decay rate continues to decrease, with an average decrease of over 30%.

[0345] The fluctuation range of the state gradually increased, and the increase exceeded 50%;

[0346] The manifold entropy increases beyond a threshold, typically 200% of the baseline value.

[0347] Long-term early warnings employ a mechanism of cross-validation using multiple independent indicators to reduce false positive rates. The warning content includes predictions of the timing and severity of potential risks, generated based on historical data and statistical models.

[0348] (4) Generate a visual early warning result that includes a risk heat map, a status trajectory map and a contribution analysis map.

[0349] To enhance the interpretability of early warnings, this invention designs a variety of visual representation methods:

[0350] Risk heatmap: Displays the distribution of risks within a future time window. The horizontal axis represents time, the vertical axis represents risk level, and the color intensity indicates the probability of risk.

[0351] State trajectory diagram: shows the system's path of motion on the manifold, marks stable and unstable equilibrium points, and predicts the future trajectory.

[0352] Contribution Analysis Chart: Shows the contribution of each part to the early warning system, helping medical staff identify the main sources of the problem.

[0353] In addition, textual explanations are generated, including core conclusions (concise risk assessment results), chains of evidence (changes in key indicators supporting the conclusions), and recommended measures (intervention recommendations based on the warning level).

[0354] like Figure 1 As shown, the system of the present invention also includes an environmental adaptation module 7 and a quality control engine 8, which are used to improve the robustness and reliability of the system.

[0355] The environmental adaptation module 7 monitors changes in ambient light conditions and performs compensation adjustments to ensure stable system operation under fluctuating intraoperative lighting conditions. Specifically, this module monitors ambient light conditions in real time through an external illuminance sensor or brightness changes in a reference area in the video. When a change in light intensity exceeding ±10% is detected, an automatic compensation mechanism is triggered to adjust camera parameters or signal processing parameters to maintain signal quality.

[0356] The environmental adaptation module 7 also includes a spectral self-calibration function, which analyzes standard grayscale reference blocks in the video to correct color shifts in real time, ensuring color consistency under different lighting conditions. This is particularly important for micro-cycle evaluation based on color changes.

[0357] The Quality Control Engine 8 is used to evaluate video quality and signal reliability in real time. When it detects interference factors such as motion artifacts, dry lips, and nail bed coating, it automatically adjusts the processing strategy and provides quality alerts. This engine employs a multi-level quality assessment strategy:

[0358] (1) Image quality assessment: Detect problems such as blurriness, overexposure / underexposure, and insufficient contrast;

[0359] (2) Signal quality assessment: Analyze indicators such as signal-to-noise ratio, signal stability, and physiological rationality;

[0360] (3) Interference factor identification: Detect specific interferences such as motion artifacts, reflections, and occlusions.

[0361] When a quality issue is detected, the Quality Control Engine 8 will take appropriate remedial measures, such as adjusting the ROI position, increasing motion compensation intensity, and enabling specific filtering strategies. Simultaneously, it will send quality alerts to the user interface to guide medical staff in making necessary adjustments.

[0362] The clinical interaction and data integration module 6 is connected to the trend prediction and early warning module 5 to display the change curve of the systemic perfusion comprehensive index, early warning information, and exchange data with the hospital monitoring system.

[0363] In a preferred embodiment of the present invention, the clinical interaction and data integration module 6 includes the following functions:

[0364] (1) Display perfusion curves at the site level and the whole body level, and multi-scale risk heat maps.

[0365] Clinical Interaction and Data Integration Module 6 provides an intuitive visual interface, including:

[0366] Real-time perfusion curve: Simultaneously displays the time series of fingernail bed LPI, lip LPI and GPCI, enabling medical staff to clearly understand the patient's circulatory status at a glance;

[0367] Multi-scale risk heat map: Displays the risk distribution within different time windows (5-60 minutes) in the form of a heat map, with the color intensity indicating the risk level;

[0368] State trajectory diagram: Visualize the motion trajectory of a system on a manifold in two-dimensional or three-dimensional space to help understand the dynamic changes of cyclic states.

[0369] (2) Provide early warning interpretation information, including the contribution analysis of each part to the early warning and the prediction of risk development trend.

[0370] To enhance the interpretability and clinical credibility of alerts, the Clinical Interaction and Data Integration Module 6 provides detailed explanatory information for alerts:

[0371] Contribution analysis: Display the contribution ratio of each indicator to the early warning in the form of bar charts or pie charts;

[0372] Risk Trend Forecast: Shows the forecast curve and confidence interval of future changes in GPCI;

[0373] Key indicator changes: Highlights the key indicators that trigger the warning and the degree of abnormality.

[0374] (3) Enable bidirectional data exchange with anesthesia monitors, extracorporeal circulation systems and electronic medical record systems.

[0375] Clinical Interaction and Data Integration Module 6 supports integration with existing medical equipment and systems within the hospital, including:

[0376] Anesthesia monitor: Receives routine vital signs data such as heart rate, blood pressure, and blood oxygen saturation;

[0377] Extracorporeal circulation system: Receives extracorporeal circulation parameters such as pump flow rate, pressure, and temperature;

[0378] Electronic medical record system: obtains basic patient information, medication records, surgical stage markers, etc.

[0379] Data exchange uses standard medical data protocols (such as HL7 and DICOM) to ensure system compatibility and interoperability. Simultaneously, GPCI and early warning information are transmitted back to the hospital's internal system, enabling information sharing and integration.

[0380] (4) Record medical and nursing intervention measures and intervention effects, and update individualized early warning thresholds based on intervention-response data.

[0381] Clinical Interaction and Data Integration Module 6 also has learning and adaptive capabilities:

[0382] Intervention Records: Provides an interface for healthcare professionals to quickly record interventions (such as medication administration, body positioning, etc.);

[0383] Effectiveness evaluation: Track the changes in GPCI after intervention and automatically calculate the response effect;

[0384] Threshold Update: Based on intervention-response data, optimize individualized early warning thresholds to improve the accuracy of future early warnings.

[0385] This closed-loop design enables the system to continuously learn and improve, with the accuracy and timeliness of warnings constantly increasing over time.

[0386] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A real-time monitoring and early warning system for the circulatory status of cardiovascular surgery patients, characterized in that, include: The video acquisition module is used to simultaneously or alternately acquire high frame rate visible light video of the patient's fingernail bed and lips; The image preprocessing module is communicatively connected to the video acquisition module and is used to perform region segmentation, motion compensation and illumination normalization on the visible light video to obtain a stable tissue reflectance intensity sequence. The microcirculation assessment module is communicatively connected to the image preprocessing module and is used to calculate the local perfusion index and pulsation stability index of various parts of the fingernail bed and lips based on the stable tissue reflection intensity sequence. The multi-site perfusion fusion module based on differential manifolds is communicatively connected to the microcirculation evaluation module and is used for: The local perfusion index and pulsatility index are mapped onto a differential manifold for characterization. Perform time-frequency dual-channel fusion based on Riemannian geometry on the differential manifold; By integrating information from various parts through an attention-weighted mechanism, a comprehensive perfusion index for the whole body is generated; The trend prediction and early warning module is communicatively connected to the multi-site perfusion fusion module based on the differential manifold. It is used to construct a dynamic system model on the manifold based on the whole-body perfusion comprehensive index, predict the trend of circulatory state changes within the future time window, and generate early warning information when the risk exceeds the threshold. The clinical interaction and data integration module is communicatively connected to the trend prediction and early warning module, and is used to display the change curve of the systemic perfusion comprehensive index, early warning information, and exchange data with the hospital monitoring system.

2. The system according to claim 1, characterized in that, The video acquisition module has a frame rate of no less than 120fps and features adaptive exposure and polarization anti-reflection functions to reduce saturation artifacts caused by highlights on the wet surfaces of the nail bed and lips.

3. The system according to claim 1, characterized in that, The image preprocessing module includes: The region segmentation unit is used to perform fine segmentation of the nail bed and lip regions based on YCrCb and HSV dual-color gamut skin color screening technology combined with morphological and texture feature priors. The motion compensation unit is used to perform non-rigid motion compensation based on optical flow and feature point tracking technology, and employs sub-pixel level registration to constrain vascular stripe stabilization. The illumination normalization unit is used to stabilize video signals under varying illumination conditions by employing Retinex logarithmic domain correction and inter-frame luminance drift linear regression elimination techniques.

4. The system according to claim 1, characterized in that, The microcirculation assessment module is used for: The green channel-dominated remote photovolume plot base signal and the RGB-PCA fused signal are calculated within the segmentation mask; Perform bandpass filtering and phase alignment with the cardiac cycle to estimate the beat amplitude and the coefficient of variation of the cycle; A local perfusion index was constructed by comparing pulsation amplitude with tissue color reflectance, and corrections were made for micro-amplitude arrival time and hue swing amplitude. A pulsatility stability index was constructed using the period variation coefficient, cycle phase jitter, and spectral peak narrowness.

5. The system according to claim 1, characterized in that, The multi-site perfusion fusion module based on differential manifolds includes: Manifold characterization unit is used to map the local perfusion index and pulsatility stability index of each part onto the differential manifold, construct the local geometric structure, and design part-specific manifold geometric models; The time-frequency fusion unit is used to perform geodesic state estimation and frequency domain coherence alignment on the differential manifold to achieve collaborative fusion of time-domain and frequency-domain information; The attention-weighted unit is used to calculate dynamic attention weights based on signal quality scores of each site, clinical context information, and historical predicted performance, and to integrate information from multiple sites to generate a comprehensive whole-body perfusion index.

6. The system according to claim 5, characterized in that, The manifold characterization unit is used for: Extract an 8-dimensional feature vector that includes amplitude features, time-domain features, and frequency-domain features; An initial manifold structure was constructed and individualized based on patient baseline data; Construct a local coordinate system on the manifold, where the first principal direction is aligned with the axis of change in pulsation amplitude and the second principal direction is aligned with the axis of change in pulsation frequency; Calculate local curvature, local volume, and local entropy to characterize the geometric properties of the current perfusion state.

7. The system according to claim 5, characterized in that, The time-frequency fusion unit is used for: In the time domain path, a geodesic state estimation algorithm is executed, including state prediction and updating along the manifold geodesic, and dynamic adjustment of gain based on signal quality score and prediction error; In the frequency domain path, adaptive wavelet transform is performed on the signals of each part to extract the energy distribution spectrum and phase synchronization characteristics, and feature alignment and fusion are performed on the spectral manifold; By integrating the fusion results of the time-domain path and the frequency-domain path, a comprehensive perfusion state characterization is generated.

8. The system according to claim 1, characterized in that, The trend prediction and early warning module is used for: The temporal evolution of the total perfusion index is modeled as a nonlinear dynamic system on a manifold; Learn vector fields on manifolds based on historical data to identify stable and unstable equilibrium points on manifolds; Calculate early warning indicators such as geodesic deviation, autocorrelation function decay rate, and manifold entropy growth rate; The design incorporates a three-tiered early warning triggering mechanism: short-term (5-15 minutes), medium-term (15-30 minutes), and long-term (30-60 minutes). Generate visualized early warning results that include risk heatmaps, status trajectory diagrams, and contribution analysis diagrams.

9. The system according to claim 1, characterized in that, Also includes: The environmental adaptation module is used to monitor changes in ambient light conditions and perform compensation adjustments to ensure stable operation of the system under fluctuating intraoperative lighting conditions; The quality control engine is used to evaluate video quality and signal reliability in real time. When interference factors such as motion artifacts, dry lips, and nail bed coating are detected, it automatically adjusts the processing strategy and provides quality prompts.

10. The system according to claim 1, characterized in that, The clinical interaction and data integration module is used for: Displays perfusion curves at the site and systemic levels, and multi-scale risk heat maps; Provide early warning interpretation information, including an analysis of the contribution of each part to the early warning and a prediction of the risk development trend; It enables bidirectional data exchange with anesthesia monitors, extracorporeal circulation systems, and electronic medical record systems; Record medical interventions and their effects, and update individualized early warning thresholds based on intervention-response data.

Images