Platelet-rich plasma and bone marrow mesenchymal stem cells promote the proliferation and migration of human umbilical cord-derived mesenchymal stem cells

Abstract

The application discloses a non-contact blood oxygen measurement analysis method and system for a plateau environment, relates to the technical field of blood oxygen measurement analysis, and comprises the following steps: S1, based on a continuous face video stream, parallel processing is realized through a double-branch main network architecture, mutual supervision optimization is realized through a joint loss function, an environment self-adaptive weighting strategy is designed for fusion, and finally, blood oxygen prediction data is output; S2, multi-source sensor data and blood oxygen prediction data are fused, a blood oxygen fluctuation attribution model is constructed through a dynamic Bayesian network, and the probability distribution of a plateau disease and corresponding intervention instructions are output; S3, the probability distribution and the corresponding intervention instructions are received, user EPAS1 gene typing data and real-time vital signs are combined, and personalized execution strategies are generated. Through the early warning mechanism of double-path cross-validation, the application breaks through the limitations of traditional single-mode detection, fuses five core technologies, and realizes high robustness and high precision of low blood oxygen monitoring in a plateau environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of blood oxygen measurement analysis, and particularly relates to a non-contact blood oxygen measurement analysis method and system for a plateau environment. BACKGROUND

[0002] Video-based non-invasive physiological monitoring technology has attracted much attention in the field of health care due to its convenience and universality. By analyzing the photoplethysmography (rPPG) signal in the face video, the blood oxygen saturation (SpO2), respiratory rate (RR) and other key physiological parameters can be indirectly calculated. Specifically, the video signal contains the weak reflection intensity changes of the skin surface, which reflect the subcutaneous blood vessel pulsation; the rPPG signal is extracted from the video through a specific algorithm (such as blind source separation), and is synchronized with the heart activity; and the blood oxygen saturation is calculated based on the difference in absorption of red light / infrared light by hemoglobin through the rPPG signal.

[0003] In the plateau mountaineering activities, the prevention and treatment of hypoxic diseases (such as acute mountain sickness, high altitude pulmonary edema) highly depends on the accurate blood oxygen saturation monitoring. The existing technology generally has the defects of motion artifact interference, light noise interference and individual physiological differences (such as EPAS1 gene mutation) affecting signal accuracy. When the video signal is collected, the extreme environment (low pressure, low temperature, high ultraviolet radiation) will significantly affect the signal quality, resulting in errors and false alarms of blood oxygen monitoring. The patent with publication number CN110300545A discloses a wrist sensor type pulse blood oxygen measurement device and method, but the wrist-worn blood oxygen equipment does not fully suppress the micro-motion of the skin contact surface due to the mechanical structure, resulting in low-frequency motion noise (0.1-1Hz) when the wearer moves, which is superimposed with the effective blood oxygen signal frequency band (0.8-3Hz) in the time domain. In the walking or arm swinging scene, the amplitude of the low-frequency noise is more than 40% of the physiological signal baseline, causing signal baseline drift, blood oxygen saturation measurement error up to ±3%, and there is a problem of spectral confusion false alarm, which violates the requirement of ISO 80601-2-61:2017 Article 201.12.1.101 for signal effectiveness; the patent with publication number US6771994B2 discloses a pulse oximeter probe drop detection system, when the probe is partially or completely dropped (similar to sensor occlusion equipment failure) and the relevant signal exists at the same time as the true hypoxemia (pathological condition that may be accompanied by SpO2 drop), the probe may still detect the alternating current signal within the working range of the pulse oximeter, and it is not possible to clearly distinguish between equipment failure (such as the relevant failure state that should be prompted when the probe is dropped) and pathological risk (such as the relevant risk state that should be prompted when the true hypoxia occurs). In this case, the device may mistakenly regard the failure signal as a pathological signal or vice versa, resulting in confusion between failure and pathology.

[0004] In summary, the highland blood oxygen monitoring field needs to be broken through in the following aspects: dynamic anti-interference (motion / optical noise suppression); interpretable decision (abnormal attribution + intervention generation); genetic individualization (genotype stratified early warning). SUMMARY

[0005] The purpose of the present application is to overcome the deficiencies of the prior art and provide a non-contact blood oxygen measurement analysis method and system for highland environment.

[0006] The purpose of the present application is achieved by the following technical solutions:

[0007] In a first aspect, the present application provides a non-contact blood oxygen measurement analysis method for highland environment, comprising the following steps:

[0008] S1, based on continuous face video stream, parallel processing thereof through a double-branch main network architecture, the double-branch including an rPPG signal branch and an end-to-end prediction branch;

[0009] In the rPPG signal branch, the face ROI region is extracted through 3DCNN spatiotemporal modeling and optical flow tracking technology, and the rPPG signal is generated in combination with an adaptive skin color segmentation algorithm; then a feature extraction subnetwork is deployed in the middle of the rPPG branch, the spatiotemporal features in the rPPG signal are learned through a CNN plus LSTM network model, a nonlinear mapping relationship from pulse waveform to blood oxygen saturation is established, and a blood oxygen saturation prediction value is output;

[0010] In the end-to-end prediction branch, a Transformer encoder-decoder structure is adopted, a nonlinear mapping relationship from video frame sequence to blood oxygen saturation is learned, and thus motion patterns are captured;

[0011] A joint loss function is used to realize mutual supervision optimization between the double branches, and an environment adaptive weighting strategy is designed for fusion, and finally the blood oxygen prediction data is output;

[0012] S2, receiving the final blood oxygen prediction data, then fusing the multi-source sensor data and the blood oxygen prediction data, the multi-source sensor data including three-axis motion acceleration IMU, ambient light intensity LUX and barometric altimeter BARO; a blood oxygen fluctuation attribution model is constructed through a dynamic Bayesian network, and the probability distribution of highland symptoms and the corresponding intervention instructions are output; the highland symptoms include mirror reflection artifact, early pulmonary edema, equipment failure and normal fluctuation;

[0013] S3, receiving the probability distribution of highland symptoms and the corresponding intervention instructions, combining user EPAS1 genotyping data and real-time vital signs, and generating individualized execution strategies through a dynamic priority arbitration mechanism.

[0014] Based on the first aspect, the generating the rPPG signal in step S1 specifically comprises the following steps:

[0015] S101, using an end-to-end VVT (Video-Text Transformer) framework, using continuous face video as input, where T1 represents the number of frames, W represents the width of the video, H represents the height of the video, and C represents the number of channels; the spatio-temporal features of the video are extracted through a 3D convolution layer, which includes a three-dimensional convolution conv3D, a batch normalization BatchNorm, an activation function ReLU, and a maximum pooling layer MaxPool; the spatial and temporal dimensions are operated at the same time, and a feature map containing spatio-temporal information is output wherein represents a 3D convolution kernel with a size of wherein K represents a size parameter of the 3D convolution kernel;

[0016] S102, inputting the feature map into an encoder Transformer module, enhancing the timing information through a self-attention mechanism and then outputting, wherein the self-attention formula in the Transformer module is: wherein Q represents a query matrix, Key represents a key matrix, and V represents a value matrix, wherein d_k represents the dimension of the key vector, and T2 represents transposition;

[0017] S103, further compressing the features output by the Transformer module through a pooling layer, and mapping the spatio-temporal features to a first feature vector ; wherein the pooling operation includes global average pooling: N wherein N represents the total number of features, and each position represents a feature value;

[0018] S104, inputting the first feature vector into a fully connected layer FC to map out a predicted rPPG signal .

[0019] Based on the first aspect, in step S1, the end-to-end prediction branch and the rPPG signal branch share a main network architecture, based on 3D convolution, Transformer extracts spatio-temporal features and models timing, and the end-to-end prediction branch predicts a first blood oxygen saturation .

[0020] Based on the first aspect, step S1 further comprises the following steps: ​​​

[0021] S111, adding a sub-network in the rPPG signal branch to extract the second blood oxygen saturation and the basic respiratory frequency from the rPPG signal; the sub-network is a CNN plus LSTM network model, which extracts the local features of the rPPG signal through CNN and captures the time sequence dependence of the rPPG signal through LSTM;

[0022] The features of the rPPG signal are extracted through a 1DCNN : where W represents a convolution kernel, is a bias term;

[0023] Then an LSTM network is used to capture the time sequence dependence of the rPPG signal: where, is the hidden state of the tth time step, is the hidden state of the tth time step;

[0024] Finally, a fully connected layer is used to output the second blood oxygen saturation ;

[0025] S112, calculating the basic respiratory frequency of the rPPG signal;

[0026] The band-pass filtering is calculated by the formula , where is a band-pass filter with a passband of [0.15, 0.65] Hz, T3 is the current timestamp, is a time offset variable;

[0027] STFT spectrum analysis is performed , where N’ = 90 is a 3-second time window, is a Hanning window function, is a Fourier basis function;

[0028] The respiratory main frequency is detected and calculated: ;

[0029] Finally, the basic respiratory frequency is calculated by the formula , and its unit is times / minute.

[0030] Based on the first aspect, the joint loss function used in step S1 realizes mutual supervision optimization between the two branches, which specifically includes the following steps:

[0031] S121, combining the losses of the two branches by weighting to obtain a joint loss function L: , which ensures the collaborative learning of the two branches, where​ predicted blood oxygen saturation representing the rPPG signal branch, predicted blood oxygen saturation representing the video to blood oxygen saturation branch, true blood oxygen saturation, and hyper-parameter used to adjust the loss contribution of the two branches;

[0032] MSE is the standard mean square error loss, which is calculated as , where represents the true blood oxygen value of the ith sample, represents the predicted blood oxygen value of the ith sample, and N represents the number of samples;

[0033] S122, dynamically fusing the predicted blood oxygen saturations of the two branches , finally outputting a third blood oxygen saturation ; wherein and represent dynamic weights determined by the quality of each branch signal: , ; wherein is the signal-to-noise ratio output by the rPPG branch, is the signal-to-noise ratio output by the end-to-end branch.

[0034] Based on the first aspect, step S2 specifically comprises the following steps:

[0035] S201, based on the finally output third blood oxygen saturation , fusing multi-source sensor data, including three-axis acceleration , ambient light intensity , and air pressure altitude , wherein is the acceleration of the device in the horizontal left-right direction, is the acceleration of the device in the horizontal front-back direction, is the acceleration of the device in the vertical direction, is the atmospheric pressure;

[0036] S202, calculating blood oxygen variability SDNN by the formula for quantifying autonomic nervous regulation function, wherein T4 represents the window time, the window time T4=30, and the window time T4 conforms to the adaptive mechanism T4=30*(1+0.002h), wherein h represents the current altitude, represents the change amount of blood oxygen saturation at adjacent time points, i.e., the difference between the current blood oxygen value and the previous blood oxygen value , ;

[0037] S203, calculate motion-blood oxygen mutual information entropy by formula , for revealing the coupling relationship between motion artifact and blood oxygen fluctuation, wherein, represents the joint probability distribution of the third blood oxygen saturation and the vertical acceleration , represents the probability distribution of the third blood oxygen saturation, represents the probability distribution of the vertical acceleration ;

[0038] S204, calculate the normalization coefficient by formula , for solving the data scale problem, and is the information entropy of a random variable, which is used to quantify the uncertainty of the variable itself; calculate the light pressure interference factor by formula , for suppressing mirror reflection interference, wherein represents the air pressure compensation, represents the optical flow tracking, which is calculated as , is the unit directional vector of the skin surface pointing to the light source, is the unit vector perpendicular to the facial skin surface; is the unit directional vector of the skin surface pointing to the light source, is the modulus of the unit vector perpendicular to the facial skin surface;

[0039] S205, calculate the blood oxygen mutation gradient : , wherein represents the sensitive parameter in the early stage of high altitude disease, which is calculated as , the second derivative of t, wherein represents the high altitude pathological response time window, and frame, i.e. 0.33s; is the blood oxygen value at time t, is the blood oxygen value at time , is the blood oxygen value at time ;

[0040] S206, based on the basic respiratory frequency obtained by the rPPG signal branch, calculate the final respiratory frequency by clinical constraint application , ; wherein represents the compensatory respiratory frequency, which is calculated as , wherein represents the high altitude compensation factor,​​​ , represents the weighted respiratory rate, which is calculated by , represents the dynamic weight, which is calculated by , represents the motion weight, which is quantified by the formula , where represents a 3-second window, is the i-th sampling point in the k-th time window.

[0041] Based on the first aspect, step S2 further comprises the following steps:

[0042] S211, based on the third blood oxygen saturation , three-axis acceleration IMU , ambient light intensity LUX and barometric altitude BARO , feature calculation is performed to obtain a second feature vector F, and the second feature vector F is input into a Bayesian network module to calculate a pathological probability ; wherein represents an energy function, which is calculated by , represents a weight distribution, represents a plateau adaptation parameter, which is calculated by ; wherein ; is the mean value of the plain baseline, is the feature standard deviation, is the j-th dimensional feature value;

[0043] S212, the probability distributions of specular reflection artifact, early pulmonary edema, device failure, and normal fluctuation are respectively defined as , ; the probability output threshold is set, if , then the polarized filter is started; if , then the oxygen therapy intervention is triggered; if , then the device self-check is performed.

[0044] S213, the triple conditions for confirming pulmonary edema are determined

[0045]

[0046] wherein, represents the Bayesian probability, whose calculation formula is , wherein is the weight matrix of the Bayesian network parameter; represents the slope, whose calculation formula is , wherein​​ is the current blood oxygen value, The blood oxygen value at the historical moment, is the current timestamp, For historical timestamps, current timestamp With historical timestamps The fixed interval is Second; Indicates and;

[0047] S214. Two conditions for determining pulmonary edema:

[0048] , THEN enable ;

[0049] Determine the equipment failure judgment conditions:

[0050]

[0051] Determine normal traffic conditions:

[0052] .

[0053] Based on the first aspect, if the diagnosis of pulmonary edema is met If the three conditions are met, the system will directly enter the medical instruction channel. If the equipment failure condition is met, the system will directly enter the safety degradation channel. If the diagnosis of pulmonary edema is not met, the system will directly enter the medical instruction channel. If the three conditions do not meet the equipment failure condition, step S3 is executed, and step S3 specifically includes the following steps:

[0054] S31. Use the physical carrier to read the genetic data, then input the EPAS1 genotype, including AA genotype, GA genotype, and GG genotype, calculate the gene threshold, and output the blood oxygen saturation warning threshold. :

[0055]

[0056] S32, then perform blood pressure compensation calculation:

[0057]

[0058] MAP represents mean arterial pressure, which is calculated as MAP = DBP + 1 / 3(SBP - DBP), SBP represents systolic blood pressure, and DBP represents diastolic blood pressure. The final threshold after compensation is calculated using the formula ;

[0059] S33, the final threshold Input into the decision matrix, perform decision fusion based on multi-dimensional parameters, and output the decision index :

[0060] ;

[0061] wherein, is a weight coefficient of a pulmonary edema risk term, is a weight coefficient of a blood oxygen deviation degree, is a blood oxygen saturation rate of change, is a weight coefficient of the rate of change;

[0062] S34, dividing the decision index into five risk levels, each level corresponding to different operations.

[0063] In a second aspect, the application discloses a non-contact blood oxygen measurement analysis system for a plateau environment, which is used for the non-contact blood oxygen measurement analysis method for the plateau environment and comprises:

[0064] The rPPG-based blood oxygen double-branch collaborative network module is used for realizing high-robustness conversion from a video to blood oxygen through a main network double-branch plus sub-network verification architecture.

[0065] The causal decision engine module receives output data of the rPPG-based blood oxygen double-branch collaborative network module, synchronously fuses three-axis motion acceleration IMU, ambient light intensity LUX and barometric altimeter BARO, constructs a blood oxygen fluctuation attribution model through a dynamic Bayesian network, and outputs probability distribution of four kinds of plateau diseases and corresponding intervention instructions.

[0066] The edge execution control module receives the probability distribution of the four kinds of plateau diseases and the corresponding intervention instructions output by the causal decision engine module, combines user EPAS1 genotyping data and real-time vital signs, and generates a personalized execution strategy through a dynamic priority arbitration mechanism.

[0067] The application has the following beneficial effects:

[0068] 1) The application proposes a double-path cross-validation plateau blood oxygen early warning mechanism, which breaks through the limitation of traditional single-mode detection. Through the fusion of five core technologies of video spatiotemporal feature extraction, motion artifact dynamic compensation, Bayesian causal reasoning, EPAS1 gene hierarchical decision and edge lightweight deployment, high robustness and high accuracy of low blood oxygen monitoring in a plateau environment are realized.

[0069] 2) The application aims to realize three technical breakthroughs: double-path collaborative network technology, which adopts a double-branch network structure, significantly improves the accuracy and stability of blood oxygen prediction; multi-dimensional causal intervention technology, which analyzes the causes of blood oxygen fluctuation by using causal reasoning, and formulates intervention measures according to the reasoning results; gene-environment adaptation technology, which dynamically adjusts the blood oxygen monitoring threshold according to individual gene characteristics and changes in the plateau environment, and realizes personalized monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 A flowchart of the plateau environment-oriented non-contact blood oxygen measurement analysis method of the embodiment of the present application;

[0071] Figure 2 A flowchart of the rppg-based double-branch collaborative blood oxygen prediction network of the embodiment of the present application;

[0072] Figure 3 A flowchart of the end-to-end subnetwork of the embodiment of the present application;

[0073] Figure 4 A structural diagram of the plateau environment-oriented non-contact blood oxygen measurement analysis system of the embodiment of the present application;

[0074] Figure 5 A flowchart of the causal decision module of the plateau environment-oriented non-contact blood oxygen measurement analysis system of the embodiment of the present application;

[0075] Figure 6 A flowchart of the edge control module of the plateau environment-oriented non-contact blood oxygen measurement analysis system of the embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions of the present application will be described in detail below with reference to the embodiments, obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0077] The present application discloses a plateau environment-oriented non-contact blood oxygen measurement analysis method and system, proposes a double-path cross-validation plateau blood oxygen early warning mechanism, breaks through the limitations of traditional single-mode detection. By fusing five core technologies of video spatio-temporal feature extraction, motion artifact dynamic compensation, Bayesian causal reasoning, EPAS1 gene hierarchical decision and edge lightweight deployment, high robustness and high precision of low blood oxygen monitoring in plateau environment are realized. The present application aims to realize the following technical breakthroughs: double-path collaborative network technology, using a double-branch network structure, significantly improving the accuracy and stability of blood oxygen prediction; multi-dimensional causal intervention technology, using causal reasoning to analyze the causes of blood oxygen fluctuation, and formulating intervention measures according to the reasoning results; gene-environment adaptation technology, dynamically adjusting the blood oxygen monitoring threshold according to individual gene characteristics and plateau environment changes, realizing personalized monitoring. Through the systematic integration of double-branch collaborative network, causal reasoning engine and gene adaptation mechanism, the deficiencies in the prior art are solved. The flowchart of the method is shown in Figure 1 , which specifically includes the following steps:

[0078] S1, based on a continuous face video stream, a double-branch main network architecture is used for parallel processing, the double-branch includes an rPPG signal branch and an end-to-end prediction branch; a flowchart of a rPPG-based double-branch cooperative blood oxygen prediction network is as shown in Figure 2

[0079] In the rPPG signal branch, the ROI region of the face is extracted through 3DCNN spatiotemporal modeling and optical flow tracking technology, and a high-quality rPPG signal is generated in combination with an adaptive skin color segmentation algorithm; then a feature extraction subnetwork is deployed in the middle of the rPPG branch, the spatiotemporal features in the rPPG signal are learned through a CNN plus LSTM network model, a nonlinear mapping relationship from the pulse waveform to the blood oxygen saturation is established, and a blood oxygen saturation prediction value is output; without explicit calculation of AC / DC and R value;

[0080] In the end-to-end prediction branch, a Transformer encoder-decoder structure is used, a nonlinear mapping relationship from the video frame sequence to the blood oxygen saturation is learned, and thus the motion pattern is captured; an end-to-end subnetwork flowchart is as shown in Figure 3 ; wherein 1DCNN is a one-dimensional convolutional neural network, used for processing time series signals (such as rPPG waveform), and local features are extracted through convolution kernels. Conv is a convolution layer, used for performing convolution operation, extracting local features of input signals, and 64 Conv in the figure indicates that 64 convolution kernels are used. RELU is a linear rectification activation function, which performs nonlinear transformation on the convolution output, and enhances the nonlinear fitting ability of the model. MaxPooling is a maximum pooling layer, which down-samples the feature map, retains significant features and reduces the amount of calculation. Transformer Block is a Transformer module, which processes time series data based on the self-attention mechanism (Self-Attention) to capture long-distance dependencies. LSTM is a long short-term memory network, which is a variant of recurrent neural network (RNN), and solves the long-time series dependency problem through a gating mechanism, and is used for modeling the time series dynamics of blood oxygen signals.

[0081] To enhance the cooperative verification ability of the double-path results, a dynamic consistency loss function is used to realize mutual supervision optimization between the double branches, and an environment adaptive weighting strategy is designed for fusion, and the final blood oxygen prediction data is output for further analysis; the anti-interference performance of blood oxygen monitoring in plateau motion is significantly improved;

[0082] ​S2, receive the final blood oxygen prediction data, then fuse the multi-source sensor data with the blood oxygen prediction data, the multi-source sensor data including three-axis motion acceleration IMU, ambient light intensity LUX, barometric altimeter BARO and the like; build a blood oxygen fluctuation attribution model through a dynamic Bayesian network, output a probability distribution of high altitude diseases and a corresponding intervention instruction; the high altitude diseases include mirror reflection artifact, early pulmonary edema, equipment failure and normal fluctuation; to improve the diagnostic specificity, the clinical knowledge graph constraint and the time mutation gradient detection mechanism are innovatively introduced, which significantly reduces the false positive rate in the high altitude activity scene;

[0083] S3, receive the probability distribution of high altitude diseases and the corresponding intervention instruction, combine the user EPAS1 gene typing data and real-time vital signs, generate a personalized execution strategy through a dynamic priority arbitration mechanism; innovatively design a double-channel execution bus (medical instruction channel + safety degradation channel) and a gene adaptive threshold algorithm to ensure operation reliability in extreme environments.

[0084] Exemplarily, the generating rPPG signal in step S1 specifically includes the following steps: S101, using an end-to-end VVT (Video-Text Transformer) framework, the VVT (Video-Text Transformer) is a multi-modal Transformer model combining video and text data, using continuous face video as input, where T1 represents the number of frames, W represents the width of the video, H represents the height of the video, and C represents the number of channels; extract the spatio-temporal features of the video through a 3D convolution layer, the 3D convolution layer including a three-dimensional convolution conv3D, a batch normalization BatchNorm, an activation function ReLU and a maximum pooling layer MaxPool; the role of the 3D convolution layer is to extract features in both spatial and temporal dimensions, thereby capturing spatio-temporal information, and output a feature map containing spatio-temporal information , wherein represents a 3D convolution kernel with a size of wherein K is a size parameter of the 3D convolution kernel, defining the range of the convolution operation in the time, height and width dimensions;

[0085] S102, input the feature map into an encoder Transformer module, and output after enhancing the time series information through a self-attention mechanism, wherein the self-attention formula in the Transformer module is: , wherein Q represents a query matrix, wherein K represents a key matrix, and V represents a value matrix, denotes the dimension of the key vector, T2 denotes the transpose; the self-attention mechanism assigns a weight to each time step by calculating the similarity (by dot product) between the query matrix Query and the key matrix , so as to capture long-distance dependencies;

[0086] S103, further compress the features output by the Transformer module through a pooling layer, and map the spatio-temporal features to a fixed-size first feature vector ; wherein the pooling operation includes global average pooling (Global Average Pooling): , N denotes the total number of features, denotes the feature value of each position;

[0087] S104, the first feature vector is mapped to a predicted rPPG signal through a fully connected layer FC . This model combining 3D convolution and Transformer can effectively extract spatio-temporal features from video signals and enhance the modeling ability of time series data through self-attention mechanism, and finally be used for rPPG signal prediction.

[0088] Exemplarily, in step S1, the end-to-end prediction branch and the rPPG signal branch share the main network architecture, based on 3D convolution, Transformer to extract spatio-temporal features and model time series, the end-to-end prediction branch predicts the first blood oxygen saturation . Through such a structure, the video signal can be effectively used to predict the blood oxygen saturation, combined with the output of the rPPG signal for the final blood oxygen prediction, thereby improving the prediction accuracy.

[0089] Exemplarily, step S1 further includes the following steps:

[0090] S111, a subnetwork is added in the rPPG signal branch to extract the second blood oxygen saturation and the basic respiratory frequency from the rPPG signal; the subnetwork is a CNN plus LSTM network model, which extracts the local features of the rPPG signal through CNN and captures the time series dependency of the rPPG signal through LSTM; so as to better understand the trend of the signal changing over time. Because the output rPPG signal is a one-dimensional signal, the present application extracts the features of the rPPG signal through a 1DCNN : wherein W denotes a convolution kernel, is a bias term;

[0091] Then use the LSTM (Long Short Term Memory Network) network to capture the time series dependency of the rPPG signal: wherein, is the hidden state of the t-th time step, is the hidden state of the t-th time step; is the hidden state of the t-th time step;

[0092] Finally, a fully connected layer is passed to output the second blood oxygen saturation ;

[0093] S112, calculate the basic respiratory frequency of the rPPG signal;

[0094] The band-pass filter is calculated by the formula , where is a band-pass filter with a passband of [0.15, 0.65] Hz (corresponding to a respiratory frequency of 9-39 times / minute), T3 is the current timestamp, is a time offset variable (integrates all historical time points);

[0095] STFT spectral analysis is performed , where N’ = 90 is a 3-second window (30 fps), is a Hanning window function, is a Fourier basis function, which is a complex exponential component in STFT, used to project a time-domain signal into the frequency domain;

[0096] The respiratory main frequency is detected and calculated: ; the unit is hertz;

[0097] Finally, the basic respiratory frequency is calculated by the formula ; the unit is times / minute.

[0098] Exemplarily, the joint loss function used in step S1 to realize mutual supervision optimization between the two branches specifically includes the following steps:

[0099] S121, for the blood oxygen saturation of the two branches, a joint loss function L is designed, which combines the losses of the two branches by weighting to obtain the joint loss function L: , to ensure the collaborative learning of the two branches. During training, the optimization goal is to make the prediction of the rPPG branch and the video-to-blood oxygen branch as close to the true blood oxygen saturation as possible; wherein represents the predicted blood oxygen saturation of the rPPG signal branch, represents the predicted blood oxygen saturation of the video-to-blood oxygen saturation branch, represents the true blood oxygen saturation, and represent hyperparameters for adjusting the loss contribution of the two branches;

[0100] MSE is a standard mean square error loss, which is calculated as wherein represents the true blood oxygen value of the i-th sample, represents the predicted blood oxygen value of the i-th sample, and N represents the number of samples;

[0101] S122, dynamically fusing the predicted blood oxygen saturation of the double branches , and finally outputting the third blood oxygen saturation ; wherein and represent dynamic weights determined by the quality of each branch signal: , ; wherein is the signal-to-noise ratio output by the rPPG branch, is the signal-to-noise ratio output by the end-to-end branch.

[0102] Exemplarily, step S2 specifically comprises the following steps:

[0103] S201, based on the final output of the third blood oxygen saturation , fusing multi-source sensor data, including three-axis acceleration , ambient light intensity , and air pressure altitude , wherein is the acceleration of the device in the horizontal left-right direction, is the acceleration of the device in the horizontal front-back direction, is the acceleration of the device in the vertical direction, is the atmospheric pressure (Atmospheric Pressure); the three-axis accelerometer needs to satisfy the sampling rate ≥ 100 Hz, the range ≥ ± 8g; the ambient light sensor range covers 0-100,000 Lux; the barometric altimeter accuracy is better than ± 1 hPa (equivalent to an altitude error < 8 meters). Bosch BMI270 accelerometer, AMS TSL2591 light sensor, and BMP388 barometer are used in this embodiment, but are not limited thereto;

[0104] S202, based on a feature engineering algorithm, quantifying the coupling relationship between physiological signals and environmental interference through multiple dimensions, to solve the three industry bottlenecks of motion artifact interference, light noise interference, and individual physiological differences in plateau scenarios: through the formula to calculate the blood oxygen variability SDNN, which is used to quantify the autonomic nervous regulation function, and SDNN is calculated by the change amount of blood oxygen saturation at adjacent time points The fluctuation amplitude reflects the dynamic regulatory ability of the sympathetic-vagal nerve on the cardiovascular system. Under hypoxic stress, the imbalance of the autonomic nervous system leads to increased blood oxygen fluctuations and an abnormal increase in the SDNN value. The SDNN (Standard Deviation of NN intervals) derived from heart rate variability (HRV) analysis is used to quantify the functional state of the autonomic nervous system. The NN interval is replaced by a blood oxygen difference sequence; where T4 represents the window time, and the window time T4=30 (1 second @ 30Hz) meets the AHA clinical standard (the American Heart Association sets a minimum of 30 seconds of physiological signal analysis). It is first used in blood oxygen fluctuation analysis (traditionally used for heart rate), with a window adaptive mechanism: T4=30*(1+0.002h) (the higher the altitude, the longer the window), where h represents the current altitude; the change in blood oxygen saturation at adjacent time points , that is, the current blood oxygen value Compared with the blood oxygen value at the previous moment The difference, ;

[0105] S203, through the formula Calculating motion-blood oxygen mutual information entropy , used to reveal the coupling relationship between motion artifacts and blood oxygen fluctuations, where Indicates the third blood oxygen saturation and vertical acceleration The joint probability distribution of is calculated based on the Shannon entropy formula. represents the probability distribution of the third blood oxygen saturation, Indicates vertical acceleration The probability distribution of vertical acceleration As a proxy variable for motion noise, the joint probability distribution is estimated by histogram (256 bins) to quantify the motion artifacts;

[0106] S204, introduce the normalization coefficient to solve the data scale problem, through the formula Calculate the normalization coefficient , and is the information entropy of the random variable, which is used to quantify the uncertainty of the variable itself; through the formula Calculate the optical pressure interference factor , used to suppress mirror reflection interference, where Indicates pressure compensation. The numerator and denominator are divided by the sea level pressure (101.3 kPa) to eliminate dimensional differences and perform plateau correction. At the same time, azimuth compensation technology is introduced to innovatively compensate for Lambert's cosine law. Represents optical flow tracking, which is calculated as , is the unit direction vector from the skin surface to the light source, is the unit vector perpendicular to the facial skin surface; is the unit direction vector modulus of the skin surface pointing to the light source, is the modulus of the unit vector perpendicular to the facial skin surface, and for the first time quantifies the atmospheric optical effect as a physiological monitoring interference factor;

[0107] S205. Calculate blood oxygen mutation gradient : ,in It represents the early sensitive parameter of altitude sickness, and its calculation method is: , find the second-order derivative at time t, where represents the time window of plateau pathological response, and frame, i.e. 0.33s; is the blood oxygen value at time t, for Blood oxygen value at all times, for The blood oxygen value at the moment; its physiological significance is to capture only the accelerated decline phase (corresponding to the pathological development stage of pulmonary edema);

[0108] S206: Based on the basic respiratory rate obtained from the rPPG signal branch, the final respiratory rate is calculated by applying clinical constraints , ;in represents the compensatory respiratory rate, which is calculated as ,in represents the plateau compensation factor, , represents the weighted respiratory rate, which is calculated as , Represents the dynamic weight, which is calculated as follows , Represents the motion weight, through the formula Weighting for exercise Quantization, where Indicates a 3-second window, is the i-th sampling point in the k-th time window, which is a time index and is used to implement sliding window calculation.

[0109] Exemplarily, step S2 further includes the following steps:

[0110] S211, based on the third blood oxygen saturation , three-axis acceleration IMU , ambient light intensity LUX and pressure altitude BAROPerform feature calculation to obtain the second eigenvector F, F = [SDNN, mutual information entropy, light pressure factor, mutation gradient]. Input the second eigenvector F into the Bayesian network module to calculate the pathological probability: ;in represents the energy function, which is calculated as , represents the weight distribution, which is obtained based on the statistics of 378 cases in the General Hospital of Tibet Military Region. represents the plateau adaptation parameter, which is calculated as follows: ;in ; is the plain benchmark mean; is the characteristic standard deviation, reflecting the degree of characteristic dispersion; is the j-th dimension eigenvalue;

[0111] S212, the illusion caused by optical interference is mirror reflection artifact, the initial stage of high altitude pulmonary edema is early pulmonary edema, sensor detachment / frost is equipment failure, and physiological blood oxygen fluctuation is normal fluctuation. The probability distributions of mirror reflection artifact, early pulmonary edema, equipment failure, and normal fluctuation are defined as and ; Set the probability output threshold, if , then start the polarization filter; if , then trigger oxygen therapy intervention; if , then the device self-checks;

[0112] S213, introduce clinical knowledge constraint mechanism formula to confirm the diagnosis of pulmonary edema Triple conditions (core decision-making mechanism):

[0113]

[0114] in, Expressed as Bayesian probability, its calculation formula is in is the weight matrix of the Bayesian network parameters; It represents the slope, and its calculation formula is , is the current blood oxygen value, The blood oxygen value at the historical moment, is the current timestamp, For historical timestamps, current timestamp With historical timestamps The fixed interval is Second; The negative sign indicates a decrease in blood oxygen, which clinically means a continuous decrease in blood oxygen in patients with pulmonary edema (average 1.2% / min).

[0115] S214, determine the double conditions of pulmonary edema:

[0116] , THEN enable ;

[0117] Determine the device failure decision condition:

[0118]

[0119] Determine the normal traffic condition:

[0120] .

[0121] Exemplarily, if the triple conditions of pulmonary edema diagnosis are met, directly enter the medical instruction channel, if the device failure condition is met, directly enter the safety degradation channel, if the triple conditions of pulmonary edema diagnosis are not met and the device failure condition is not met, execute step S3, which specifically includes the following steps: S31, read the gene data using a physical carrier, then input the EPAS1 genotype, including AA genotype, GA genotype and GG genotype, perform gene threshold calculation, and output the blood oxygen saturation warning threshold

[0122]

[0123]

[0124] S32, then perform blood pressure compensation calculation:

[0125]

[0126] Wherein MAP represents mean arterial pressure, which is calculated as MAP= DBP + 1 / 3(SBP - DBP), SBP represents systolic blood pressure, and DBP represents diastolic blood pressure; the final threshold after compensation is calculated by the formula ;

[0127] S33, input the final threshold to the decision matrix, perform decision fusion based on multi-dimensional parameters, and output the decision index :

[0128] ;

[0129] Wherein, is the weight coefficient of the pulmonary edema risk item, has the characteristics of dynamic adjustment, when the altitude is >3000 meters, =1.2 (the weight of pulmonary edema risk in plateau environment increases), when the altitude is ≤3000 meters, ​​​= 0.8, is a blood oxygen deviation degree, is a weight coefficient of the blood oxygen deviation degree, is a blood oxygen change rate term, is a blood oxygen saturation change rate, is a weight coefficient of the change rate; also has a dynamic adjustment characteristic, when the altitude > 3000 meters, = 2.0 (the change rate weight is doubled in a plateau environment), when the altitude ≤ 3000 meters, = 1.0.

[0130] S34, divide the decision index into five risk levels, each level corresponds to different operations, the decision index is a comprehensive quantitative evaluation of potential risks when the diagnosis standard is not reached, and is used to guide intervention measures under different risk levels.

[0131] The application also discloses a non-contact blood oxygen measurement and analysis system for a plateau environment, a structure diagram of which is shown in Figure 4 The application also discloses a non-contact blood oxygen measurement and analysis system for a plateau environment, a structure diagram of which is shown in

[0132] The rPPG-based blood oxygen double-branch collaborative network module is used for realizing high-robustness conversion from video to blood oxygen through a main network double-branch plus sub-network verification architecture.

[0133] The causal decision engine module receives output data of the rPPG-based blood oxygen double-branch collaborative network module, synchronously fuses three-axis motion acceleration IMU, ambient light intensity LUX and barometric altimeter BARO, constructs a blood oxygen fluctuation attribution model through a dynamic Bayesian network, and outputs probability distribution of four plateau diseases and corresponding intervention instructions; a flow chart of the causal decision module is shown in Figure 5 .

[0134] The edge execution control module receives the probability distribution of the four plateau diseases and the corresponding intervention instructions output by the causal decision engine module, combines user EPAS1 genotyping data and real-time vital signs, and generates a personalized execution strategy through a dynamic priority arbitration mechanism; a flow chart of the edge control module is shown in Figure 6 .

[0135] The above description is only the preferred embodiment of the application, and it should be understood that the application is not limited to the form disclosed herein, and should not be considered as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein, through the above teaching or related technical or knowledge. The modification and change made by the person skilled in the art without departing from the spirit and scope of the application shall be within the protection scope of the appended claims of the application.

Claims

1. A non-contact blood oxygen measurement and analysis method for plateau environments, characterized by: The following steps are involved: S1. Based on the continuous face video stream, it is processed in parallel through a dual-branch main network architecture, wherein the dual branches include an rPPG signal branch and an end-to-end prediction branch; In the rPPG signal branch, the facial ROI area is extracted through 3DCNN spatiotemporal modeling and optical flow tracking technology, and the rPPG signal is generated by combining it with an adaptive skin color segmentation algorithm. Then, a feature extraction subnetwork is deployed in the rPPG branch. The spatiotemporal features of the rPPG signal are learned through a CNN plus LSTM network model. A nonlinear mapping relationship is established from the pulse waveform to blood oxygen saturation, and the blood oxygen saturation prediction value is output. In the end-to-end prediction branch, the Transformer encoder-decoder structure is used to capture the motion pattern by learning the nonlinear mapping relationship between the video frame sequence and the blood oxygen saturation. The end-to-end prediction branch and the rPPG signal branch share the main network architecture. Based on 3D convolution and Transformer to extract spatiotemporal features and model timing, the end-to-end prediction branch predicts the first blood oxygen saturation. ; A joint loss function is used to achieve mutual supervision optimization between the two branches, and an environment-adaptive weighted strategy is designed for fusion to output the final blood oxygen prediction data; S2. Receive the final blood oxygen prediction data and then fuse it with multi-source sensor data, including the three-axis motion acceleration IMU, ambient light intensity LUX, and barometric altimeter BARO. Construct a blood oxygen fluctuation attribution model using a dynamic Bayesian network to output the probability distribution of altitude sickness and corresponding intervention instructions. Altitude sickness includes specular artifacts, early pulmonary edema, equipment failure, and normal fluctuations. S3 receives the probability distribution of altitude sickness and corresponding intervention instructions, combines the user's EPAS1 genotyping data and real-time vital signs, and generates a personalized execution strategy through a dynamic priority arbitration mechanism.

2. The non-contact blood oxygen measurement and analysis method for plateau environment according to claim 1, characterized in that: Generating the rPPG signal in step S1 specifically includes the following steps: S101, using the end-to-end VVT (Video-Text Transformer) framework, using continuous face video As input, where T1 represents the number of frames, W represents the width of the video, H represents the height of the video, and C represents the number of channels; the spatiotemporal features of the video are extracted through the 3D convolution layer, which includes three-dimensional convolution conv3D, batch normalization BatchNorm, activation function ReLU and maximum pooling layer MaxPool; the spatial and temporal dimensions are operated at the same time, and the feature map containing spatiotemporal information is output. , ,in Represents a 3D convolution kernel, whose size is , K represents the size parameter of the 3D convolution kernel; S102, feature map The input is sent to the encoder Transformer module, and the temporal information is enhanced through the self-attention mechanism and then output. The self-attention formula in the Transformer module is: , represents the query matrix Query, represents the key matrix Key, V represents the value matrix Value, represents the dimension of the key vector, T2 represents transpose; S103: Further compress the features output by the Transformer module through the pooling layer, and map the spatiotemporal features to the first feature vector ; The pooling operation includes global average pooling: , N represents the total number of features, Represents the eigenvalue of each position; S104, the first eigenvector After a fully connected layer FC, the predicted rPPG signal is mapped out .

3. The non-contact blood oxygen measurement and analysis method for plateau environment according to claim 2, characterized in that: Step S1 further includes the following steps: S111. Add a subnetwork to the rPPG signal branch to extract the second blood oxygen saturation and basal respiratory rate from the rPPG signal; the subnetwork is a CNN plus LSTM network model, where the CNN extracts local features of the rPPG signal and the LSTM captures the temporal dependency of the rPPG signal; Extracting features of rPPG signals through a 1DCNN : , where W represents the convolution kernel, is the bias term; Then use LSTM network to capture the temporal dependencies of rPPG signals: ,in, is the hidden state at the t-th time step, It is The hidden state of time steps; Finally, a fully connected layer outputs the second blood oxygen saturation ; S112, calculate the basic respiratory rate of the rPPG signal; by formula Compute the bandpass filter, where is a bandpass filter with a passband of [0.15, 0.65] Hz, T3 is the current timestamp, is the time offset variable; Perform STFT spectrum analysis ,in N’ =90 is a 3-second time window, is the Hanning window function, is the Fourier basis function; Dominant breathing frequency Perform detection calculations: ; Finally, the formula Calculate basal respiratory rate , the unit is times / minute.

4. The non-contact blood oxygen measurement and analysis method for plateau environment according to claim 3, characterized in that: The use of the joint loss function in step S1 to achieve mutual supervision optimization between two branches specifically includes the following steps: S121. Perform a weighted combination of the losses of the two branches to obtain the joint loss function L: , ensuring the collaborative learning of the two branches, where represents the predicted blood oxygen saturation of the rPPG signal branch, represents the predicted blood oxygen saturation from the video to blood oxygen saturation branch, Indicates the actual blood oxygen saturation. and represents a hyperparameter used to adjust the loss contribution of the two branches; MSE is the standard mean square error loss, which is calculated as ,in represents the true blood oxygen value of the i-th sample, represents the predicted blood oxygen value of the i-th sample, and N represents the number of samples; S122. Dynamic fusion of predicted blood oxygen saturation of both branches , and finally output the third blood oxygen saturation ;in and Represents the dynamic weight, which is determined by the signal quality of each branch: , ;in is the signal-to-noise ratio of the rPPG branch output, is the signal-to-noise ratio of the end-to-end branch output.

5. The non-contact blood oxygen measurement and analysis method for plateau environment according to claim 4, characterized in that: Step S2 specifically includes the following steps: S201: Based on the third blood oxygen saturation finally output , fusing multi-source sensor data, including three-axis acceleration , ambient light intensity and pressure altitude ,in is the acceleration of the device in the horizontal left and right directions, is the acceleration of the device in the horizontal forward and backward direction, is the acceleration of the device in the vertical direction, is the atmospheric pressure; S202, through the formula Calculate blood oxygen variability SDNN to quantify autonomic nervous system regulation function, where T4 represents the window time, the window time T4=30, and the window time T4 complies with the adaptive mechanism T4=30*(1+0.002h), where h represents the current altitude. Indicates the change in blood oxygen saturation at adjacent time points, that is, the current blood oxygen value Compared with the blood oxygen value at the previous moment The difference, ; S203, through the formula Calculating motion-blood oxygen mutual information entropy , used to reveal the coupling relationship between motion artifacts and blood oxygen fluctuations, where Indicates the third blood oxygen saturation and vertical acceleration The joint probability distribution of represents the probability distribution of the third blood oxygen saturation, Indicates vertical acceleration The probability distribution of S204, through the formula Calculate the normalization coefficient , used to solve the data scale problem, and is the information entropy of the random variable, which is used to quantify the uncertainty of the variable itself; through the formula Calculate the optical pressure interference factor , used to suppress mirror reflection interference, where Indicates air pressure compensation, Represents optical flow tracking, which is calculated as , is the unit direction vector from the skin surface to the light source, is the unit vector perpendicular to the facial skin surface; is the unit direction vector modulus of the skin surface pointing to the light source, is the modulus of the unit vector perpendicular to the facial skin surface; S205. Calculate blood oxygen gradient : ,in It represents the early sensitive parameter of altitude sickness, and its calculation method is: , find the second-order derivative at time t, where represents the time window of plateau pathological response, and , i.e. 0.33s; is the blood oxygen value at time t, for Blood oxygen value at all times, for Blood oxygen value at all times; S206: Based on the basic respiratory rate obtained from the rPPG signal branch, the final respiratory rate is calculated by applying clinical constraints , ;in represents the compensatory respiratory rate, which is calculated as ,in represents the plateau compensation factor, , represents the weighted respiratory rate, which is calculated as , Represents the dynamic weight, which is calculated as follows , Represents the motion weight, through the formula Weighting for exercise Quantization, where Indicates a 3-second window, is the i-th sampling point in the k-th time window.

6. The non-contact blood oxygen measurement and analysis method for plateau environments according to claim 5, characterized in that: Step S2 further includes the following steps: S211, based on the third blood oxygen saturation , three-axis acceleration IMU , ambient light intensity LUX and pressure altitude BARO Perform feature calculation to obtain the second feature vector F, input the second feature vector F into the Bayesian network module to calculate the pathological probability ;in represents the energy function, which is calculated as , represents the weight distribution, represents the plateau adaptation parameter, which is calculated as follows: ;in ; is the plain benchmark mean, is the characteristic standard deviation, is the j-th dimension eigenvalue; S212. Define the probability distributions of specular artifact, early pulmonary edema, equipment failure, and normal fluctuation as and ; Set the probability output threshold, if , then start the polarization filter; if , then trigger oxygen therapy intervention; if , then the device self-checks; S213, confirm the diagnosis of pulmonary edema Triple conditions: in, Expressed as Bayesian probability, its calculation formula is ,in is the weight matrix of the Bayesian network parameters; It represents the slope, and its calculation formula is ,in is the current blood oxygen value, The blood oxygen value at the historical moment, is the current timestamp, For historical timestamps, current timestamp With historical timestamps The fixed interval is Second; Indicates and; S214. Two conditions for determining pulmonary edema: ; THEN enable ; Determine the equipment failure judgment conditions: Determine normal traffic conditions: 。 7. The non-contact blood oxygen measurement and analysis method for plateau environments according to claim 6, characterized in that: If pulmonary edema is diagnosed If the three conditions are met, the system will directly enter the medical instruction channel. If the equipment failure condition is met, the system will directly enter the safety degradation channel. If the diagnosis of pulmonary edema is not met, the system will directly enter the medical instruction channel. If the three conditions do not meet the equipment failure condition, step S3 is executed, and step S3 specifically includes the following steps: S31. Use the physical carrier to read the genetic data, then input the EPAS1 genotype, including AA genotype, GA genotype, and GG genotype, calculate the gene threshold, and output the blood oxygen saturation warning threshold. : S32, then perform blood pressure compensation calculation: MAP represents mean arterial pressure, which is calculated as MAP = DBP + 1 / 3(SBP - DBP), SBP represents systolic blood pressure, and DBP represents diastolic blood pressure. The final threshold after compensation is calculated using the formula ; S33, the final threshold Input into the decision matrix, perform decision fusion based on multi-dimensional parameters, and output the decision index : ;in, is the weight coefficient of the pulmonary edema risk term, is the weight coefficient of blood oxygen deviation, is the rate of change of blood oxygen saturation, is the weight coefficient of the change rate; S34. Divide the decision-making index into five risk levels, each level corresponding to different operations.

8. A non-contact blood oxygen measurement and analysis system for plateau environments, used in the non-contact blood oxygen measurement and analysis method for plateau environments according to any one of claims 1 to 7, characterized in that: include: A dual-branch collaborative network module based on rPPG for blood oxygen measurement, which is used to achieve highly robust conversion from video to blood oxygen measurement through a dual-branch main network plus a sub-network verification architecture; The causal decision engine module receives the output data of the rPPG-based blood oxygen dual-branch collaborative network module and simultaneously integrates the three-axis motion acceleration IMU, ambient light intensity LUX, and barometric altimeter BARO. It constructs a blood oxygen fluctuation attribution model through a dynamic Bayesian network and outputs the probability distribution of four high-altitude illnesses and corresponding intervention instructions. The edge execution control module receives the probability distribution of four high-altitude illnesses and corresponding intervention instructions output by the causal decision engine module; combines the user's EPAS1 genotyping data and real-time vital signs, and generates a personalized execution strategy through a dynamic priority arbitration mechanism.

Images