A non-contact multi-vital sign monitoring method and device

Abstract

The present invention relates to the field of physiological signal detection, and in particular to a non-contact multi-vital sign monitoring method and device. The method comprises: performing face detection on video data to obtain facial video data; inputting the facial video data into a trained non-contact physiological signal detection model for inference to obtain pulse wave and blood pressure data; performing signal processing on the pulse wave data to obtain heart rate and heart rate variability data; and obtaining non-contact multi-vital sign monitoring results according to the pulse wave data, blood pressure data, heart rate data and heart rate variability data. The present invention can simultaneously obtain important vital signs such as the pulse wave, heart rate, heart rate variability, and blood pressure of the monitored object through facial video. Compared with the existing methods, the present invention is based on a selective structural state space model, which can capture long-range dependencies while maintaining linear complexity, is more lightweight, and maintains a better extraction effect in multiple tasks such as heart rate estimation and blood pressure estimation.

Description

Technical Field

[0001] The present invention relates to the technical field of physiological signal detection, and in particular to a non-contact multiple vital signs monitoring method and device. Background Art

[0002] rPPG (remote Photo Plethysmo Graphy) technology can obtain the human body's pulse wave from facial videos, and further obtain important physiological signals such as heart rate, heart rate variability, blood oxygen saturation, blood pressure, etc. In the early days, rPPG was mainly based on signal processing methods. With the development of machine learning, more and more data-driven methods have emerged. Among them, the framework used by the fully supervised methods shows a trend from two-dimensional convolutional neural networks to three-dimensional convolutional neural networks and then to transformers. However, none of them can handle the two core problems of rPPG at the same time: discovering weak rPPG signals from a large number of spatiotemporally redundant video clips; understanding the periodic patterns of rPPG from a long context, which is a trade-off between complexity and long-range dependencies. It is a challenge for rPPG that requires low computation but also requires long-range dependencies to discover periodic patterns.

[0003] Blood pressure measurement can be divided into invasive measurement and non-invasive measurement. Invasive measurement is the most accurate, but it is not suitable for daily blood pressure measurement. Non-invasive measurement can be divided into cuff measurement and cuffless measurement. Generally, cuff measurement is more accurate. However, repeated cuff measurements can cause discomfort and even harm the human body. In recent years, cuffless blood pressure measurement based on PPG (Photo Plethysmo Graphy) has received great attention in recent years, but they all collect PPG signals through direct or indirect contact between the sensor and the human body. This contact method causes inconvenience in measurement to a certain extent and is not conducive to long-term monitoring. In recent years, rPPG is often regarded as a non-contact alternative to PPG, which can be measured non-contactly through a camera, bringing the possibility of long-term monitoring.

[0004] Among the physiological signals that can be obtained by rPPG, there is a relatively complete theory about the connection between heart rate, heart rate variability and pulse wave, but there is no theory that can well explain the mechanism behind the process of estimating blood pressure from pulse wave features. Estimating blood pressure is a relatively difficult task. Most of the existing rPPG-based blood pressure estimation methods are developed from PPG-based methods. The two main types of methods require estimating the pulse wave from the video first. The quality of the pulse wave estimation seriously restricts the accuracy of BP estimation. However, most of the existing blood pressure estimation methods are based on traditional signal processing methods or convolutional neural network-based methods when estimating pulse waves, ignoring long-term context modeling. The extraction accuracy and granularity of the pulse wave are not ideal, and the accuracy of blood pressure estimation is also limited. Summary of the invention

[0005] In order to solve two core problems of rPPG that are not solved by the existing technologies at the same time: discovering weak rPPG signals from a large number of spatiotemporally redundant video clips; understanding the periodic pattern of rPPG from a long context, the embodiment of the present invention provides a non-contact multi-vital sign monitoring method and device. The technical solution is as follows:

[0006] In one aspect, a non-contact multiple vital signs monitoring method is provided, the method being implemented by a non-contact multiple vital signs monitoring device, the method comprising:

[0007] S1. Obtain the video data to be monitored.

[0008] S2. Perform face detection on the video data to obtain facial video data.

[0009] S3. Input the facial video data into the trained non-contact physiological signal detection model for inference to obtain pulse wave data and blood pressure data.

[0010] S4. Perform signal processing on the pulse wave data to obtain heart rate data and heart rate variability data.

[0011] S5. Obtain non-contact multi-vital sign monitoring results based on the pulse wave data, blood pressure data, heart rate data, and heart rate variability data.

[0012] Optionally, the non-contact physiological signal detection model includes a segmentation short stem sub-model, a pulse wave network sub-model and a blood pressure network sub-model.

[0013] S3 inputs facial video data into the trained non-contact physiological signal detection model for inference to obtain pulse wave data and blood pressure data, including:

[0014] S31. Input the facial video data into the segmentation short-stem sub-model to obtain the temporal segmentation sequence extracted from the left face and the temporal segmentation sequence extracted from the right face.

[0015] S32. Input the time segmentation sequence extracted from the left face into the pulse wave network sub-model to obtain the pulse wave data extracted from the left face, and input the time segmentation sequence extracted from the right face into the pulse wave network sub-model to obtain the pulse wave data extracted from the right face.

[0016] S33. Input the pulse wave data extracted from the left face and the pulse wave data extracted from the right face into the blood pressure network sub-model to obtain blood pressure data.

[0017] Optionally, the word segmentation short stem sub-model includes a difference fusion module, a self-attention module, and a frame-level average pooling module.

[0018] In S31, the facial video data is input into the word segmentation short stem sub-model to obtain a time segmentation sequence extracted from the left face and a time segmentation sequence extracted from the right face, including:

[0019] S311, input the facial video data into the difference fusion module, obtain the displaced facial video data through time shifting, perform pairwise difference on the facial video data and the displaced facial video data in chronological order to obtain frame difference, perform primary feature extraction on the frame difference and the facial video data respectively through the first short stem submodule in the segmentation short stem submodel, fuse the extracted primary features to obtain fused primary features, perform feature extraction on the fused primary features through the second short stem submodule in the segmentation short stem submodel, and obtain output features of the difference fusion module.

[0020] S312. Input the output features of the difference fusion module into the self-attention module to obtain the output of the self-attention module.

[0021] S313. Input the output of the self-attention module into the frame-level average pooling module to obtain the temporal word sequence extracted from the left face and the temporal word sequence extracted from the right face.

[0022] Optionally, the pulse wave network sub-model and the blood pressure network sub-model include a multi-time Mamba block module and a frequency domain feedforward network module respectively.

[0023] Inputting the time segmentation sequence extracted from the left face into the pulse wave network sub-model in S32 to obtain the pulse wave data extracted from the left face includes:

[0024] S321. Input the time segmentation sequence extracted from the left face into the multi-time Mamba block module to obtain a multi-time Mamba block output.

[0025] S322, residually connect the time segmentation sequence extracted from the left face and the output of the multi-time Mamba block, and input the residually connected data into the frequency domain feedforward network module to obtain the output of the frequency domain feedforward network module.

[0026] S323, residually connect the data after residual connection and the output of the frequency domain feedforward network module to obtain the pulse wave data extracted from the left face.

[0027] Optionally, in S321, the time segmentation sequence extracted from the left face is input into the multi-time mamba block module to obtain a multi-time mamba block output, including:

[0028] S3211, project the time segmentation sequence extracted from the left face through a linear layer to obtain a projected time segmentation sequence, and send the projected time segmentation sequence to four paths respectively and split them; among them, the i-th path divides the projected time segmentation sequence into The length of A total of 15 subsequences are obtained. , Indicates the length of the input time word segmentation sequence.

[0029] S3212. The 15 subsequences are sequentially passed through a convolutional layer, an activation layer, and a selective state space model to extract hidden information, thereby obtaining 15 subsequences after the hidden information is extracted.

[0030] S3213. After the hidden information is extracted, the 15 subsequences are spliced ​​according to the time relationship during segmentation, and a total of 4 sequences of length T outputted by 4 paths are obtained, thereby obtaining a multi-time Mamba block output.

[0031] Optionally, the multi-time Mamba block output is expressed as follows:

[0032] (1)

[0033] In the formula, represents the multi-time Mamba block output, represents the output sequence of the i-th path, represents the ELU activation function, represents a linear projection, Represents the multi-temporal Mamba block module input.

[0034] Optionally, the frequency domain feedforward network module includes a first linear layer submodule, a frequency domain transformation submodule, a channel interaction submodule, an inverse transformation submodule and a second linear layer submodule.

[0035] The output of the channel interaction submodule is shown in the following equations (2)-(4):

[0036] (2)

[0037] (3)

[0038] (4)

[0039] In the formula, represents the real part of the output of the channel interaction submodule, represents the real part of the complex input of the channel interaction submodule, represents the complex weight matrix of the real part, represents the imaginary part of the complex input of the channel interaction submodule, represents the complex weight matrix of the imaginary part, represents the complex bias of the real part, represents the imaginary part of the output of the channel interaction submodule, represents the complex bias of the imaginary part, Indicates plural, Represents the output of the channel interaction submodule.

[0040] The output of the frequency domain feedforward network module is shown in the following formula (5):

[0041] (5)

[0042] In the formula, represents the frequency domain feedforward network module output, represents a linear projection, represents the inverse fast Fourier transform, Represents the output of the channel interaction submodule.

[0043] On the other hand, a non-contact multiple vital signs monitoring device is provided, which is applied to a non-contact multiple vital signs monitoring method, and the device comprises:

[0044] The acquisition module is used to acquire the video data to be monitored.

[0045] The face detection module is used to perform face detection on video data and obtain facial video data.

[0046] The input module is used to input facial video data into the trained non-contact physiological signal detection model for inference to obtain pulse wave data and blood pressure data.

[0047] The signal processing module is used to perform signal processing on the pulse wave data to obtain the heart rate data and heart rate variability data.

[0048] The output module is used to obtain non-contact multi-vital sign monitoring results based on pulse wave data, blood pressure data, heart rate data and heart rate variability data.

[0049] Optionally, the non-contact physiological signal detection model includes a segmentation short stem sub-model, a pulse wave network sub-model and a blood pressure network sub-model.

[0050] Input module, further used for:

[0051] S31. Input the facial video data into the segmentation short-stem sub-model to obtain the temporal segmentation sequence extracted from the left face and the temporal segmentation sequence extracted from the right face.

[0052] S32. Input the time segmentation sequence extracted from the left face into the pulse wave network sub-model to obtain the pulse wave data extracted from the left face, and input the time segmentation sequence extracted from the right face into the pulse wave network sub-model to obtain the pulse wave data extracted from the right face.

[0053] S33. Input the pulse wave data extracted from the left face and the pulse wave data extracted from the right face into the blood pressure network sub-model to obtain blood pressure data.

[0054] Optionally, the word segmentation short stem sub-model includes a difference fusion module, a self-attention module, and a frame-level average pooling module.

[0055] Input module, further used to:

[0056] S311, input the facial video data into the difference fusion module, obtain the displaced facial video data through time shifting, perform pairwise difference on the facial video data and the displaced facial video data in chronological order to obtain frame difference, perform primary feature extraction on the frame difference and the facial video data respectively through the first short stem submodule in the segmentation short stem submodel, fuse the extracted primary features to obtain fused primary features, perform feature extraction on the fused primary features through the second short stem submodule in the segmentation short stem submodel, and obtain output features of the difference fusion module.

[0057] S312. Input the output features of the difference fusion module into the self-attention module to obtain the output of the self-attention module.

[0058] S313. Input the output of the self-attention module into the frame-level average pooling module to obtain the temporal word sequence extracted from the left face and the temporal word sequence extracted from the right face.

[0059] Optionally, the pulse wave network sub-model and the blood pressure network sub-model include a multi-time Mamba block module and a frequency domain feedforward network module respectively.

[0060] Input module, further used for:

[0061] S321. Input the time segmentation sequence extracted from the left face into the multi-time Mamba block module to obtain a multi-time Mamba block output.

[0062] S322, residually connect the time segmentation sequence extracted from the left face and the output of the multi-time Mamba block, and input the residually connected data into the frequency domain feedforward network module to obtain the output of the frequency domain feedforward network module.

[0063] S323, residually connect the data after residual connection and the output of the frequency domain feedforward network module to obtain the pulse wave data extracted from the left face.

[0064] Optionally, the input module is further configured to:

[0065] S3211, project the time segmentation sequence extracted from the left face through a linear layer to obtain a projected time segmentation sequence, and send the projected time segmentation sequence to four paths respectively and split them; among them, the i-th path divides the projected time segmentation sequence into The length of A total of 15 subsequences are obtained. , Indicates the length of the input time word segmentation sequence.

[0066] S3212. The 15 subsequences are sequentially passed through a convolutional layer, an activation layer, and a selective state space model to extract hidden information, thereby obtaining 15 subsequences after the hidden information is extracted.

[0067] S3213. After the hidden information is extracted, the 15 subsequences are spliced ​​according to the time relationship during segmentation, and a total of 4 sequences of length T outputted by 4 paths are obtained, thereby obtaining a multi-time Mamba block output.

[0068] Optionally, the multi-time Mamba block output is expressed as follows:

[0069] (1)

[0070] In the formula, represents the multi-time Mamba block output, represents the output sequence of the i-th path, represents the ELU activation function, represents a linear projection, Represents the multi-temporal Mamba block module input.

[0071] Optionally, the frequency domain feedforward network module includes a first linear layer submodule, a frequency domain transformation submodule, a channel interaction submodule, an inverse transformation submodule and a second linear layer submodule.

[0072] The output of the channel interaction submodule is shown in the following equations (2)-(4):

[0073] (2)

[0074] (3)

[0075] (4)

[0076] In the formula, represents the real part of the output of the channel interaction submodule, represents the real part of the complex input of the channel interaction submodule, represents the complex weight matrix of the real part, represents the imaginary part of the complex input of the channel interaction submodule, represents the complex weight matrix of the imaginary part, represents the complex bias of the real part, represents the imaginary part of the output of the channel interaction submodule, represents the complex bias of the imaginary part, Indicates plural, Represents the output of the channel interaction submodule.

[0077] The output of the frequency domain feedforward network module is shown in the following formula (5):

[0078] (5)

[0079] In the formula, represents the frequency domain feedforward network module output, represents a linear projection, represents the inverse fast Fourier transform, Represents the output of the channel interaction submodule.

[0080] On the other hand, a non-contact multi-vital signs monitoring device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned non-contact multi-vital signs monitoring methods is implemented.

[0081] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned non-contact multiple vital signs monitoring methods.

[0082] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0083] In the embodiment of the present invention, an end-to-end blood pressure estimation method is first designed, with video as input and blood pressure value as output. The estimation process is specifically divided into two stages within the model: pulse wave estimation and blood pressure estimation. The long-range dependency modeling capability brought by the selective state space model is used to refine the granularity of pulse wave recovery and improve the accuracy of blood pressure estimation. At the same time, the pulse wave generated in the first stage is signal processed to obtain vital signs such as heart rate and heart rate variability, thereby realizing simultaneous monitoring of multiple vital signs. In particular, due to the use of the selective state space model, the model maintains linear complexity, has a low number of parameters and computational complexity, and is convenient for actual deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0085] Figure 1 is a flow chart of a non-contact multiple vital signs monitoring method provided by an embodiment of the present invention;

[0086] Figure 2 is a flow chart of a non-contact physiological signal detection method based on a selective state space model provided by an embodiment of the present invention;

[0087] Figure 3 is a diagram of a non-contact physiological signal detection model provided by an embodiment of the present invention;

[0088] Figure 4 It is a schematic diagram of the pulse wave network and blood pressure network structure provided by an embodiment of the present invention;

[0089] Figure 5 is a multi-time Mamba block flow chart provided by an embodiment of the present invention;

[0090] Figure 6 is a schematic diagram of a frequency domain feedforward network structure provided by an embodiment of the present invention;

[0091] Figure 7 is a block diagram of a non-contact multi-vital sign monitoring device provided by an embodiment of the present invention;

[0092] Figure 8 It is a structural schematic diagram of a non-contact multi-vital sign monitoring device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0093] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0094] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0095] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0096] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0097] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0098] The embodiment of the present invention provides a non-contact multiple vital signs monitoring method, which can be implemented by a non-contact multiple vital signs monitoring device, which can be a terminal or a server. Figure 1 The process flow chart of the non-contact multi-vital sign monitoring method shown in the figure may include the following steps:

[0099] S1. Obtain the video data to be monitored.

[0100] In a feasible implementation manner, the video data acquisition method is a prior art and is not limited herein in the present invention.

[0101] S2. Perform face detection on the video data to obtain facial video data.

[0102] In a feasible implementation manner, the flow chart of the non-contact physiological signal detection method based on the selective state space model is as follows: Figure 2 As shown, the facial video data can be obtained by using the Haar Cascade algorithm to detect facial feature key points in the first frame of each video, crop the facial area, and maintain the operation in subsequent frames.

[0103] S3. Input the facial video data into the trained non-contact physiological signal detection model for inference to obtain pulse wave data and blood pressure data.

[0104] Optionally, the non-contact physiological signal detection model includes a segmentation short stem sub-model, a pulse wave network sub-model and a blood pressure network sub-model.

[0105] like Figure 3 As shown, the above step S3 may include the following steps S31-S33:

[0106] S31. Input the facial video data into the segmentation short-stem sub-model to obtain the temporal segmentation sequence extracted from the left face and the temporal segmentation sequence extracted from the right face.

[0107] In a feasible implementation, the tokenization short stem processes the facial video input into two temporal token sequences extracted from the left and right faces respectively.

[0108] Optionally, the word segmentation short stem sub-model includes a difference fusion module, a self-attention module, and a frame-level average pooling module.

[0109] The above step S31 may include the following steps S311-S313:

[0110] S311, input the facial video data into the difference fusion module, obtain the displaced facial video data through time shifting, perform pairwise difference on the facial video data and the displaced facial video data in chronological order to obtain frame difference, perform primary feature extraction on the frame difference and the facial video data respectively through the first short stem submodule in the segmentation short stem submodel, fuse the extracted primary features to obtain fused primary features, perform feature extraction on the fused primary features through the second short stem submodule in the segmentation short stem submodel, and obtain output features of the difference fusion module.

[0111] In a feasible implementation, the difference fusion module integrates the frame difference into the original frame, giving the frame-level representation the ability to perceive pulse wave changes.

[0112] Specifically, for video input , obtained by time shift , and then make the difference between them in chronological order from the beginning to the end . Through the first short stem module Perform primary feature extraction, It includes two-dimensional convolution layer, batch normalization layer, and ReLU activation layer. and the original frame It can be expressed as:

[0113] (1)

[0114] (2)

[0115] Further, they are fused and passed through the second short stem module Further feature extraction. Including two-dimensional convolution layer, batch normalization layer, ReLU activation layer:

[0116] (3)

[0117] in, and is the fusion coefficient.

[0118] S312. Input the output features of the difference fusion module into the self-attention module to obtain the output of the self-attention module.

[0119] In a feasible implementation, the self-attention module focuses on the information-rich area of ​​the rPPG and increases its weight. The mask of the self-attention module can be expressed as:

[0120] (4)

[0121] In the formula, Indicates the height of the input video. Indicates the width of the input video. represents the ELU activation function, Represents the output of the difference fusion module.

[0122] Among them, the third short stem submodule It includes three-dimensional convolution layer, batch normalization layer, and ReLU activation layer.

[0123] S313. Input the output of the self-attention module into the frame-level average pooling module to obtain the temporal word sequence extracted from the left face and the temporal word sequence extracted from the right face.

[0124] In a feasible implementation, frame-level global average pooling extracts single-frame information into one word segmentation to form two temporal word segmentation sequences.

[0125] S32. Input the time segmentation sequence extracted from the left face into the pulse wave network sub-model to obtain the pulse wave data extracted from the left face, and input the time segmentation sequence extracted from the right face into the pulse wave network sub-model to obtain the pulse wave data extracted from the right face.

[0126] In a feasible implementation, the pulse wave network then processes the two extracted time token sequences into two pulse waves.

[0127] Optionally, the pulse wave network sub-model and the blood pressure network sub-model include a multi-time Mamba block module and a frequency domain feedforward network module respectively.

[0128] In a feasible implementation, Figure 4 As shown in the figure, the pulse wave network and the blood pressure network have the same overall structure except for the difference in input and output dimensions, including a multi-time Mamba block and a frequency domain feedforward network. The module input, the output of the multi-time Mamba block, and the output residual of the frequency domain feedforward network are connected.

[0129] Inputting the time segmentation sequence extracted from the left face into the pulse wave network sub-model in S32 to obtain the pulse wave data extracted from the left face may include the following steps S321-S323:

[0130] S321, such as Figure 5 As shown, the temporal word sequence extracted from the left face is input into the multi-temporal Mamba block module to obtain the multi-temporal Mamba block output.

[0131] Optionally, the above step S321 may include the following steps S3211-S3213:

[0132] S3211, project the time segmentation sequence extracted from the left face through a linear layer to obtain a projected time segmentation sequence, and send the projected time segmentation sequence to four paths respectively and split them; among them, the i-th path divides the projected time segmentation sequence into The length of A total of 15 subsequences are obtained. , Indicates the length of the input time word segmentation sequence.

[0133] S3212. The 15 subsequences are sequentially passed through a convolutional layer, an activation layer, and a selective state space model to extract hidden information, thereby obtaining 15 subsequences after the hidden information is extracted.

[0134] S3213. After the hidden information is extracted, the 15 subsequences are spliced ​​according to the time relationship during segmentation, and a total of 4 sequences of length T outputted by 4 paths are obtained, thereby obtaining a multi-time Mamba block output.

[0135] In one feasible implementation, for input , after being projected through the linear layer, it will be sent to four paths for processing and sliced ​​into sequences of different lengths. For the i-th path, it will be equally divided into The length of For each subsequence, the hidden information is extracted by passing through the convolution layer, activation layer, and selective state space model in sequence, where all paths of the convolution layer and the selective state space model share parameters. Subsequently, multiple subsequences of a single path are reassembled into sequences of the original length according to the time correspondence, expressed as The overall output can be expressed as:

[0136] (5)

[0137] In the formula, represents the multi-time Mamba block output, represents the output sequence of the i-th path, represents the ELU activation function, represents a linear projection, Represents the multi-temporal Mamba block module input.

[0138] Among them, the selective state space model is a linear time-invariant system that maps input to output through hidden layers.

[0139] S322, residually connect the time segmentation sequence extracted from the left face and the output of the multi-time Mamba block, and input the residually connected data into the frequency domain feedforward network module to obtain the output of the frequency domain feedforward network module.

[0140] S323, residually connect the data after residual connection and the output of the frequency domain feedforward network module to obtain the pulse wave data extracted from the left face.

[0141] In a feasible implementation, Figure 6 As shown in the figure, the frequency domain feedforward network mainly includes five stages: linear projection, frequency domain transformation, channel interaction, inverse transformation, and linear projection. Among them, the frequency domain transformation is realized by fast Fourier transform and inverse fast Fourier transform. The rPPG signal can be decomposed into its component frequencies by Fourier transform, which is convenient for identifying periodic patterns in rPPG signals. Channel interaction in the frequency domain is realized by linear transformation. Because the signal is converted into complex numbers after frequency domain transformation, the transformation is based on the complex multiplication rule. Specifically, for complex input , given the complex weight matrix and complex bias ,The output of the channel interaction phase It can be expressed as:

[0142] (6)

[0143] (7)

[0144] (8)

[0145] In the formula, the subscripts re and im represent the real part and imaginary part of the corresponding complex number, respectively. represents the real part of the output of the channel interaction submodule, represents the real part of the complex input of the channel interaction submodule, represents the complex weight matrix of the real part, represents the imaginary part of the complex input of the channel interaction submodule, represents the complex weight matrix of the imaginary part, represents the complex bias of the real part, represents the imaginary part of the output of the channel interaction submodule, represents the complex bias of the imaginary part, Indicates plural, Represents the output of the channel interaction submodule.

[0146] The output of channel interaction will be transformed back to the time domain through inverse transformation and output after projection, which can be expressed as:

[0147] (9)

[0148] In the formula, represents the frequency domain feedforward network module output, represents a linear projection, represents the inverse fast Fourier transform, Represents the output of the channel interaction submodule.

[0149] S33. Input the pulse wave data extracted from the left face and the pulse wave data extracted from the right face into the blood pressure network sub-model to obtain blood pressure data.

[0150] In one possible implementation, the blood pressure network estimates the blood pressure based on pulse waves extracted from the left and right faces respectively.

[0151] S4. Perform signal processing on the pulse wave data to obtain heart rate data and heart rate variability data.

[0152] In a feasible implementation, the processing of heart rate includes detrending, second-order Butterworth filtering, and power spectrum analysis using the Welch algorithm; the processing of heart rate variability includes detrending, second-order Butterworth filtering, calculating the time interval between consecutive heartbeats measured from the start of one normal heartbeat to the start of the next normal heartbeat, time domain analysis, and frequency domain analysis.

[0153] S5. Obtain non-contact multi-vital sign monitoring results based on the pulse wave data, blood pressure data, heart rate data, and heart rate variability data.

[0154] In-dataset and cross-dataset experiments were conducted on public datasets. The present invention achieved similar results to the most advanced methods in both heart rate estimation and blood pressure estimation tasks with fewer parameters and computational complexity. In addition, the present invention can simultaneously estimate multiple vital signs such as pulse wave, heart rate, heart rate variability, and blood pressure.

[0155] The present invention realizes a non-contact multi-vital sign monitoring method and device, which can simultaneously obtain important vital signs of the monitored object, such as pulse wave, heart rate, heart rate variability, blood pressure, etc. through facial video. Compared with previous methods, the present invention is based on a selective structural state space model, which can capture long-range dependencies while maintaining linear complexity, and is more lightweight while maintaining better extraction effects in multiple tasks such as heart rate estimation and blood pressure estimation.

[0156] In the embodiment of the present invention, an end-to-end blood pressure estimation method is first designed, with video as input and blood pressure value as output. The estimation process is specifically divided into two stages within the model: pulse wave estimation and blood pressure estimation. The long-range dependency modeling capability brought by the selective state space model is used to refine the granularity of pulse wave recovery and improve the accuracy of blood pressure estimation. At the same time, the pulse wave generated in the first stage is signal processed to obtain vital signs such as heart rate and heart rate variability, thereby realizing simultaneous monitoring of multiple vital signs. In particular, due to the use of the selective state space model, the model maintains linear complexity, has a low number of parameters and computational complexity, and is convenient for actual deployment.

[0157] Figure 7 1 is a block diagram of a non-contact multiple vital signs monitoring device according to an exemplary embodiment, wherein the device is used in a non-contact multiple vital signs monitoring method. Figure 7 The device includes an acquisition module 310, a face detection module 320, an input module 330, a signal processing module 340 and an output module 350. Among them:

[0158] The acquisition module 310 is used to acquire the video data to be monitored.

[0159] The face detection module 320 is used to perform face detection on the video data to obtain facial video data.

[0160] The input module 330 is used to input the facial video data into the trained non-contact physiological signal detection model for inference to obtain pulse wave data and blood pressure data.

[0161] The signal processing module 340 is used to perform signal processing on the pulse wave data to obtain the heart rate data and the heart rate variability data.

[0162] The output module 350 is used to obtain non-contact multi-vital sign monitoring results based on the pulse wave data, blood pressure data, heart rate data and heart rate variability data.

[0163] Optionally, the non-contact physiological signal detection model includes a segmentation short stem sub-model, a pulse wave network sub-model and a blood pressure network sub-model.

[0164] The input module 330 is further configured to:

[0165] S31. Input the facial video data into the segmentation short-stem sub-model to obtain the temporal segmentation sequence extracted from the left face and the temporal segmentation sequence extracted from the right face.

[0166] S32. Input the time segmentation sequence extracted from the left face into the pulse wave network sub-model to obtain the pulse wave data extracted from the left face, and input the time segmentation sequence extracted from the right face into the pulse wave network sub-model to obtain the pulse wave data extracted from the right face.

[0167] S33. Input the pulse wave data extracted from the left face and the pulse wave data extracted from the right face into the blood pressure network sub-model to obtain blood pressure data.

[0168] Optionally, the word segmentation short stem sub-model includes a difference fusion module, a self-attention module, and a frame-level average pooling module.

[0169] The input module 330 is further configured to:

[0170] S311, input the facial video data into the difference fusion module, obtain the displaced facial video data through time shifting, perform pairwise difference on the facial video data and the displaced facial video data in chronological order to obtain frame difference, perform primary feature extraction on the frame difference and the facial video data respectively through the first short stem submodule in the segmentation short stem submodel, fuse the extracted primary features to obtain fused primary features, perform feature extraction on the fused primary features through the second short stem submodule in the segmentation short stem submodel, and obtain output features of the difference fusion module.

[0171] S312. Input the output features of the difference fusion module into the self-attention module to obtain the output of the self-attention module.

[0172] S313. Input the output of the self-attention module into the frame-level average pooling module to obtain the temporal word sequence extracted from the left face and the temporal word sequence extracted from the right face.

[0173] Optionally, the pulse wave network sub-model and the blood pressure network sub-model include a multi-time Mamba block module and a frequency domain feedforward network module respectively.

[0174] The input module 330 is further configured to:

[0175] S321. Input the time segmentation sequence extracted from the left face into the multi-time Mamba block module to obtain a multi-time Mamba block output.

[0176] S322, residually connect the time segmentation sequence extracted from the left face and the output of the multi-time Mamba block, and input the residually connected data into the frequency domain feedforward network module to obtain the output of the frequency domain feedforward network module.

[0177] S323, residually connect the data after residual connection and the output of the frequency domain feedforward network module to obtain the pulse wave data extracted from the left face.

[0178] Optionally, the input module 330 is further configured to:

[0179] S3211, project the time segmentation sequence extracted from the left face through a linear layer to obtain a projected time segmentation sequence, and send the projected time segmentation sequence to four paths respectively and split them; among them, the i-th path divides the projected time segmentation sequence into The length of A total of 15 subsequences are obtained. , Indicates the length of the input time word segmentation sequence.

[0180] S3212. The 15 subsequences are sequentially passed through a convolutional layer, an activation layer, and a selective state space model to extract hidden information, thereby obtaining 15 subsequences after the hidden information is extracted.

[0181] S3213. After the hidden information is extracted, the 15 subsequences are spliced ​​according to the time relationship during segmentation, and a total of 4 sequences of length T outputted by 4 paths are obtained, thereby obtaining a multi-time Mamba block output.

[0182] Optionally, the multi-time Mamba block output is expressed as follows:

[0183] (1)

[0184] In the formula, represents the multi-time Mamba block output, represents the output sequence of the i-th path, represents the ELU activation function, represents a linear projection, Represents the multi-temporal Mamba block module input.

[0185] Optionally, the frequency domain feedforward network module includes a first linear layer submodule, a frequency domain transformation submodule, a channel interaction submodule, an inverse transformation submodule and a second linear layer submodule.

[0186] The output of the channel interaction submodule is shown in the following equations (2)-(4):

[0187] (2)

[0188] (3)

[0189] (4)

[0190] In the formula, represents the real part of the output of the channel interaction submodule, represents the real part of the complex input of the channel interaction submodule, represents the complex weight matrix of the real part, represents the imaginary part of the complex input of the channel interaction submodule, represents the complex weight matrix of the imaginary part, represents the complex bias of the real part, represents the imaginary part of the output of the channel interaction submodule, represents the complex bias of the imaginary part, Indicates plural, Represents the output of the channel interaction submodule.

[0191] The output of the frequency domain feedforward network module is shown in the following formula (5):

[0192] (5)

[0193] In the formula, represents the frequency domain feedforward network module output, represents a linear projection, represents the inverse fast Fourier transform, Represents the output of the channel interaction submodule.

[0194] In the embodiment of the present invention, an end-to-end blood pressure estimation method is first designed, with video as input and blood pressure value as output. The estimation process is specifically divided into two stages within the model: pulse wave estimation and blood pressure estimation. The long-range dependency modeling capability brought by the selective state space model is used to refine the granularity of pulse wave recovery and improve the accuracy of blood pressure estimation. At the same time, the pulse wave generated in the first stage is signal processed to obtain vital signs such as heart rate and heart rate variability, thereby realizing simultaneous monitoring of multiple vital signs. In particular, due to the use of the selective state space model, the model maintains linear complexity, has a low number of parameters and computational complexity, and is convenient for actual deployment.

[0195] Figure 8 is a structural diagram of a non-contact multi-vital sign monitoring device provided by an embodiment of the present invention, such as Figure 8 As shown, the non-contact multi-vital sign monitoring device may include the above Figure 7 Optionally, the non-contact multi-vital-signs monitoring device 410 may include a first processor 2001 .

[0196] Optionally, the non-contact multi-vital signs monitoring device 410 may further include a memory 2002 and a transceiver 2003 .

[0197] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0198] Combine the following Figure 8 The components of the non-contact multi-vital sign monitoring device 410 are specifically introduced:

[0199] The first processor 2001 is the control center of the non-contact multi-vital sign monitoring device 410, and may be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0200] Optionally, the first processor 2001 may perform various functions of the non-contact multi-vital sign monitoring device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0201] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 8 CPU0 and CPU1 are shown in FIG.

[0202] In a specific implementation, as an embodiment, the non-contact multi-vital sign monitoring device 410 may also include multiple processors, such as Figure 8 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0203] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0204] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Figure 8 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0205] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0206] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 8 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0207] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 8 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0208] It should be noted that Figure 8 The structure of the non-contact multi-vital sign monitoring device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0209] In addition, the technical effects of the non-contact multi-vital signs monitoring device 410 can refer to the technical effects of the non-contact multi-vital signs monitoring method described in the above method embodiment, which will not be repeated here.

[0210] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0211] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0212] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0213] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0214] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0215] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0216] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0217] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0218] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0219] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0220] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0221] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0222] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A non-contact multi-vital sign monitoring method, characterized in that: The method comprises: S1. Obtain the video data to be monitored; S2, performing face detection on the video data to obtain facial video data; S3, inputting the facial video data into a trained non-contact physiological signal detection model for inference to obtain pulse wave data and blood pressure data; S4, performing signal processing on the pulse wave data to obtain heart rate data and heart rate variability data; S5. Obtaining non-contact multi-vital sign monitoring results according to the pulse wave data, blood pressure data, heart rate data, and heart rate variability data; The non-contact physiological signal detection model includes a segmentation short stem sub-model, a pulse wave network sub-model and a blood pressure network sub-model; The step S3 of inputting the facial video data into a trained non-contact physiological signal detection model for inference to obtain pulse wave data and blood pressure data includes: S31, inputting the facial video data into the word segmentation short-stem sub-model to obtain a time segmentation sequence extracted from the left face and a time segmentation sequence extracted from the right face; S32, inputting the time segmentation sequence extracted from the left face into the pulse wave network sub-model to obtain the pulse wave data extracted from the left face, and inputting the time segmentation sequence extracted from the right face into the pulse wave network sub-model to obtain the pulse wave data extracted from the right face; S33, inputting the pulse wave data extracted from the left face and the pulse wave data extracted from the right face into the blood pressure network sub-model to obtain blood pressure data.

2. The non-contact multi-vital signs monitoring method according to claim 1, characterized in that: The word segmentation short stem sub-model includes a difference fusion module, a self-attention module and a frame-level average pooling module; The step of inputting the facial video data into the word segmentation short-stem sub-model in S31 to obtain a time segmentation sequence extracted from the left face and a time segmentation sequence extracted from the right face includes: S311, inputting the facial video data into the difference fusion module, obtaining the displaced facial video data through time shifting, performing pairwise difference on the facial video data and the displaced facial video data in chronological order to obtain frame difference, performing primary feature extraction on the frame difference and the facial video data respectively through the first short stem submodule in the segmentation short stem submodel, fusing the extracted primary features to obtain fused primary features, performing feature extraction on the fused primary features through the second short stem submodule in the segmentation short stem submodel, and obtaining output features of the difference fusion module; S312, inputting the output features of the difference fusion module into the self-attention module to obtain the output of the self-attention module; S313. Input the output of the self-attention module into the frame-level average pooling module to obtain a temporal word sequence extracted from the left face and a temporal word sequence extracted from the right face.

3. The non-contact multi-vital signs monitoring method according to claim 1, characterized in that: The pulse wave network sub-model and the blood pressure network sub-model respectively include a multi-time Mamba block module and a frequency domain feedforward network module; The step S32 of inputting the time segmentation sequence extracted from the left face into the pulse wave network sub-model to obtain the pulse wave data extracted from the left face includes: S321, inputting the time segmentation sequence extracted from the left face into the multi-time Mamba block module to obtain a multi-time Mamba block output; S322, performing residual concatenation on the time segmentation sequence extracted from the left face and the output of the multi-time Mamba block, and inputting the residual concatenated data into the frequency domain feedforward network module to obtain the frequency domain feedforward network module output; S323, performing residual connection on the residual-connected data and the output of the frequency-domain feedforward network module to obtain the pulse wave data extracted from the left face.

4. The non-contact multi-vital signs monitoring method according to claim 3, characterized in that: The step of inputting the time segmentation sequence extracted from the left face into the multi-time mamba block module in S321 to obtain a multi-time mamba block output includes: S3211, projecting the time segmentation sequence extracted from the left face through a linear layer to obtain a projected time segmentation sequence, and sending the projected time segmentation sequence to four paths respectively for segmentation; wherein the i-th path divides the projected time segmentation sequence into two equal parts i-1 The length of the part is T / 2 i-1 A total of 15 subsequences are obtained, i = 1, 2, 3, 4, T represents the length of the input time segmentation sequence; S3212, extracting hidden information from the 15 subsequences through a convolutional layer, an activation layer, and a selective state space model in sequence, to obtain 15 subsequences after the hidden information is extracted; S3213. Splice the 15 subsequences after extracting the hidden information according to the time relationship during segmentation, and obtain 4 sequences of length T output by 4 paths, and then obtain a multi-time Mamba block output.

5. The non-contact multi-vital signs monitoring method according to claim 4, characterized in that: The multi-time Mamba block output is shown in the following formula (1): Where, X Mamba represents the multi-time Mamba block output, represents the output sequence of the i-th path, σ represents the ELU activation function, Proj represents the linear projection, X in Represents the multi-temporal Mamba block module input.

6. The non-contact multi-vital signs monitoring method according to claim 3, characterized in that: The frequency domain feedforward network module includes a first linear layer submodule, a frequency domain transformation submodule, a channel interaction submodule, an inverse transformation submodule and a second linear layer submodule; The output of the channel interaction submodule is shown in the following equations (2)-(4): H′ re =H re W re -H im W im +B re (2) H′ im =H re W im +H im W re +B im (3) H′=H′ re +j·H′ im (4) In the formula, H′ re represents the real part of the output of the channel interaction submodule, H re represents the real part of the complex input of the channel interaction submodule, W re represents the complex weight matrix of the real part, H im represents the imaginary part of the complex input of the channel interaction submodule, W im represents the complex weight matrix of the imaginary part, B re represents the complex bias of the real part, H′ im represents the imaginary part of the output of the channel interaction submodule, B im represents the complex bias of the imaginary part, j represents a complex number, and H′ represents the output of the channel interaction submodule; The frequency domain feedforward network module output is shown in the following formula (5): X out =Proj(iFFT(H′)) (5) Where, X out represents the output of the frequency domain feedforward network module, Proj represents linear projection, iFFT represents inverse fast Fourier transform, and H′ represents the output of the channel interaction submodule.

7. A non-contact multi-vital sign monitoring device, the non-contact multi-vital sign monitoring device is used to implement the non-contact multi-vital sign monitoring method according to any one of claims 1 to 6, characterized in that: The device comprises: An acquisition module, used for acquiring video data to be monitored; A face detection module, used to perform face detection on the video data to obtain facial video data; An input module, used for inputting the facial video data into a trained non-contact physiological signal detection model for inference, and obtaining pulse wave data and blood pressure data; A signal processing module, used for performing signal processing on the pulse wave data to obtain heart rate data and heart rate variability data; The output module is used to obtain non-contact multi-vital sign monitoring results based on the pulse wave data, blood pressure data, heart rate data and heart rate variability data.

8. A non-contact multi-vital sign monitoring device, characterized in that: The non-contact multi-vital signs monitoring device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 6.

Images