Lightweight non-contact multi-parameter physiological detection method and system

Abstract

The present invention provides a lightweight non-contact multi-parameter physiological detection method, system, storage medium and electronic device, which relate to the field of non-contact multifunctional physiological detection. In the present invention, full use is made of the dual-spectrum information of thermal infrared and visible light, and the respiratory wave signal is extracted from the thermal infrared spectrum by adopting the technology of affine transformation and dual-light fusion, and the iPPG signal is extracted from the dual-light video based on the time-frequency domain feature analysis method. At the same time, by designing a lightweight signal denoising network, the respiratory signal and the iPPG signal are automatically denoised in a multi-task learning manner, and the correlation between different physiological parameter estimation tasks is mined, so that in scenarios with limited computing resources, accurate estimation of the three physiological parameters of respiratory rate, heart rate, and heart rate variability can be achieved, so as to analyze the individual's physiological health status in real time and provide objective and accurate physiological perception information for mental health screening.

Description

Technical Field

[0001] The present invention relates to the field of non-contact multifunctional physiological detection, and in particular to a lightweight non-contact multi-parameter physiological detection method, system, storage medium and electronic equipment. Background Art

[0002] The human body's physiological parameters such as respiratory rate, heart rate, and heart rate variability are the most basic and important vital indicators of the human body. Monitoring of these physiological parameters can timely grasp changes in the body's condition.

[0003] With the development of sensor technology, information collection and processing technology, and Internet technology, non-contact multifunctional physiological detection technology has emerged. In related technologies, non-contact multifunctional physiological detection methods are generally based on deep learning networks such as "2DCNN+LSTM" or 3DCNN, using an end-to-end multi-task framework to analyze the image sequence of the facial area of ​​interest, reducing the negative impact of noise such as ambient lighting and motion artifacts on physiological signals.

[0004] However, the above solution requires a lot of computing resources and cannot meet the actual application needs in resource-constrained scenarios such as mobile terminals. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a lightweight non-contact multi-parameter physiological detection method, system, storage medium and electronic device, which solves the technical problem that a large amount of computing resources are required and cannot meet the actual application needs in resource-constrained scenarios such as mobile terminals.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A lightweight non-contact multi-parameter physiological detection method is based on a preset non-contact multi-parameter physiological detection framework; the method comprises:

[0010] Obtain real-time thermal infrared video and visible light video of the subject;

[0011] Reconstructing the thermal infrared video and the visible light video respectively, linearly mapping the two reconstructed video frame sequences to obtain an initial iPPG signal, and using the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal;

[0012] Based on the positioning of the key facial feature points of the visible light video, an affine transformation method is used to locate the ROI area of ​​the first frame facial image of the thermal infrared video, a target tracking algorithm of a Siamese network is used to track the facial nose area of ​​the thermal infrared video, and an initial respiratory wave signal is obtained based on the change in the average pixel value of the tracked facial nose area;

[0013] The iPPG signal and the respiratory wave signal are fused, and the fusion result is used as the input of a lightweight signal denoising network. Based on the one-dimensional convolutional layer and the recurrent neural network layer of the signal denoising network, the respiratory rate, heart rate, and heart rate variability of the subject are predicted respectively.

[0014] Preferably, the method of using the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal comprises:

[0015] The initial iPPG signal is subjected to two alternating fast Fourier transform, convolution operation, inverse fast Fourier transform, residual connection, and convolution operation operations, and then subjected to one more convolution operation and flattened to obtain a preprocessed iPPG signal.

[0016] Preferably, the facial key feature point positioning based on the visible light video adopts an affine transformation method to locate the ROI area of ​​the first frame facial image of the thermal infrared video; comprising:

[0017] Using a face detection algorithm based on the Mediapipe framework to locate multiple key facial feature points in the visible light video;

[0018] Affine transformation solves the transformation matrix from visible light to thermal infrared;

[0019] Based on the transformation matrix and the key facial feature points of the first frame facial image of the visible light video, the key facial feature points of the first frame facial image of the thermal infrared video are located to locate the ROI area on the first frame facial image of the thermal infrared video.

[0020] Preferably, the Siamese network is composed of two CNN networks with the same structure and shared parameters, each CNN network includes five convolutional layers Conv-1 to Conv-5, four ReLU activation layers and two Maxpooling layers, and a ReLU layer follows the first four convolutional layers Conv-1 to Conv-4; the target tracking algorithm based on the Siamese network tracks the nose area of ​​the face in the thermal infrared video; including:

[0021] Starting from the second frame facial image of the thermal infrared video, taking the current frame facial image as input of the first CNN network, and obtaining first feature vectors corresponding to different candidate target areas of the current frame facial image;

[0022] Selecting the nose area of ​​the face image of the previous frame as the target ROI area, using the target ROI area as the input of the second CNN network, and obtaining a second feature vector;

[0023] The similarity between each of the first feature vectors and the second feature vector is evaluated by distance metric to generate a similarity score matrix, and the nose area of ​​the face image of the current frame is determined based on the position of the maximum value in the similarity score matrix.

[0024] Preferably, the method of obtaining the initial respiratory wave signal based on the change in the average pixel value of the tracked facial nose area comprises:

[0025] Calculate the average pixel value of the nose area of ​​the current frame face image:

[0026]

[0027] Among them, t represents the frame index, pixel ij is the coordinate (x i ,y j ), (x i ,y j ) is the plane coordinate of the pixel located at the i-th row and j-th column;

[0028] After traversing and calculating the average pixel value of the nose area of ​​each frame of the facial image, the initial breathing wave signal is formed:

[0029]

[0030] Where m is the total number of frames of the thermal infrared video.

[0031] Preferably, the signal denoising network comprises three 1DCNN layers, three MaxPooling layers, three Dropout layers, two GRU layers and two Dense layers; the iPPG signal and the respiratory wave signal are fused, and the fusion result is used as the input of the lightweight signal denoising network, and the respiratory rate, heart rate and heart rate variability of the subject are predicted based on the one-dimensional convolutional layer and the recurrent neural network layer of the signal denoising network; including:

[0032] The fusion results are sequentially used as inputs of three 1DCNN layers to extract corresponding local features; wherein each of the 1DCNN layers is connected to a MaxPooling layer and a Dropout layer;

[0033] The processed signals are used as the input of two GRU layers in turn, and the long-term dependencies in the signals are captured based on the gating mechanism.

[0034] The signals processed by the GRU layer are used as the inputs of the two Dense layers in turn, and after linear combination, the predicted results of the breathing rate, heart rate, and heart rate variability of the test subject are output respectively.

[0035] Preferably, the non-contact multi-parameter physiological detection framework selects the following loss function during the training phase:

[0036]

[0037] Among them, Loss represents the total loss function; Loss rr 、Loss hr 、Loss hrv , Loss Peaks Respectively represent the calculation of respiratory rate, heart rate, heart rate variability, and iPPG signal containing heart rate information iPPG HR , iPPG signal containing breathing information iPPG RR , the loss function of the weighted sum of the peak position and interval of the iPPG signal; α, β, γ, η, δ, λ represent the weight values ​​of the corresponding loss function respectively;

[0038] rr pre ,rr tru Represent the predicted value and true value of respiratory rate respectively; hr pre hr tru Represents the predicted value and true value of heart rate respectively; hrv pre hrv tru Represent the predicted and true values ​​of heart rate variability respectively; x and y represent the predicted and true iPPG respectively HR signal, x' and y' represent the predicted and true iPPG, respectively RR Signal;

[0039] T represents the signal length; w Pos 、w In Respectively represent the weights of the peak position and peak interval of the iPPG signal; B represents iPPG HR The batch size of the signal; L Pos,i and L In,i They represent the i-th peak position loss and the i-th adjacent peak interval loss respectively.

[0040] A lightweight non-contact multi-parameter physiological detection system is based on a preset non-contact multi-parameter physiological detection framework; the method comprises:

[0041] An acquisition module is used to acquire real-time thermal infrared video and visible light video of the subject;

[0042] A first processing module is used to reconstruct the thermal infrared video and the visible light video respectively, linearly map the two reconstructed video frame sequences to obtain an initial iPPG signal, and use the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal;

[0043] The second processing module is used to locate the ROI area of ​​the first frame of the facial image of the thermal infrared video based on the facial key feature points of the visible light video, using an affine transformation method, track the facial nose area of ​​the thermal infrared video using a target tracking algorithm of a Siamese network, and obtain an initial respiratory wave signal based on the change in the average pixel value of the tracked facial nose area;

[0044] The prediction module is used to fuse the iPPG signal and the respiratory wave signal, use the fusion result as the input of a lightweight signal denoising network, and predict the respiratory rate, heart rate, and heart rate variability of the subject based on the one-dimensional convolutional layer and recurrent neural network layer of the signal denoising network.

[0045] A storage medium stores a computer program for lightweight non-contact multi-parameter physiological detection, wherein the computer program enables a computer to execute the lightweight non-contact multi-parameter physiological detection method as described above.

[0046] An electronic device, comprising:

[0047] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including a method for executing the lightweight non-contact multi-parameter physiological detection method as described above.

[0048] (III) Beneficial effects

[0049] The present invention provides a lightweight non-contact multi-parameter physiological detection method, system, storage medium and electronic device. Compared with the prior art, it has the following beneficial effects:

[0050] In the present invention, the dual-spectrum information of thermal infrared and visible light is fully utilized, and the respiratory wave signal is extracted from the thermal infrared spectrum by adopting the technology of affine transformation and dual-light fusion, and the iPPG signal is extracted from the dual-light video based on the time-frequency domain feature analysis method. At the same time, by designing a lightweight signal denoising network, the respiratory signal and iPPG signal are automatically denoised in a multi-task learning manner, and the correlation between different physiological parameter estimation tasks is mined, so that in scenarios with limited computing resources, the three physiological parameters of respiratory rate, heart rate, and heart rate variability can be accurately estimated, so as to analyze the individual's physiological health status in real time and provide objective and accurate physiological perception information for mental health screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A block diagram of a lightweight non-contact multi-parameter physiological detection method provided by an embodiment of the present invention;

[0053] Figure 2 A flowchart of a lightweight non-contact multi-parameter physiological detection method provided by an embodiment of the present invention;

[0054] Figure 3 A flowchart of iPPG signal extraction provided by an embodiment of the present invention;

[0055] Figure 4 A flow chart of a respiratory wave signal extraction provided by an embodiment of the present invention (the dual-light feature point positioning and registration stage is not shown);

[0056] Figure 5 A flowchart of a multi-task based physiological parameter estimation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The embodiments of the present application solve the technical problem that a large amount of computing resources are required and cannot meet the actual application needs in resource-constrained scenarios such as mobile terminals by providing a lightweight non-contact multi-parameter physiological detection method, system, storage medium and electronic device.

[0059] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0060] The embodiment of the present invention proposes a lightweight non-contact multi-parameter physiological detection method based on dual-spectrum video. First, in order to accurately locate the facial region of interest of the dual-light video, a dual-light feature point positioning and registration method is designed to accurately track the thermal infrared facial region of interest, so as to improve the extraction quality of physiological signals. Secondly, in order to solve the problems of weak anti-interference and weak correlation between contextual features in physiological signals, the multi-task non-contact physiological parameter estimation is used to capture the local and global information in the fused physiological signals, as well as the key contextual features such as period and amplitude, to achieve automatic signal denoising, and improve the estimation accuracy of non-contact physiological parameters such as respiration, heart rate, and heart rate variability. Finally, based on the lightweight iPPG signal extraction module and denoising, not only the computing resources for physiological signal extraction are reduced, but also the computing resources for non-contact physiological parameter estimation such as respiration, heart rate, and heart rate variability are reduced, so as to better serve physiological health scenarios with limited computing resources such as mobile terminals.

[0061] Specifically, the key points include the following:

[0062] (1) Based on the facial feature point registration method of dual-light video, a facial region of interest tracking method based on dual-light registration is designed to accurately locate the facial region of interest in the dual-light video to improve the extraction quality of iPPG signals and respiratory signals. On this basis, an iPPG signal extraction method based on time-frequency domain information analysis is proposed. It integrates the facial region of interest image sequence of dual-light video and designs an iPPG signal analysis method that considers both frequency domain information and time domain information. This realizes the extraction of iPPG signals containing physiological information such as respiration and heart rate.

[0063] (2) Automatic noise reduction method based on multi-task. Traditional physiological signal processing methods usually include acquiring physiological signals from the extracted region of interest, followed by signal preprocessing operations such as detrending, normalization, and fast Fourier transform. However, traditional signal noise reduction processing operations can easily lead to a decrease in the accuracy of physiological parameter estimation within an abnormal range. In comparison, the embodiment of the present invention designs an automatic noise reduction network based on "1DCNN+GRU". By fusing iPPG signals and respiratory signals, an end-to-end multi-task learning method is used to explore the correlation between different physiological parameter estimation subtasks, thereby achieving accurate estimation of physiological parameters such as respiratory rate, heart rate, and heart rate variability, so as to overcome the shortcomings of traditional signal preprocessing methods.

[0064] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0065] Embodiment 1:

[0066] like Figure 1 As shown, an embodiment of the present invention provides a lightweight non-contact multi-parameter physiological detection method, characterized in that it is based on a preset non-contact multi-parameter physiological detection framework; the method includes:

[0067] S1. Obtain real-time thermal infrared video and visible light video of the subject;

[0068] S2, reconstructing the thermal infrared video and the visible light video respectively, linearly mapping the two reconstructed video frame sequences to obtain an initial iPPG signal, and using the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal;

[0069] S3, based on the positioning of the key facial feature points of the visible light video, using an affine transformation method to locate the ROI area of ​​the first frame facial image of the thermal infrared video, using a target tracking algorithm of a Siamese network to track the facial nose area of ​​the thermal infrared video, and based on the change in the average pixel value of the tracked facial nose area, obtaining an initial respiratory wave signal;

[0070] S4. Fuse the iPPG signal and the respiratory wave signal, use the fusion result as the input of a lightweight signal denoising network, and predict the respiratory rate, heart rate, and heart rate variability of the subject based on the one-dimensional convolutional layer and recurrent neural network layer of the signal denoising network.

[0071] In the embodiment of the present invention, the dual-spectrum information of thermal infrared and visible light is fully utilized, and the respiratory wave signal is extracted from the thermal infrared spectrum by adopting the technology of affine transformation and dual-light fusion, and the iPPG signal is extracted from the dual-light video based on the time-frequency domain feature analysis method. At the same time, by designing a lightweight signal denoising network, the respiratory signal and iPPG signal are automatically denoised in a multi-task learning manner, and the correlation between different physiological parameter estimation tasks is mined, so that in scenarios with limited computing resources, the three physiological parameters of respiratory rate, heart rate, and heart rate variability can be accurately estimated, so as to analyze the individual's physiological health status in real time and provide objective and accurate physiological perception information for mental health screening.

[0072] like Figure 2 As shown, Figure 2 A flowchart of a lightweight, non-contact, multi-parameter physiological detection method is given. Figure 2 The following are the steps of the above solution:

[0073] In step S1, real-time thermal infrared video and visible light video of the subject are obtained.

[0074] To ensure that the face video of the subject can be captured, the visible light and thermal infrared cameras aimed at the face area are kept 0.5m to 0.8m away from the subject in this step. After clicking the start acquisition button, the acquisition device records the real-time dual-light video of the subject.

[0075] In step S2, the thermal infrared video and the visible light video are reconstructed respectively, and the two reconstructed video frame sequences are linearly mapped to obtain an initial iPPG signal, and the initial iPPG signal is used as the input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal.

[0076] It should be noted that based on the Lambert-Beer law and the principle of thermal imaging, image photoplethysmography (iPPG) extraction can be achieved by analyzing the image sequence of the facial region of interest in the dual-light video. The core of iPPG signal extraction is to extract the color changes caused by the mean value of the pixels in the facial region of interest caused by the heartbeat by analyzing the image sequence of the region of interest (ROI) in the dual-light video.

[0077] First of all, it should be made clear that the acquired images usually include three color channels: red, green, and blue. Secondly, in the iPPG signal extraction based on visible light video, the focus is usually on the green channel. This is because hemoglobin absorbs more green light, so it is easier to capture the color changes caused by the heartbeat. In addition, in the iPPG signal extraction based on thermal infrared video, it is necessary to analyze physiological information such as breathing and heart rate from the image sequence of the facial area of ​​interest based on the slight changes in facial temperature.

[0078] Therefore, in this step, the facial ROI area of ​​the dual-light video is selected to achieve image sequence segmentation of the region of interest to extract the iPPG signal.

[0079] Specifically, the main steps of iPPG signal extraction include video frame sequence reconstruction, linear mapping, and iPPG signal preprocessing, such as Figure 3 The detailed steps are as follows:

[0080] (1) Video frame sequence reconstruction

[0081] Since the collected visible light and thermal infrared videos are presented in RGB three-channel image sequences, in order to facilitate the subsequent acquisition of the original signal sequence, it is necessary to collect the visible light video frames and thermal infrared video frames Reconstruction processing, Represents the set of real numbers.

[0082] The reconstruction process involves converting each frame of the visible and thermal infrared image into a sequence A sequence set consisting of N groups of visible light video frame signals and thermal infrared video frame signals is formed; wherein N represents the total number of image frames, C represents the number of image channels, and W and H represent the width and height of the image, respectively.

[0083] The above transformation helps to better capture the timing information in the video and facilitates further signal processing and analysis. By reconstructing the visible light and thermal infrared video frames into a sequence form, deep learning methods can be more effectively used to obtain richer and more accurate physiological signal information.

[0084] (2) Linear mapping of video frame sequence

[0085] In the iPPG signal extraction process, the iPPG signals related to heart rate and breathing can be obtained by analyzing the facial video. The linear mapping of the video frame sequence plays an important role in this process. One of the purposes of linear mapping of the video frame sequence is to enhance the visibility of the pulse signal and improve the signal-to-noise ratio. Since the iPPG signal is usually embedded in the slight color changes of the facial skin, these slight changes can be highlighted through appropriate linear mapping, making them easier to detect and extract. This helps to reduce the interference of factors such as ambient light and motion on signal extraction.

[0086] The reconstructed video frame X v and X i After linear mapping of the video frame sequence, we get Y v and Y i , the linear mapping of the video frame sequence can be described by the following formula:

[0087]

[0088] Where m and n represent the row vectors of visible light and thermal infrared video frames, respectively, and b i and b' j are all bias terms. (R i ,G i ,B i )、(R j ,G j ,B j ) represent the reconstructed visible light and thermal infrared image sequences respectively; T represents transpose.

[0089] The color transformation of the image sequence of the region of interest of the dual-light video is realized through the convolution layer. The linear convolution layer is designed as a 1D convolutional neural network (1DCNN) with a kernel size of 3 and a step size of 3. This linear mapping outputs a multi-channel sequence. After linear mapping, the initial iPPG signal can be obtained. in, and

[0090] The above transformation helps to convert the color information of the image sequence of the region of interest in the dual-light video into time domain features, providing more useful input for subsequent iPPG signal processing. Through such linear mapping processing, the temporal changes in the iPPG signal can be better captured, providing more effective information for subsequent processing steps.

[0091] (3) iPPG signal preprocessing based on time-frequency domain information

[0092] Since the traditional iPPG signal analysis method cannot obtain comprehensive time domain and frequency domain information, resulting in poor signal extraction quality, it is necessary to fuse the time domain and frequency domain signals to more comprehensively understand the characteristics of the signal, including its time domain waveform, frequency component, phase information, etc.

[0093] The method of using the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal comprises:

[0094] like Figure 3 As shown, the initial iPPG signal is subjected to two alternating fast Fourier transform, convolution operation, inverse fast Fourier transform, residual connection, and convolution operation operations, and then subjected to one more convolution operation and flattened to obtain the preprocessed iPPG signal.

[0095] Among them, the fast Fourier transform, convolution operation, inverse fast Fourier transform, residual connection, and convolution operation operations; specifically, include:

[0096] ① Fast Fourier Transform: In the frequency domain, the signal is decomposed into components of different frequencies through the Fast Fourier Transform operation, which helps to analyze and identify specific frequency components related to physiological parameters such as heart rate, heart rate, breathing, etc. In order to obtain effective frequency domain information, the frequency domain information of the iPPG signal is obtained by performing Fast Fourier Transform on the original time domain iPPG signal:

[0097] Y Freq =Y real ⊙Y imag (3)

[0098] Among them, Y Freq Include the real part of the signal and the imaginary part information, and They represent the input signal Y i Real and imaginary functions for Fast Fourier Transform operation, ⊙ represents Concat operation.

[0099] ②1DCNN convolution operation: The 1DCNN network plays an important role in iPPG signal analysis. Its main purpose is to extract key physiological features from time series through convolution operations. It can effectively capture the time domain characteristics, heartbeat periodicity and potential noise in iPPG signals, and realize the effective extraction of physiological characteristics such as heart rate, respiration, and heart rate variability, thereby improving the quality and accuracy of the signal.

[0100] Understandably, Figure 3 Medium, 1DCNN i (Ki ,S i ) represents the i-th 1DCNN convolution operation. i represents the parameters of the i-th convolution kernel, S i The parameter of the i-th step size, i = 1, 2, ..., 5. The 1DCNN convolution operation can be described by the following formula:

[0101] Y conv-i =f 1DCNN-i (Y i ') (4)

[0102] Among them, f 1DCNN-i (·) is the i-th 1DCNN convolution operation layer, Y conv-i is the input signal Y i ’Signal features after the convolution operation.

[0103] ③ Inverse Fast Fourier Transform: In order to recover the original time domain iPPG signal from the frequency domain data for further processing and analysis, an Inverse Fast Fourier Transform (IFFT) is required. The IFFT operation can extract the time domain waveform related to physiological signals such as heart rate and respiration from the frequency domain, and help eliminate the noise or distortion that may be introduced in the FFT process, thereby improving the accuracy and reliability of the iPPG signal. The IFFT operation can be described by the following formula:

[0104] Y ITTF-i =f IFFT-i (Y conv-i ) (5)

[0105] Among them, f IFFT-i (·) is the i-th inverse fast Fourier transform operation layer, Y ITTF-i Yes conv-i The signal after inverse fast Fourier transform processing.

[0106] ④ Residual connection operation: The Fourier transform and inverse transform process can analyze and reconstruct the signal in the frequency domain. Through the convolution operation, the model can learn the high-level features of the signal in the frequency domain. The residual connection allows the model to learn the residual based on these high-level features, better capture the small and important changes in the signal, and enable the model to better learn the relationship between the frequency domain and time domain information in the iPPG signal, thereby improving the performance and generalization ability of the model.

[0107] In step S3, based on the positioning of the facial key feature points of the visible light video, an affine transformation method is used to locate the ROI area of ​​the first frame facial image of the thermal infrared video, and a target tracking algorithm of a Siamese network is used to track the facial nose area of ​​the thermal infrared video. Based on the change in the average pixel value of the tracked facial nose area, an initial respiratory wave signal is obtained.

[0108] The main steps of respiratory wave signal extraction include dual-light feature point positioning and registration, facial region of interest tracking, and respiratory wave signal extraction, such as Figure 4 As shown (the dual-light feature point positioning and registration stage is not shown), the detailed steps are as follows:

[0109] (1) Dual-light feature point positioning and registration

[0110] In order to solve the problem of accurate positioning of dual-spectrum facial feature points, it is necessary to fuse dual-spectrum videos and design a facial feature point positioning method for dual-spectrum video fusion. The specific steps are as follows:

[0111] 1) Visible light video feature point positioning: Using the face detection algorithm based on the Google Mediapipe framework, 468 key facial feature points in the RGB visible light video are positioned, capturing the key features of multiple important facial areas, including eyes, mouth, nose, etc. This feature point positioning process provides a basis for subsequent physiological signal extraction, making it possible to analyze and process physiological information in visible light videos at a more refined level.

[0112] 2) Dual-light registration based on affine transformation: Since the relative positions of the visible light and thermal infrared cameras are fixed when acquiring images, the presented images can be regarded as pictures of the same object at two different positions on the left side of the same space, and can therefore be converted through affine transformation.

[0113] It should be noted that affine transformation is usually used in geometry to perform linear transformation and translation operations on a vector space so that it is mapped to another vector space.

[0114] Therefore, the transformation matrix M from visible light to thermal infrared can be solved by affine transformation AT Then, according to the affine transformation matrix M AT And visible light facial key feature points, locate the key feature points of thermal infrared face. For the vector of the key feature points of thermal infrared face Pan And rotate the scaling matrix M RS , and then get the vector of the feature points corresponding to the visible light face The affine transformation process is described by the following formula:

[0115]

[0116] This affine transformation is equivalent to the following formula in homogeneous coordinates:

[0117]

[0118] Based on the pre-trained affine transformation matrix M AT As well as the key facial feature points of the visible light video, the key facial feature points of the thermal infrared video can be obtained through the linear change operation of the above formula (6) or formula (7), so as to locate the facial ROI area of ​​the thermal infrared video image.

[0119] Specifically, the region of interest of the visible light facial image can be represented by M facial key points {(x1, y1), (x2, y2), ..., (x N ,y N )} to determine. Based on the face detection algorithm of Google Mediapipe framework, the facial key points of the specified area can be obtained. The visible light facial area of ​​interest is composed of the upper left corner point A=(x min ,y min ) and the lower right corner point B=(x max ,y max ) to confirm.

[0120] x min =min{x1, x2, ..., x M} (8)

[0121] y min =min{y1, y2, ..., y M} (9)

[0122] x max =max{x1, x2, ..., x M} (10)

[0123] y max =max{y1, y2, ..., y M} (11)

[0124] Therefore, through the affine transformation formula, the coordinates of the region of interest corresponding to the thermal infrared image can be obtained as A'=(x min ',y min ') and B'=(x max ',y max ').

[0125] (2) Facial ROI tracking based on thermal infrared video

[0126] Accurate tracking of the facial region of interest in thermal infrared video is one of the key steps to obtain respiratory signals. In order to obtain stable and reliable respiratory wave signals from thermal infrared video, the facial ROI region coordinate position of the first frame of the thermal infrared video is obtained through the above-mentioned two-light registration method based on affine transformation, and the facial region of interest of subsequent frames is realized through the designed target tracking algorithm.

[0127] In order to accurately extract spatiotemporal features and generate high-quality breathing signals, a target tracking algorithm based on Siamese network is designed to track the facial region of interest in subsequent frames in thermal infrared videos.

[0128] like Figure 4 As shown, the Siamese network consists of two lightweight convolutional neural network (CNN) networks with the same structure and shared parameters. Each CNN network includes five convolutional layers Conv-1 to Conv-5, four ReLU activation layers and two Maxpooling layers, and a ReLU layer follows the first four convolutional layers Conv-1 to Conv-4; the relationship between the convolutional layer and the adjacent layer is established through the channel mapping characteristics between them.

[0129] In addition, the first CNN network in the Siamese network is set to extract the features of the current frame facial image Z, while the second CNN network is used to extract the features of the target ROI region (eg, the facial nose region) X of the previous frame facial image.

[0130] Accordingly, the target tracking algorithm based on the Siamese network tracks the nose area of ​​the face in the thermal infrared video; including:

[0131] Starting from the second frame facial image of the thermal infrared video, the current frame facial image is used as the input of the first CNN network to obtain the first feature vector Z′ corresponding to different candidate target areas of the current frame facial image.

[0132] The nose area of ​​the face in the previous frame of the face image is selected as the target ROI area, and the target ROI area is used as the input of the second CNN network to obtain the second feature vector X′.

[0133] The similarity between each of the first eigenvector Z′ and the second eigenvector X′ is evaluated by distance measurement to generate a similarity score matrix, and the nose area of ​​the face image of the current frame is determined based on the position of the maximum value in the similarity score matrix; wherein the formula of the distance measurement is expressed as follows:

[0134]

[0135] Among them, sim represents the similarity function; ||·|| represents the norm.

[0136] Specifically, the position of the target ROI area in the current frame facial image is determined by the coordinate information of the selected rectangular box. Describe, where the coordinates of the upper left corner of the rectangular box are The width and height of the rectangle are w i and h i express.

[0137] (3) Respiratory signal extraction

[0138] Since the temperature change in the nose area is the most obvious during human breathing, the image sequence of the nose area of ​​the thermal infrared facial video is used to extract the respiratory wave signal. During normal breathing, the gas exhaled from the mouth and nose is mainly carbon dioxide, which is produced by metabolism in the exhaled gas from the nose. The temperature of the exhaled gas is significantly higher than the temperature of the air around the nose, while the inhaled air is significantly lower than the temperature of the exhaled gas. This causes a slight change in the temperature of the air around the nose, which is manifested as a change in the average pixel value in the region of interest of the thermal infrared face.

[0139] Accordingly, the method of obtaining the initial respiratory wave signal based on the change of the average pixel value of the nose area of ​​the tracked face includes:

[0140] Calculate the average pixel value (AP) of the nose area of ​​the current frame facial image:

[0141]

[0142] Among them, t represents the frame index, pixel ij is the coordinate (x i ,y j ), (x i ,y i ) is the plane coordinate of the pixel located at the i-th row and j-th column;

[0143] After traversing and calculating the average pixel value of the nose area of ​​each frame of the facial image, the initial breathing wave signal is formed:

[0144]

[0145] Where m is the total number of frames of the thermal infrared video.

[0146] According to actual needs, this step can further process the initial respiratory signal through signal smoothing, normalization, detrending, etc. to obtain a preprocessed respiratory signal.

[0147] In step S4, the iPPG signal and the respiratory wave signal are fused, and the fusion result is used as the input of a lightweight signal denoising network. Based on the one-dimensional convolutional layer and the recurrent neural network layer of the signal denoising network, the respiratory rate, heart rate, and heart rate variability of the subject are predicted respectively.

[0148] Since the preprocessed iPPG signal and respiratory wave signal both have certain noise, they cannot be directly applied to the accurate estimation of physiological parameters such as respiratory rate, heart rate and heart rate variability. Therefore, it is necessary to design a lightweight signal denoising network by fusing iPPG signals and respiratory wave signals, and a multi-task physiological parameter estimation method to enable it to learn the correlation between different subtasks, so as to improve the estimation accuracy of physiological parameters such as respiratory rate, heart rate and heart rate variability; Figure 5 The detailed steps are as follows:

[0149] (1) Dual optical signal fusion

[0150] According to the above iPPG signal from the visible light video and the respiratory wave signal from the thermal infrared video, the fused physiological signal iPPG is obtained through fusion processing. fuse (Contains both frequency domain and time domain information). The fused signal iPPG fuse It can be described using the following formula:

[0151]

[0152] The above fused physiological signal iPPG fuse It can be used as the input of the subsequent signal denoising network.

[0153] (2) Physiological parameter estimation based on multi-task

[0154] Traditional multi-task learning models use 2D-CNN or 3D-CNN networks to analyze the sequence of facial image regions of interest and achieve non-contact physiological parameter estimation in an end-to-end learning manner, which requires a lot of computing resources. At the same time, in order to improve the anti-interference performance of physiological signal processing, the following methods are designed: Figure 5 The lightweight "1DCNN+GRU" automatic denoising network shown in the figure is designed to effectively capture key context features such as cycle and amplitude, and extract features such as breathing and heart rate in physiological information to achieve automatic denoising of physiological signals (such as iPPG signals and breathing signals).

[0155] like Figure 5 As shown, the lightweight signal denoising network includes three 1DCNN layers, three MaxPooling layers, three Dropout layers, two GRU layers and two Dense layers.

[0156] It can be understood that the above design aims to balance performance and computational efficiency and improve the practicality and adaptability of the model. Specifically, the signal denoising module based on the 1DCNN+GRU network is a deep learning model, which consists of multiple network layers, including a one-dimensional convolution layer, a MaxPooling layer, a Dropout layer, an exponential linear unit layer (Exponential linear unit, Elu), a GRU layer, and a Dense layer. By working together, these network layers can effectively denoise physiological signals and extract key features. The one-dimensional convolution layer is responsible for performing convolution operations on the input physiological signals. By performing convolution operations with a set of convolution kernels, this layer can extract local features in the signal. These local features can reflect key information such as the waveform shape and frequency of the signal, which is very important for understanding the physiological information of breathing and heart rate.

[0157] The MaxPooling layer follows, dividing the feature map extracted by the convolution layer into several blocks of the same size and selecting the maximum value in each block as the output. This approach can reduce the dimension of the feature map while retaining the most important features. The purpose of this is to reduce the complexity of the data and improve the generalization ability of the model. During the training process, the Dropout layer randomly discards a part of the neurons to prevent the model from over-relying on any one neuron. This approach can effectively avoid the occurrence of overfitting and improve the generalization ability of the model. Elu, as a commonly used activation function, is used in the exponential linear unit layer of the signal denoising module. It has a good effect in dealing with problems such as gradient disappearance and gradient explosion in deep neural networks. By introducing the exponential linear unit layer, the signal denoising module can better deal with the activation function problem in deep neural networks. The GRU layer is a recurrent neural network layer specifically used to process time series data. In the signal denoising module, the GRU layer controls the flow of information by introducing a gating mechanism, which can effectively capture the long-term dependencies in the sequence data. This approach helps the signal denoising module better understand the temporal changes in physiological signals such as breathing and heart rate. Finally, the Dense layer is a fully connected layer that is used to linearly combine the output of the GRU layer to obtain the final regression output result. By linearly combining the output of the GRU layer, the signal denoising module can better predict the changing trend of physiological information such as breathing and heart rate.

[0158] Accordingly, the iPPG signal and the respiratory wave signal are fused, and the fusion result is used as the input of a lightweight signal denoising network, and based on the one-dimensional convolutional layer and the recurrent neural network layer of the signal denoising network, the respiratory rate, heart rate, and heart rate variability of the subject are predicted respectively; including:

[0159] The fusion results are sequentially used as inputs of three 1DCNN layers to extract corresponding local features; wherein each of the 1DCNN layers is connected to a MaxPooling layer and a Dropout layer;

[0160] The processed signals are used as the input of two GRU layers in turn, and the long-term dependencies in the signals are captured based on the gating mechanism.

[0161] The signals processed by the GRU layer are used as the inputs of the two Dense layers in turn, and after linear combination, the predicted results of the breathing rate, heart rate, and heart rate variability of the test subject are output respectively.

[0162] So far, the embodiment of the present invention has completed the entire process of lightweight non-contact multi-parameter physiological detection.

[0163] In particular, in order to improve the prediction effect, during the training stage of the non-contact multi-parameter physiological detection framework, the embodiment of the present invention collects facial videos of different scenes and test subjects, and records the real pulse wave signals and respiratory wave signals of the test subjects through a pulse oximeter and a respiratory belt to obtain a diverse data set for training, so that it can learn more scenes and relevant features of the test subjects.

[0164] In addition, it should be noted that physiological parameters such as respiratory rate, heart rate, and heart rate variability are not only important indicators for evaluating the physiological health status of patients, but also can provide objective and effective diagnostic basis for mental health screening. However, there are complex relationships between these parameters, and traditional single-task learning methods are difficult to capture these relationships. In addition, traditional physiological parameter monitoring methods are often limited by computing resources, making it difficult to monitor individuals' physiological health status such as respiration, heart rate, and heart rate variability in real time. In this context, physiological parameter estimation based on lightweight multi-task learning networks has become an effective solution to this problem.

[0165] By combining deep learning technology with a multi-task learning architecture, we explore how to learn more generalized feature representations through shared information, thereby improving the accuracy of physiological parameter estimation. Specifically, by integrating physiological signal data based on dual-light video, we build a multi-task learning model based on a lightweight "1DCNN+GRU" network to achieve real-time monitoring of physiological parameters such as respiration, heart rate, and heart rate variability. In order to better learn the correlation between different tasks, the non-contact multi-parameter physiological detection framework selects the following loss function during the training phase:

[0166]

[0167] Among them, Loss represents the total loss function; Loss rr 、Loss hr、Loss hrv , Loss Peaks Respectively represent the calculation of respiratory rate, heart rate, heart rate variability, and iPPG signal containing heart rate information iPPG HR , iPPG signal containing breathing information iPPG RR , the loss function of the weighted sum of the peak position and interval of the iPPG signal; α, β, γ, η, δ, λ represent the weight values ​​of the corresponding loss function respectively;

[0168] rr pre ,rr tru Represent the predicted value and true value of respiratory rate respectively; hr pre hr tru Represents the predicted value and true value of heart rate respectively; hrv pre hrv tru Represent the predicted and true values ​​of heart rate variability respectively; x and y represent the predicted and true iPPG respectively HR signal, x' and y' represent the predicted and true iPPG, respectively RR Signal;

[0169] T represents the signal length; w Pos 、w In Respectively represent the weights of the peak position and peak interval of the iPPG signal; B represents iPPG HR The batch size of the signal; L Pos,i and L In,i They represent the i-th peak position loss and the i-th adjacent peak interval loss respectively.

[0170] Understandable, Loss rr 、Loss hr 、Loss hrv , used to control the loss between the real physiological parameters (including respiration, heart rate, heart rate variability) and the predicted values; Loss iPPGHR 、Loss iPPGRR Used to better restore iPPG in visible light videos HR Signal and iPPG RR Signal; To obtain better iPPG HR Signal, Loss Peaks It is used to improve the estimation accuracy of heart rate variability by controlling the peak position loss and the adjacent peak interval loss.

[0171] Embodiment 2:

[0172] The embodiment of the present invention provides a lightweight non-contact multi-parameter physiological detection system based on a preset non-contact multi-parameter physiological detection framework; the method includes:

[0173] An acquisition module is used to acquire real-time thermal infrared video and visible light video of the subject;

[0174] A first processing module is used to reconstruct the thermal infrared video and the visible light video respectively, linearly map the two reconstructed video frame sequences to obtain an initial iPPG signal, and use the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal;

[0175] The second processing module is used to locate the ROI area of ​​the first frame of the facial image of the thermal infrared video based on the facial key feature points of the visible light video, using an affine transformation method, track the facial nose area of ​​the thermal infrared video using a target tracking algorithm of a Siamese network, and obtain an initial respiratory wave signal based on the change in the average pixel value of the tracked facial nose area;

[0176] The prediction module is used to fuse the iPPG signal and the respiratory wave signal, use the fusion result as the input of a lightweight signal denoising network, and predict the respiratory rate, heart rate, and heart rate variability of the subject based on the one-dimensional convolutional layer and recurrent neural network layer of the signal denoising network.

[0177] Embodiment 3:

[0178] An embodiment of the present invention provides a storage medium storing a computer program for lightweight non-contact multi-parameter physiological detection, wherein the computer program enables a computer to execute the lightweight non-contact multi-parameter physiological detection method as described in Example 1.

[0179] Embodiment 4:

[0180] An embodiment of the present invention provides an electronic device, including:

[0181] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the lightweight non-contact multi-parameter physiological detection method as described in Example 1.

[0182] It can be understood that the lightweight non-contact multi-parameter physiological detection system, storage medium and electronic device provided in the embodiments of the present invention correspond to the lightweight non-contact multi-parameter physiological detection method provided in the embodiments of the present invention, and the explanations, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the lightweight non-contact multi-parameter physiological detection method, which will not be repeated here.

[0183] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0184] 1. In the embodiment of the present invention, the dual-spectrum information of thermal infrared and visible light is fully utilized, and the respiratory wave signal is extracted from the thermal infrared spectrum by adopting the technology of affine transformation and dual-light fusion, and the iPPG signal is extracted from the dual-light video based on the time-frequency domain feature analysis method. At the same time, by designing a lightweight signal denoising network, the respiratory signal and iPPG signal are automatically denoised in a multi-task learning manner, and the correlation between different physiological parameter estimation tasks is mined, so that in the scenario of limited computing resources, the three physiological parameters of respiratory rate, heart rate, and heart rate variability can be accurately estimated, so as to analyze the individual's physiological health status in real time and provide objective and accurate physiological perception information for mental health screening.

[0185] 2. The embodiment of the present invention designs a facial key feature point positioning and registration method based on dual-light video to accurately track the facial region of interest, thereby achieving accurate estimation of physiological parameters such as respiratory rate and heart rate.

[0186] 3. The embodiment of the present invention fuses dual-light video and designs a time-frequency domain feature analysis method for facial region of interest image sequence based on dual-light video, so as to effectively extract iPPG signals.

[0187] 4. The embodiment of the present invention provides a multi-physiological parameter detection method that is adaptable to a computing resource-constrained environment. The design focuses on designing an intelligent, lightweight multi-task denoising network to reduce computing resource costs while improving the adaptability of the multi-functional physiological detection method.

[0188] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0189] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight non-contact multi-parameter physiological detection method, characterized in that: Based on a preset non-contact multi-parameter physiological detection framework; the method includes: Obtain real-time thermal infrared video and visible light video of the test subject; Reconstructing the thermal infrared video and the visible light video respectively, linearly mapping the two reconstructed video frame sequences to obtain an initial iPPG signal, and using the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal; Based on the positioning of the key facial feature points of the visible light video, an affine transformation method is used to locate the ROI area of ​​the first frame facial image of the thermal infrared video, a target tracking algorithm of a Siamese network is used to track the facial nose area of ​​the thermal infrared video, and an initial respiratory wave signal is obtained based on the change in the average pixel value of the tracked facial nose area; The iPPG signal and the respiratory wave signal are fused, and the fusion result is used as the input of a lightweight signal denoising network, and based on the one-dimensional convolutional layer and the recurrent neural network layer of the signal denoising network, the respiratory rate, heart rate, and heart rate variability of the subject are predicted respectively; The signal denoising network includes three 1DCNN layers, three MaxPooling layers, three Dropout layers, two GRU layers and two Dense layers; the iPPG signal and the respiratory wave signal are fused, and the fusion result is used as the input of the lightweight signal denoising network. Based on the one-dimensional convolutional layer and the recurrent neural network layer of the signal denoising network, the respiratory rate, heart rate and heart rate variability of the subject are predicted respectively; including: The fusion results are sequentially used as inputs of three 1DCNN layers to extract corresponding local features; wherein each of the 1DCNN layers is connected to a MaxPooling layer and a Dropout layer; The processed signals are used as the input of two GRU layers in turn, and the long-term dependencies in the signals are captured based on the gating mechanism. The signals processed by the GRU layer are used as the inputs of the two Dense layers in turn, and after linear combination, the predicted results of the breathing rate, heart rate, and heart rate variability of the test subject are output respectively.

2. The lightweight non-contact multi-parameter physiological detection method according to claim 1, characterized in that: The method of using the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal comprises: The initial iPPG signal is subjected to two alternating fast Fourier transform, convolution operation, inverse fast Fourier transform, residual connection, and convolution operation operations, and then subjected to one more convolution operation and flattened to obtain a preprocessed iPPG signal.

3. The lightweight non-contact multi-parameter physiological detection method according to claim 1, characterized in that: The facial key feature point positioning based on the visible light video adopts an affine transformation method to locate the ROI area of ​​the first frame facial image of the thermal infrared video; comprising: Using a face detection algorithm based on the Mediapipe framework to locate multiple key facial feature points in the visible light video; Affine transformation solves the transformation matrix from visible light to thermal infrared; Based on the transformation matrix and the key facial feature points of the first frame facial image of the visible light video, the key facial feature points of the first frame facial image of the thermal infrared video are located to locate the ROI area on the first frame facial image of the thermal infrared video.

4. The lightweight non-contact multi-parameter physiological detection method according to claim 3, characterized in that: The Siamese network is composed of two CNN networks with the same structure and shared parameters, each CNN network includes five convolutional layers Conv-1 to Conv-5, four ReLU activation layers and two Maxpooling layers, and a ReLU layer follows the first four convolutional layers Conv-1 to Conv-4; the target tracking algorithm based on the Siamese network tracks the nose area of ​​the face in the thermal infrared video; including: Starting from the second frame facial image of the thermal infrared video, taking the current frame facial image as input of the first CNN network, and obtaining first feature vectors corresponding to different candidate target areas of the current frame facial image; Selecting the nose area of ​​the face image of the previous frame as the target ROI area, using the target ROI area as the input of the second CNN network, and obtaining a second feature vector; The similarity between each of the first feature vectors and the second feature vector is evaluated by distance metric to generate a similarity score matrix, and the nose area of ​​the face image of the current frame is determined based on the position of the maximum value in the similarity score matrix.

5. The lightweight non-contact multi-parameter physiological detection method according to claim 1, characterized in that: The method of obtaining an initial respiratory wave signal based on the change in the average pixel value of the nose area of ​​the tracked face comprises: Calculate the average pixel value of the nose area of ​​the current frame face image: Among them, t represents the frame index, pixel ij is the coordinate (x i ,y j ), (x i ,y j ) is the plane coordinate of the pixel located at the i-th row and j-th column; After traversing and calculating the average pixel value of the nose area of ​​each frame of the facial image, the initial breathing wave signal is formed: Where m is the total number of frames of the thermal infrared video.

6. The lightweight non-contact multi-parameter physiological detection method according to any one of claims 1 to 5, characterized in that: The non-contact multi-parameter physiological detection framework selects the following loss function during the training phase: Among them, Loss represents the total loss function; Loss rr 、Loss hr 、Loss hrv , Loss Peaks Respectively represent the calculation of respiratory rate, heart rate, heart rate variability, and iPPG signal containing heart rate information iPPG HR , iPPG signal containing breathing information iPPG RR , the loss function of the weighted sum of the peak position and interval of the iPPG signal; α, β, γ, η, δ, λ represent the weight values ​​of the corresponding loss function respectively; rr pre ,rr tru Represent the predicted value and true value of respiratory rate respectively; hr pre hr tru Represents the predicted value and true value of heart rate respectively; hrv pre hrv tru Represent the predicted and true values ​​of heart rate variability respectively; x and y represent the predicted and true iPPG respectively HR signal, x′ and y′ represent the predicted and true iPPG, respectively RR Signal; T represents the signal length; w Pos 、w In Respectively represent the weights of the peak position and peak interval of the iPPG signal; B represents iPPG HR The batch size of the signal; L Pos，i and L In，i They represent the i-th peak position loss and the i-th adjacent peak interval loss respectively.

7. A lightweight non-contact multi-parameter physiological detection system, characterized in that: A non-contact multi-parameter physiological detection framework based on presets; The method is used to perform the non-contact multi-parameter physiological detection method as claimed in claim 1, comprising: An acquisition module is used to acquire real-time thermal infrared video and visible light video of the subject; A first processing module is used to reconstruct the thermal infrared video and the visible light video respectively, linearly map the two reconstructed video frame sequences to obtain an initial iPPG signal, and use the initial iPPG signal as an input of an iPPG signal preprocessing network based on time-frequency domain information to obtain a preprocessed iPPG signal; The second processing module is used to locate the ROI area of ​​the first frame of the facial image of the thermal infrared video based on the facial key feature points of the visible light video, using an affine transformation method, track the facial nose area of ​​the thermal infrared video using a target tracking algorithm of a Siamese network, and obtain an initial respiratory wave signal based on the change in the average pixel value of the tracked facial nose area; The prediction module is used to fuse the iPPG signal and the respiratory wave signal, use the fusion result as the input of a lightweight signal denoising network, and predict the respiratory rate, heart rate, and heart rate variability of the subject based on the one-dimensional convolutional layer and recurrent neural network layer of the signal denoising network.

8. A storage medium, characterized in that: The computer program for lightweight non-contact multi-parameter physiological detection is stored therein, wherein the computer program enables the computer to execute the lightweight non-contact multi-parameter physiological detection method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including a program for executing the lightweight non-contact multi-parameter physiological detection method according to any one of claims 1 to 6.

Images