A non-contact heart rate measurement method and system based on face video

Through the contactless heart rate measurement method based on face video, the video acquisition and signal processing algorithm is used to realize contactless heart rate detection, solving the discomfort and accuracy of the existing contact measurement methods, and is suitable for daily monitoring and remote monitoring. Medical.

CN114387479BActive Publication Date: 2024-11-22HEBEI UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210012842.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-07
Publication Date
2024-11-22
Estimated Expiration
2042-01-07

AI Technical Summary

Technical Problem

Existing contact heart rate measurement methods limit users' mobility and are not suitable for daily use, especially for newborns or patients with skin lesions. It is impossible to achieve comfortable, simple, and non-invasive heart rate monitoring, and it is difficult to meet the needs of daily monitoring and telemedicine.

Method used

The heart rate signal is extracted through video acquisition, face detection, RGB channel separation and spatial average preprocessing, fixed point algorithm and improved adaptive noise full ensemble empirical modal decomposition algorithm. , realize contactless heart rate detection.

Benefits of technology

Improves the accuracy and efficiency of heart rate detection, reduces noise interference, is suitable for long-term heart rate monitoring and disease prevention, and is suitable for daily monitoring and telemedicine applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114387479B_ABST
    Figure CN114387479B_ABST
Patent Text Reader

Abstract

The present invention provides a non-contact heart rate measurement method based on face video, which relates to the field of biomedical information processing technology, and specifically includes the following steps: video acquisition; face detection; region of interest selection; RGB channel separation and spatial averaging and preprocessing; fixed-point algorithm; improved adaptive noise complete set empirical mode decomposition; heart rate extraction. A non-contact heart rate measurement system based on face video includes the following modules: video acquisition module, face recognition and tracking module, heart rate calculation module, and result display module. The method provided by the present invention adopts a signal processing method combining independent component analysis and empirical mode decomposition algorithm, which can realize non-contact heart rate detection and improve the accuracy and efficiency of heart rate detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of biomedical information processing, and in particular to a non-contact heart rate measurement method and system based on face video. Background Art

[0002] According to statistics from the World Health Organization (WHO), more people die each year from cardiovascular diseases than from other diseases, and cardiovascular diseases have become an important factor affecting people's quality of life. Surveys and studies on the prevention and treatment of cardiovascular diseases show that most cardiovascular diseases are preventable and monitorable. Heart rate is a physiological indicator closely related to cardiovascular disease, so heart rate detection is particularly important for the monitoring and prevention of cardiovascular disease. As the population continues to age, it would be helpful for doctors to have their patients track factors such as heart rate more continuously, which would enable doctors to prescribe more effective treatments. Currently, heart rate detection methods are divided into contact and non-contact methods, depending on whether the device is in contact with the human body.

[0003] Contact heart rate measurement methods include electrocardiograms and pulse oximeters. Contact heart rate measurement methods require the placement of sensors on certain parts of the body and require contact with the subject's body surface. This direct connection measurement method limits the user's mobility and may cause discomfort and inconvenience to the user. These methods are also particularly unsuitable for newborns or patients with skin injuries. In addition, some devices can only be operated by professional medical personnel and are not suitable for daily use by patients.

[0004] Non-contact heart rate detection based on face video can remotely detect the heart rate parameters of the subject without contacting the subject. This method has the advantages of low cost, simple operation, and easy implementation. Nowadays, non-contact heart rate measurement methods are attracting more and more attention, and it will bring more convenience to mankind. In view of the above advantages, non-contact heart rate measurement is the trend of future measurement methods. In order to meet the growing needs of the people, comfortable, simple, and non-invasive human heart rate measurement is realized, which has important practical value and application prospects for daily monitoring and telemedicine.

[0005] Photoplethysmograph (PPG) is a non-invasive detection method that uses photoelectric means to detect changes in blood volume in living tissues. PPG technology requires the emission of a light beam of a certain wavelength to the surface of the skin, which is transmitted to the photoelectric receiving device by reflection or transmission. Studies have found that the absorption of light by skin, tissue, muscle, etc. remains constant throughout the blood circulation, while the blood volume in the capillaries changes periodically with the cycle of the heartbeat under the action of the heartbeat, and the blood flow per unit area in the peripheral blood vessels changes periodically with the cycle of the heartbeat. As the heart beats, the amount of blood in the blood vessels is constantly changing. When the heart pumps blood outward, the amount of blood in the blood vessels increases, and the absorption of light by the blood vessels on the surface of the skin also increases, while the absorption of light by other tissues remains basically unchanged, and the light reflected by the skin decreases accordingly; when blood flows back to the heart, the amount of blood in the blood vessels decreases, and the absorption of light by the blood vessels on the surface of the skin also decreases, while the absorption of light by other tissues remains basically unchanged, and the light reflected by the skin increases accordingly.

[0006] Image Photo Plethysmo Graphy (IPPG) is an improvement on PPG technology. It uses imaging devices such as cameras instead of specific sensors. It does not need to touch the measured part during measurement. It uses video acquisition equipment to collect videos of the measured part for a period of time, and analyzes the changes in the brightness of the measured part in the video image (that is, the changes in the intensity of reflected light). The brightness waveform obtained corresponds to the changes in arterial blood volume, which can reflect the cardiovascular activity. That is, the changes in the reflected light of the skin can reflect the cardiovascular activity. By processing the waveform of the changes in the reflected light of the skin over a period of time, physiological signals such as heart rate can be obtained.

[0007] Traditional measurement of photoplethysmography requires a specific light beam as a light source, but the present invention uses a face image to separate the reflected light from the face under a natural light source based on IPPG technology, and obtains human physiological parameters without contact. Similarly, video-based heart rate detection requires the extraction of blood volume pulse (BVP) signals. Non-contact heart rate measurement based on IPPG technology can improve user comfort, and is suitable for long-term heart rate monitoring and disease prevention. Summary of the invention

[0008] The present invention aims to overcome at least one of the above-mentioned defects in the prior art and provides a non-contact heart rate measurement method based on face video, which can realize non-contact heart rate detection and improve the accuracy and efficiency of heart rate detection.

[0009] The present invention is achieved by the following method:

[0010] A non-contact heart rate measurement method based on face video, which specifically includes the following steps:

[0011] ——Video acquisition: continuous video recording of the face to be tested;

[0012] ——Face detection: detect and identify the face data information in the recorded video, and obtain the face area of ​​the image frame by frame;

[0013] ——Region of interest selection: After face detection for each frame of video, a relatively stable area containing face information is selected as the region of interest (ROI);

[0014] ——RGB channel separation, spatial averaging and preprocessing: Separate the RGB red, green and blue three-channel data for the region of interest in each frame, and perform spatial pixel averaging processing to form RGB red, green and blue three-channel signals;

[0015] ——Fixed-Point algorithm: The three channel signals after spatial pixel averaging are used as the observation signal X, and the estimated value of the source signal is obtained from the observation signal X through the separation matrix W. The source signal is separated from the mixed observation signal X by the Fixed-Point algorithm to obtain three independent source signals s1, s2, and s3, and they are correlated with the green channel signal to obtain the noisy pulse wave signal;

[0016] ——Improved adaptive noise complete ensemble empirical mode decomposition: After decomposing and reconstructing the noisy pulse wave signal, the improved adaptive noise complete ensemble empirical mode decomposition method is used to select the IMF component that can reflect the structural characteristics of the heart rate spectrum as the estimation signal of the heart rate signal;

[0017] ——Heart rate extraction: The estimated signal of the heart rate signal is converted into a frequency domain signal. The frequency corresponding to the maximum peak of the power spectrum density is the heart rate signal. The component with a frequency within the heartbeat frequency band is the estimated pulse wave. The frequency corresponding to the maximum peak of the power spectrum density is the heartbeat frequency f h , heart rate is: HR = 60 × f h .

[0018] Optimized,video acquisition process, under indoor natural light conditions, an ordinary camera was used to capture the tester’s video at a distance of 50-60 cm from the camera.

[0019] Optimized,In the face detection process, a comprehensive Adaboost and Cascade algorithm is used for detection.

[0020] Optimized, the forehead area is selected as the ROI area when selecting the region of interest.

[0021] Furthermore, when RGB channels are separated and spatial average preprocessed, the one-dimensional source signal of the corresponding channel is obtained according to formula (1):

[0022]

[0023] Where: n is the number of image frames;

[0024] N is the total number of image frames;

[0025] X(n) is the one-dimensional source signal of the R, G or B channel;

[0026] x i,j (n) is the color intensity value of pixel (i, j) in the corresponding channel;

[0027] h and w are the height and width of the image respectively.

[0028] Furthermore, the fixed-point algorithm includes the following specific analysis processes:

[0029] Three independent source signals s(t) = [s1(t), s2(t), s3(t)] T , the observed signal is the RGB three color channel signal, then:

[0030] x(t)=As(t) (2)

[0031] Where t is the sequence of image frames, A∈R3×3 is an unknown non-singular mixing matrix;

[0032]

[0033] in is the optimal estimate of the source signal;

[0034] Centralize and whiten the data to obtain z(t);

[0035] Set an initial matrix W with unit norm;

[0036] Let w i =E{2g(W i T z)}-E{g′(W i T z)}w, where function g is a nonlinear function selected based on the Gaussianity of the data;

[0037] For the matrix W = [W1(t), W2(t), W3(t)] T Perform symmetric orthogonalization:

[0038] W=(WW T )- 1 / 2 W (4)

[0039] If it does not converge, return w i =E{2g(W i T z)}-E{g′(W i T z)}w, until the unmixing matrix is ​​finally obtained, which can be substituted into equation (3) to obtain the optimal estimate of the source signal.

[0040] Furthermore, in the improved adaptive noise complete set empirical mode decomposition, the ICEEMDAN algorithm is used for decomposition, and the decomposition process is as follows:

[0041] Definition E k (·) is the k-th order IMF component generated by EMD decomposition, k = 1, 2, 3, ..., N; M(·) is the local mean of the signal; x(t) is the signal after fixed point algorithm demixing, then:

[0042] E1(x(t))=x(t)-M(x(t)) (5)

[0043] IMF k is the mode obtained by decomposing the modal signal by ICEEMDAN, r k (t) is the kth order residual component, x i (t) is the signal after adding white noise for the ith time; β0 is the amplitude coefficient of the added noise; ω i (t) is the white noise added for the ith time; N is the number of times Gaussian white noise is added;

[0044] The ICEEMDAN algorithm reduces the amount of noise present in the modal by replacing the modal estimate with a local mean estimate and subtracting it from the original signal. The specific implementation steps of the algorithm are as follows:

[0045] Step 1: Calculate by EMD

[0046] x i (t) = x(t) + β0E1(ω i (t)), i=1, 2, 3,..., I (6)

[0047] The first-order residual component is obtained as: r1(t)= <M(x i (t))> (7)

[0048] Step 2, calculate the first (k=1) IMF component as:

[0049] IMF1(t)=x(t)-r1(t) (8)

[0050] Step 3, calculate r1(t)+β1E2(ω i (t)) The average of the local means is taken as the second residual component,

[0051] That is: r2(t)= <M(r1(t))+β1E2(ω i (t))> (9)

[0052] And the second-order IMF component is obtained as:

[0053] IMF2(t)=r1(t)-r2(t)=r1(t)- <M(r1(t))+β1E2(ω i (t))> (10)

[0054] Step 4, when k = 3, ..., N, calculate the k-th order residual component as:

[0055] r k (t)= <M(r k-1 (t))+β k-1 E k (ω i (t))> (11)

[0056] Step 5: Calculate the k-th order IMF component as:

[0057] IMF k (t) = r k-1 (t)-r k (t) (12)

[0058] Step 6, for the next k, go to step 4.

[0059] Repeat steps 4 to 6 until the residual component obtained cannot be further decomposed (i.e. the residual satisfies the IMF condition or its extreme value is less than or equal to 2). At this time, x(t) can be expressed as:

[0060]

[0061] A non-contact heart rate measurement system based on face video, which includes the following modules:

[0062] ——Video acquisition module: responsible for collecting a face video under natural light, capturing video frames and displaying them on the graphical interface;

[0063] ——Face recognition and tracking module: After the camera collects video data, it performs face recognition processing on each frame of the video to keep the face in the center of the video at all times;

[0064] ——Heart rate calculation module: including face detection, facial region of interest selection, skin pixel segmentation, extraction of RGB three-channel signals, RGB three-channel spatial averaging and preprocessing, fixed-point algorithm, ICEEMDAN decomposition, obtaining BVP signals for frequency domain analysis and calculation to obtain heart rate;

[0065] ——Result display module: Displays the measured heart rate results in a graphical interface, and plots the BVP signal processed by a combination of the Fixed-Point algorithm and the ICEEMDAN algorithm and the Fourier transformed waveform.

[0066] The optimized,face recognition and tracking module based on the face tracking device of Raspberry Pi performs face recognition processing on each frame of the video,keeping the face in the center of the video at all times.

[0067] Beneficial effects of the present invention:

[0068] The present invention provides a non-contact heart rate measurement method based on face video, which has the following advantages:

[0069] The non-contact heart rate measurement method based on face video proposed in the present invention adopts a signal processing method that combines independent component analysis and empirical mode decomposition algorithm. The fixed-point algorithm can restore the mixed RGB channel signal to an independent RGB channel signal. The empirical mode decomposition algorithm avoids the complex parameter selection process of the traditional linear filter when it is used, greatly improves the adaptability of the algorithm, reduces the workload, and improves the test efficiency. The empirical mode decomposition algorithm has a good signal denoising effect, which can make the measured heart rate more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of heart rate detection according to the present invention.

[0071] Figure 2 This is a diagram of the heart rate detection system of the present invention.

[0072] Figure 3 It is a mixed signal diagram of RGB three channels.

[0073] Figure 4 This is an independent source signal diagram after being processed by the fixed-point algorithm.

[0074] Figure 5 ICEEMDAN decomposition diagram of BVP signal.

[0075] Figure 6 This is the power spectrum density diagram of the BVP signal after decomposition.

[0076] Figure 7 It is a face tracking device based on Raspberry Pi.

[0077] In the figure: 1. Camera, 2. Vertical motion servo, 3. Horizontal motion servo. DETAILED DESCRIPTION

[0078] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0079] A non-contact heart rate measurement method based on face video, which specifically includes the following steps:

[0080] ——Video acquisition: continuous video recording of the face to be tested;

[0081] ——Face detection: Detect and identify the face data information in the recorded video, and obtain the face area of ​​the image frame by frame; after the video acquisition is completed, determine the position of the face in the image to prepare for selecting the region of interest for signal processing. In order to extract the periodic change information of the human skin color, the face detection method is the key to eliminate the face background area and locate the region of interest (ROI).

[0082] ——Region of interest selection: After face detection for each frame of video, a relatively stable area containing face information is selected as the region of interest (ROI);

[0083] The collected face videos often contain a lot of background noise, so it is necessary to extract the region of interest (ROI) from the face video. During the measurement, there are often various interferences, such as the blinking of human eyes and the movement of the mouth when speaking. The selection of ROI should avoid these interferences as much as possible, thereby improving the accuracy of the measurement results. After removing the eyes, mouth and other parts, relatively fixed areas can be selected: such as the forehead and the bridge of the nose.

[0084] ——RGB channel separation, spatial averaging and preprocessing: Separate the RGB red, green and blue three-channel data for the region of interest in each frame, and perform spatial pixel averaging processing to form RGB red, green and blue three-channel signals;

[0085] Since the information directly collected by the system is the original video image containing pulse information, the RGB three-channel data is separated for the region of interest in each frame, and spatial pixel averaging is performed separately to obtain a BVP signal with a high signal-to-noise ratio.

[0086] ——Fixed-Point algorithm: The three channel signals after spatial pixel averaging are used as the observation signal X, and the estimated value of the source signal is obtained from the observation signal X through the separation matrix W. The source signal is separated from the mixed observation signal X by the Fixed-Point algorithm to obtain three ideal independent source signals s1, s2, and s3, and they are correlated with the green channel signal to obtain the noisy pulse wave signal;

[0087] The three channel signals obtained by RGB separation are actually mixed signals containing BVP signals and noise signals. The three channel signals after spatial averaging preprocessing are used as the observation signal X. The collected video observation signal X is the result of linear mixing of independent source signals S through the mixing matrix A. The role of the fixed-point algorithm is to obtain the estimated value of the source signal from the observation signal X through the separation matrix W. The fixed-point algorithm is used to separate the source signal from the mixed observation signal X, and obtain three independent source signals close to the ideal independent source signals s1, s2, and s3.

[0088] Since blood absorbs different amounts of visible light in different wavelengths, generally speaking, visible light absorbs the most light in the wavelength range of 510-590nm, and the least light in the wavelength range of 630-750nm. According to the spectrum of visible light and the wavelength ranges of the three RGB colors, blood absorbs the most green light and the least red light. Therefore, the green signal in the collected RGB three-channel signal can best reflect the information of blood flow. The three ideal independent source signals s1, s2, and s3 separated are respectively correlated with the green channel signal, and the corresponding independent source signal with the largest absolute value of the Pearson correlation coefficient is taken for the next step of processing.

[0089] Since the three independent source signals s1, s2, and s3 obtained by the fixed-point algorithm are disordered, in order to select the required pulse wave signal, they are respectively correlated with the green channel signal (which has the least noise and is closest to the pulse signal) for correlation analysis, and the component with the largest correlation is the noisy pulse wave signal.

[0090] The Fixed-Point algorithm is a type of ICA that uses a fixed-point iterative optimization algorithm to make the convergence faster and more robust, improve real-time performance, reduce detection errors, and improve the accuracy of heart rate detection.

[0091] Improved Complete Ensemble Empirical Mode Decomposition With Adaptive Noise (ICEEMDAN): After decomposing and reconstructing the noisy pulse wave signal, the improved complete ensemble empirical mode decomposition method of adaptive noise is used to select the IMF component that can reflect the structural characteristics of the heart rate spectrum as the estimation signal of the heart rate signal;

[0092] Empirical Mode Decomposition (EMD) is a time-frequency signal analysis method that has the advantages of wavelet transform multi-resolution analysis. It uses itself as the fundamental wave for decomposition and has good adaptability. It is very suitable for processing nonlinear and non-stationary signals. The face color signal has obvious non-stationarity, so the EMD decomposition method is selected to decompose the signal. It adaptively decomposes the signal into an intrinsic mode function (IMF) with a frequency from high to low and a residual component based on the local time scale characteristics of the signal. Each IMF corresponds to a different frequency band, and the modal component corresponding to the heart rate frequency band is the pulse wave.

[0093] Complete Ensemble Empirical Mode Decomposition With Adaptive Noise (CEEMDAN) is an improved method of EMD and EEMD algorithms. CEEDMAN greatly reduces the error of IMF reconstruction by adding positive and negative pairs of Gaussian white noise to each intrinsic mode component (IMF). However, CEEMDAN still has some disadvantages:

[0094] ①The modal components decomposed by CEEMDAN contain some residual noise;

[0095] ②CEEMDAN is the same as EEMD in that it will generate false components in the early stages of signal decomposition and reconstruction.

[0096] Unlike CEEMDAN, the ICEEMDAN algorithm adds a special white noise during the decomposition process, namely the k-th order IMF component after Gaussian white noise is decomposed by EMD. The local mean of the signal plus noise is calculated for each modal component, and the decomposed IMF is defined as the difference between the residual signal and the local mean.

[0097] The ICEEMDAN algorithm greatly reduces the residual noise problem in the IMF component and improves the shortcomings of CEEMDAN in the early stage of decomposition and reconstruction, which is prone to produce false components and modal aliasing. ——Heart rate extraction: The estimated signal of the heart rate signal can be converted into a frequency domain signal through Fourier transform. The frequency corresponding to the maximum peak of the power spectrum density is the heart rate signal. The component with a frequency within the heartbeat frequency band is the estimated pulse wave. The frequency corresponding to the maximum peak of the power spectrum density is the heartbeat frequency f h , heart rate is: HR = 60 × f h .

[0098] Taking the frequency of periodic changes per second as the X-axis and the power spectrum density as the Y-axis, a power spectrum curve can be formed, such as Figure 6 Here, IMF4 is a pulse wave, and its frequency is within the heartbeat frequency band (0.5-2.0 Hz).

[0099] Optimized, during the video acquisition process, under indoor natural light conditions, use an ordinary camera to collect the tester's video at a distance of 50-60 cm from the camera. The image resolution of the ordinary camera is 1920×720 pixels or above, and no additional light source is required.

[0100] Optimized,In the face detection process, a comprehensive Adaboost and Cascade algorithm is used for detection.

[0101] The Adaboost and Cascade algorithms (also known as Harr-Cascade face detector) are combined to achieve face detection. The algorithm detection process is as follows:

[0102] 1) Use the Haar-like features of the sample to represent the face and quickly calculate the facial feature values;

[0103] 2) Use the Adaboost algorithm to select some face rectangle features and perform classifier training;

[0104] 3) Cascade several simple classifiers obtained through training to obtain a cascade classifier. Cascading can improve the speed of detection;

[0105] 4) Use the cascade classifier to detect the area to be detected, and the area that is finally tested by each layer of classifier is the face area.

[0106] Adaboost is an iterative algorithm, the core idea of ​​which is to train different classifiers (weak classifiers) for the same training set, and then combine these weak classifiers to form a stronger final classifier (strong classifier). Compared with other parts, the capillaries on the human face are particularly abundant, and the degree of light absorption is relatively large. The present invention selects the facial related area to extract the BVP signal of the subject. The Adaboost face detection algorithm is used to detect and identify the face data information in the video, and the face area of ​​the image is obtained frame by frame.

[0107] Optimized, the forehead area is selected as the ROI area when selecting the region of interest. Since the collected face video contains a lot of background noise, it is necessary to extract the region of interest (ROI) from the face video. In the measurement, there are often various interferences, such as blinking of human eyes, movement of mouth when speaking, etc., while the forehead area is relatively flat and has relatively few movements. Selecting the forehead area as the ROI area can improve the accuracy of the measurement results.

[0108] After using a camera to obtain a color video containing a face, the video is considered to be composed of many sequential static images. The face region of the image is obtained frame by frame through face detection interest region selection, and finally the face interest region of all video images is obtained. Then the face region of each frame of video is separated by three primary colors to generate an RGB three-channel image. All pixels in this area are represented in the form of (R, G, B), and the components of the RGB channels are (R, 0, 0), (0, G, 0), (0, 0, B).

[0109] Furthermore, when RGB channels are separated and spatial average preprocessed, the one-dimensional source signal of the corresponding channel is obtained according to formula (1):

[0110]

[0111] Where: n is the number of image frames;

[0112] N is the total number of image frames;

[0113] X(n) is the one-dimensional source signal of the R, G or B channel;

[0114] x i,j (n) is the color intensity value of pixel (i, j) in the corresponding channel;

[0115] h and w are the height and width of the image respectively.

[0116] The spatial pixel average reflects the skin's absorption and reflection of light, which is also the subsequent heart rate information. Spatial average preprocessing can effectively suppress noise and improve the signal-to-noise ratio of the BVP signal. In this way, the original signals x1(t), x2(t), and x3(t) of the three RGB channels can be obtained, where t is the time series of the video frame.

[0117] Furthermore, the specific analysis process of the Fixed-Point algorithm includes:

[0118] Three independent source signals s(t) = [s1(t), s2(t), s3(t)] T , the observed signal is the RGB three color channel signal, then:

[0119] x(t)=As(t) (2)

[0120] Where t is the video frame sequence, A∈R3×3 is the unknown non-singular mixing matrix;

[0121]

[0122] in is the optimal estimate of the source signal;

[0123] Centralize and whiten the data to obtain z(t);

[0124] Set an initial matrix W with unit norm;

[0125] Let w i =E{2g(W i T z)}-E{g′(W i T z)}w, where function g is a nonlinear function selected based on the Gaussianity of the data;

[0126] For the matrix W = [W1(t), W2(t), W3(t)] T Perform symmetric orthogonalization:

[0127] W=(WW T ) -1 / 2 W (4)

[0128] If it does not converge, return w i =E{2g(W i T z)}-E{g′(W i T z)}w, until the unmixing matrix is ​​finally obtained, which can be substituted into equation (3) to obtain the optimal estimate of the source signal.

[0129] Furthermore, in the improved adaptive noise complete set empirical mode decomposition, the ICEEMDAN algorithm is used for decomposition, and the decomposition process is as follows:

[0130] Definition E k (·) is the k-th order IMF component generated by EMD decomposition, k = 1, 2, 3, ..., N; M(·) is the local mean of the signal; x(t) is the signal after fixed point algorithm demixing, then:

[0131] E1(x(t))=x(t)-M(x(t)) (5)

[0132] IMF k is the mode obtained by decomposing the modal signal by ICEEMDAN, r k (t) is the kth order residual component, x i (t) is the signal after adding white noise for the ith time; β0 is the amplitude coefficient of the added noise; ω i (t) is the white noise added for the ith time; N is the number of times Gaussian white noise is added;

[0133] The ICEEMDAN algorithm reduces the amount of noise present in the modal by replacing the modal estimate with a local mean estimate and subtracting it from the original signal. The specific implementation steps of the algorithm are as follows:

[0134] Step 1: Calculate by EMD

[0135] x i (t) = x(t) + β0E1(ω i (t)), i=1, 2, 3,..., I (6)

[0136] The first-order residual component is obtained as: r1(t)= <M(x i (t))> (7)

[0137] Step 2, calculate the first (k=1) IMF component as:

[0138] IMF1(t)=x(t)-r1(t) (8)

[0139] Step 3, calculate r1(t)+β1E2(ω i (t)) The average of the local means is taken as the second residual component,

[0140] That is: r2(t)= <M(r1(t))+β1E2(ω i (t))> (9)

[0141] And the second-order IMF component is obtained as:

[0142] IMF2(t)=r1(t)-r2(t)=r1(t)- <M(r1(t))+β1E2(ω i (t))> (10)

[0143] Step 4, when k = 3, ..., N, calculate the k-th order residual component as:

[0144] r k (t)= <M(r k-1 (t))+β k-1 E k (ω i (t))> (11)

[0145] Step 5: Calculate the k-th order IMF component as:

[0146] IMF k (t) = r k-1 (t)-r k (t) (12)

[0147] Step 6, for the next k, go to step 4.

[0148] Repeat steps 4 to 6 until the residual component obtained cannot be further decomposed (i.e. the residual satisfies the IMF condition or its extreme value is less than or equal to 2). At this time, x(t) can be expressed as:

[0149]

[0150] A non-contact heart rate measurement system based on face video, which includes the following modules:

[0151] ——Video acquisition module: collect a segment of face video under natural light, capture video frames and display them on a graphical interface; specifically, use an optical camera to collect a segment of image data to be detected, and extract the static face part corresponding to each frame of the camera by processing this information data. The collected data needs to have a high degree of accuracy, and the collected data needs to be the front face part of the person. The minimum requirement is to be able to detect the PPG signal, and the image must have a pixel size greater than that containing the complete PPG signal.

[0152] ——Face recognition and tracking module: After the camera collects video data, the face recognition and tracking module performs face recognition processing on each frame of the video to keep the face in the center of the video at all times; through the face recognition and tracking module, the camera can be controlled to continuously follow the movement of the face during the movement of the face, so that within the movable range of the camera, the face is always in the middle of the camera, thereby achieving the purpose of accurate face tracking.

[0153] ——Heart rate calculation module, including face detection, facial region of interest selection, skin pixel segmentation, extraction of RGB three-channel signals, RGB three-channel spatial averaging and preprocessing, fixed-point algorithm, ICEEMDAN decomposition, and obtaining BVP signals for frequency domain analysis to calculate heart rate;

[0154] ——Result display module, which displays the measured heart rate results in a graphical interface, and draws the BVP signal processed by combining the Fixed-Point algorithm and the ICEEMDAN algorithm and the waveform after Fourier transformation.

[0155] The optimized,face recognition and tracking module based on the face tracking device of Raspberry Pi performs face recognition processing on each frame of the video,keeping the face in the center of the video at all times.

[0156] The present invention selects Raspberry Pi 4B as the hardware platform of the heart rate measurement system. The hardware platform has the advantages of complete system, short development cycle, high cost performance, strong computing power, etc., and can complete the software programming and operation of the measurement system. The platform supports the Raspbian system developed based on the Linux system and can run the algorithm files for a long time.

[0157] The face tracking system based on Raspberry Pi can first identify the face when the camera captures a face image, and then control the camera to continuously follow the face as it moves, so that the face is always in the middle of the camera within the movable range of the camera, thus achieving the purpose of face tracking.

[0158] like Figure 7 As shown, the servo is used to realize the control of camera movement, and is composed of a horizontal servo 3 and a vertical servo 2. The horizontal servo will move the camera 1 horizontally ("azimuth"), while the vertical servo will move the camera vertically (elevation). The position range of the face is obtained through the camera, and the center coordinates x and y of the face image can be obtained through simple calculation. The horizontal and vertical coordinates are simply processed and multiplied by a proportional coefficient to obtain the corresponding camera rotation angle, which can realize large-scale face tracking.

[0159] The present invention continuously records the video of the face to be tested and performs non-contact measurement and calculation, and uses an electrocardiogram monitor to measure the actual heart rate as a reference value. The experimental data are shown in Table 1:

[0160] Table 1 Experimental data

[0161]

[0162] The calculated results are compared with the measured results of the ECG monitor as shown in Table 2:

[0163] Table 2 Experimental error

[0164]

[0165] It can be seen from Table 1 and Table 2 that the heart rate results measured by the method combining the fixed-point algorithm and ICEEMDAN are very close to the results detected by the electrocardiogram monitor, and the errors are all within 1bpm, with the maximum error being 1bpm. The results measured by the fixed-point algorithm and wavelet transform, and the fixed-point algorithm and CEEMD method are also close to the values ​​measured by the electrocardiogram monitor, but the average error and the root mean square error are large. Therefore, it can be concluded that the heart rate detection method of the present invention can effectively filter out the noise in the noisy pulse wave, making the detection result more accurate.

[0166] In summary, the present invention provides a non-contact heart rate measurement method and system based on face video, which adopts a signal processing method combining a fixed-point algorithm and an empirical mode decomposition algorithm, can realize non-contact heart rate detection, and can improve the accuracy and efficiency of heart rate detection.

[0167] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A non-contact heart rate measurement method based on face video, It is characterized in that The specific steps include: ——Video acquisition: continuous video recording of the face to be tested; ——Face detection: detect and identify the face data information in the recorded video, and obtain the face area of ​​the image frame by frame; ——Region of Interest Selection: After performing face detection on each frame of video, a relatively stable area containing face information is selected as the region of interest ROI; ——RGB channel separation, spatial averaging and preprocessing: Separate the RGB red, green and blue three-channel data for the region of interest in each frame, and perform spatial pixel averaging processing to form RGB red, green and blue three-channel signals; ——Fixed-Point algorithm: The three channel signals after spatial pixel averaging are used as the observation signal X, and the estimated value of the source signal is obtained from the observation signal X through the separation matrix W. The source signal is separated from the mixed observation signal X by the Fixed-Point algorithm to obtain three independent source signals s1, s2, and s3, and they are correlated with the green channel signal to obtain the noisy pulse wave signal; ——Improved adaptive noise complete ensemble empirical mode decomposition: After decomposing and reconstructing the noisy pulse wave signal, the improved adaptive noise complete ensemble empirical mode decomposition method is used to select the IMF component that can reflect the structural characteristics of the heart rate spectrum as the estimation signal of the heart rate signal; In the improved adaptive noise complete set empirical mode decomposition, the ICEEMDAN algorithm is used for decomposition, and the decomposition process is as follows: definition is the k-th order IMF component generated by EMD decomposition, ; is the local mean of the signal; is the signal after fixed-point algorithm demixing, then: (5) set up is the mode obtained by decomposing the modal signal by ICEEMDAN, is the kth order residual component, is the signal after adding white noise for the i-th time; is the amplitude coefficient of the added noise; For the The white noise added; is the number of times Gaussian white noise is added; The ICEEMDAN algorithm reduces the amount of noise present in the mode by replacing the modal estimate with a local mean estimate and subtracting it from the original signal. The specific implementation steps of the algorithm are as follows: Step 1: Calculate by EMD (6) The first-order residual component is obtained as: (7) Step 2, calculate the first (k=1) IMF component as: (8) Step 3, calculate The average of the local means is taken as the second residual component, namely: (9) And the second-order IMF component is obtained as: (10) Step 4, when When , the k-th order residual component is calculated as: (11) Step 5: Calculate the k-th order IMF component as: (12) Step 6, for the next k, go to step 4; Repeat steps 4 to 6 until the residual component obtained cannot be further decomposed (that is, the residual satisfies the IMF condition or its extreme value is less than or equal to 2); at this time It can be expressed as: (13) ——Heart rate extraction: The estimated signal of the heart rate signal is converted into a frequency domain signal. The component with a frequency within the heartbeat frequency band is the estimated pulse wave, and the frequency corresponding to the maximum peak of the power spectrum density is the heartbeat frequency. , the heart rate is: .

2. The non-contact heart rate measurement method based on face video according to claim 1, It is characterized in that During the video acquisition process, under indoor natural light conditions, an ordinary camera was used to collect the test subject's video at a distance of 50-60 cm from the camera.

3. The non-contact heart rate measurement method based on face video according to claim 1, It is characterized in that In the process of face detection, a combination of Adaboost and Cascade algorithms is used for detection.

4. The non-contact heart rate measurement method based on face video according to claim 1, It is characterized in that When selecting the region of interest, the forehead area is selected as the ROI area.

5. The non-contact heart rate measurement method based on face video according to claim 1, It is characterized in that When RGB channels are separated and spatial average preprocessed, the one-dimensional source signal of the corresponding channel is obtained according to formula (1): (1) Where: n is the number of image frames; N is the total number of image frames; One-dimensional source signal for R, G or B channel; is the color intensity value of the pixel (i, j) in the corresponding channel; h and w are the height and width of the image respectively.

6. The non-contact heart rate measurement method based on face video according to claim 5, It is characterized in that The fixed-point algorithm analysis process includes: Three independent source signals , the observed signal is the RGB three color channel signal, then: (2) Where t is the sequence of image frames, A∈R3×3 is a non-singular mixing matrix; (3) in is the optimal estimate of the source signal; The data of the observed signal is centrally whitened to obtain z(t); Set an initial matrix W with unit norm; make , where function g is a nonlinear function selected according to the Gaussianity of the observed signal data; Pair Matrix Perform symmetric orthogonalization: (4) If W does not converge, return , until the unmixing matrix is ​​finally obtained, which is substituted into equation (3) to obtain the optimal estimate of the source signal.

7. A non-contact heart rate measurement system based on face video, used to perform the non-contact heart rate measurement method based on face video according to any one of claims 1 to 6, It is characterized in that Includes the following modules: ——Video acquisition module: responsible for collecting a face video under natural light, capturing video frames and displaying them on the graphical interface; ——Face recognition and tracking module: After collecting video data, the face recognition and tracking module performs face recognition processing on each frame of the video to keep the face in the center of the video at all times; ——Heart rate calculation module: including face detection, facial region of interest selection, skin pixel segmentation, extraction of RGB three-channel signals, RGB three-channel spatial averaging and preprocessing, fixed-point algorithm, ICEEMDAN decomposition, obtaining BVP signals for frequency domain analysis and calculation to obtain heart rate; ——Result display module: Displays the measured heart rate results in a graphical interface, and plots the BVP signal processed by a combination of the Fixed-Point algorithm and the ICEEMDAN algorithm and the Fourier transformed waveform.

8. The non-contact heart rate measurement system based on face video according to claim 7, It is characterized in that The face recognition and tracking module performs face recognition processing on each frame of the video based on the face tracking device of the Raspberry Pi, keeping the face always in the center of the video.