Non-contact multi-person synchronous heart rate measuring system and measuring method thereof
By combining deep learning and blind source separation algorithms with optical flow to filter out noise, a non-contact heart rate measurement system has been developed, which solves the problems of individual identification and signal separation and motion artifacts in multi-person, dynamic scenarios. It achieves multi-person synchronous, high-precision heart rate and heart rate variability monitoring, meeting the large-scale monitoring needs of scenarios such as team training.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-10
AI Technical Summary
Existing non-contact heart rate measurement methods face challenges in scenarios involving multiple people and dynamic motion, such as difficulties in individual identification and signal separation, severe motion artifact interference, and insufficient real-time performance and system robustness. These challenges make it difficult to achieve simultaneous, dynamic, and high-precision real-time monitoring of multiple individuals.
Multi-target recognition and segmentation are performed using deep learning-based target detection algorithms and lightweight semantic segmentation algorithms. Noise is filtered out by combining blind source separation algorithms and optical flow methods. Adaptive filters are used to eliminate motion interference. Real-time calculation and display of heart rate and heart rate variability are achieved through a signal calculation unit.
It enables real-time, independent analysis of heart rate and heart rate variability for multiple subjects, improves the efficiency of group physiological data collection, significantly reduces motion artifacts, maintains the stability and accuracy of measurements, and can display changes in physiological state in real time, providing data support for immediate decision-making.
Smart Images
Figure CN121817836A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of heart rate measurement, and particularly relates to a non-contact multi-person synchronous heart rate measurement system and a measurement method thereof. BACKGROUND
[0002] Heart rate (HR) and heart rate variability (HRV) are key physiological indicators for assessing physiological state, exercise load and health level, and have important application value in the fields of sports training, medical monitoring and health management. Traditional heart rate measurement methods are mainly divided into two categories: contact type and non-contact type.
[0003] The contact type heart rate measurement method usually relies on wearable devices such as heart rate chest bands, wrist bands or patches, and is based on photoplethysmography (PPG) technology to extract heart rate signals by detecting the periodic changes in blood volume under the skin. Although this method has high measurement accuracy and stability, its limitations are also very significant: in a multi-person, dynamic scene (such as team training, group fitness, event monitoring, etc.), independent devices need to be provided for each subject, which not only is cumbersome to deploy and costly, but also may interfere with performance and comfort, making it difficult to achieve large-scale, real-time group synchronization monitoring.
[0004] The non-contact heart rate measurement method is based on remote photoplethysmography (rPPG) technology, which extracts the weak skin color change signal caused by heartbeats from the video images of human face or body surface area collected by a camera, and then calculates the heart rate and heart rate variability. This method does not require any sensors, has the advantages of non-invasiveness, easy deployment and wide application scenarios, and is particularly suitable for physiological monitoring in multi-person, long-term and natural state.
[0005] However, the existing non-contact heart rate measurement method still faces the following technical bottlenecks when facing multi-person and dynamic motion scenes: First, multi-person recognition and signal separation are difficult: when multiple moving individuals appear in the video at the same time, it is difficult to achieve accurate individual segmentation and independent signal extraction, which easily leads to signal crosstalk between different individuals, seriously affecting the accuracy of heart rate calculation.
[0006] Second, motion artifacts are serious: head shaking, body displacement and posture changes produced by the measured person during exercise will introduce strong motion noise, which will change the pixel position and size of the signal acquisition area in the video constantly. This not only destroys the periodic characteristics of the PPG signal, but also introduces a large amount of motion artifacts, making the traditional rPPG algorithm based on fixed position invalid.
[0007] Thirdly, real-time and system robustness are insufficient: non-contact measurement needs to complete complex calculations such as video stream processing, image processing and signal analysis synchronously, and the existing technology is difficult to maintain sufficient robustness while ensuring the calculation efficiency in the face of complex scenes such as multiple people moving quickly, and problems such as system response delay, measurement interruption or result distortion often occur.
[0008] In summary, although the non-contact heart rate measurement method has significant application potential, there are still obvious deficiencies in precision, real-time and robustness in the multi-person, high dynamic scene. Therefore, it has important technical significance and broad application prospect to develop a non-contact heart rate measurement system and method that can effectively solve the above problems and realize multi-person synchronization, dynamic adaptation and high-precision real-time monitoring. SUMMARY
[0009] Therefore, the present application aims to provide a non-contact multi-person synchronous heart rate measurement system and its measurement method to solve the technical problems of existing non-contact heart rate measurement methods in multi-person, dynamic scenes, such as difficulty in individual recognition and separation, large motion interference, and poor real-time performance.
[0010] To achieve the above-mentioned purpose, the technical scheme of the present application is as follows: A non-contact multi-person synchronous heart rate measurement system, comprising a camera, a processor and a display, the camera being used to collect video images containing multiple subjects; the processor is used to receive and process the video images collected by the camera; the processor includes a target detection unit, a region of interest segmentation unit, a region of interest tracking unit, a signal extraction unit, a noise filtering unit and a signal calculation unit; wherein the target detection unit is used to detect multiple targets for each frame of the video image to identify the positions of each subject in the video image; the region of interest segmentation unit is used to separate the region of interest of each subject from the video image; the region of interest tracking unit is used to dynamically track the separated region of interest of each subject; the signal extraction unit is used to extract the photoplethysmographic wave signal from the region of interest of each subject; the noise filtering unit is used to filter the noise in the photoplethysmographic wave signal; the signal calculation unit processes and analyzes the photoplethysmographic wave signal after filtering the noise, and calculates the heart rate and heart rate variability index of each subject; the display is used to display the video image processed by the calculation unit in real time, and superimpose the real-time heart rate and heart rate variability index of each subject on the image.
[0011] Further, the signal extraction unit comprises: An original signal extraction module is configured to calculate, for each frame t in a time window, average values of all pixels in a region of interest of each subject in three color channels R, G and B, to obtain three original time series signals ; A signal detrending processing module is configured to perform detrending processing on the three original time series signals using a smoothing filter. A signal normalization processing module is configured to perform normalization processing on the three time series signals after the detrending processing, so that the mean value is 0 and the standard deviation is 1, to obtain three standardized channel signals . A blind source separation module is configured to perform source signal separation on the three standardized channel signals using a blind source separation algorithm based on second-order statistics; a formula for the source signal separation is: . wherein, observed signal vector, . separated source signal vector, . demixing matrix; T represents a transpose symbol. A pulse signal component selection module is configured to perform frequency domain analysis on each separated source signal , calculate a power spectral density thereof, and select a component having a most significant periodic peak value in a human heart rate frequency range as a photoplethysmography signal .
[0012] Further, the noise filtering unit includes: A reference noise generation module is configured to estimate an average motion vector in a region of interest of each subject using an optical flow method, and generate a reference noise signal related to motion of each subject . A reference noise filtering module is configured to filter the reference noise signal from the photoplethysmography signal using an adaptive filter, to obtain a filtered photoplethysmography signal ; wherein a filtering formula is: . wherein, weight of the adaptive filter.
[0013] Further, the signal calculation unit includes: A Fourier transform module is configured to perform fast Fourier transform on the photoplethysmography signal , to obtain a power spectral density. The peak frequency finding module is used to find the peak frequency of the power spectral density within a preset heart rate frequency range. ; Heart rate calculation module, used to calculate based on peak frequency Calculate heart rate (HR); the calculation formula is: ; The peak detection module is used to detect pulse wave signals using a wavelet transform-based peak detection algorithm. Peak detection was performed to obtain the pulse wave peak sequence. ;in, This indicates the photoplethysmography (PPG) signal. The first one identified in +1 pulse wave peak value corresponding to the time point This indicates the photoplethysmography (PPG) signal. The first one identified in The time point corresponding to the peak value of each pulse wave; The SDNN index calculation module is used to calculate the index based on the pulse wave peak sequence. Calculate the standard deviation of all interpeak periods using SDNN; the formula is: ; Where N represents the total number of detected pulse wave peaks. This represents the interpeak period of the i-th peak. This represents the average value of all interpeak intervals; The RMSSD index calculation module is used to calculate the peak pulse wave sequence. Calculate the root mean square (RMSSD) of the interpeak duration difference between adjacent peaks; the formula is as follows: ; in, This represents the interpeak period of the (i+1)th peak.
[0014] Furthermore, the target detection unit employs a deep learning-based target detection algorithm to perform multi-target detection on the video images; The Region of Interest (ROI) segmentation unit uses a lightweight semantic segmentation algorithm to separate the ROI of each subject from the video image; The region of interest tracking unit uses a deep learning-based multi-target tracking algorithm to dynamically track the region of interest for each subject.
[0015] A non-contact, multi-person synchronous heart rate measurement method, implemented using the aforementioned non-contact, multi-person synchronous heart rate measurement system, includes the following steps: S1: Capture video images containing multiple subjects using a camera; S2: The processor performs multi-target detection on each frame of the video image to identify the position of each subject in the video image, then separates the region of interest of each subject from the video image and dynamically tracks it, and extracts the photoplethysmography signal from the region of interest of each subject. After noise filtering, it is processed and analyzed to calculate the heart rate and heart rate variability index of each subject. S3: The video images processed by the computing unit are displayed in real time on the monitor, and the real-time heart rate and heart rate variability index are superimposed on the image of each subject.
[0016] Furthermore, in step S2, the process of extracting the photoplethysmography (PPG) signal from the region of interest of each subject includes the following steps: S201: At each frame t within the time window, calculate the average value of all pixels in the R, G, and B color channels of the region of interest for each subject, thus obtaining three raw time series signals. ; S202: Using a smoothing filter on the three original time series signals Perform detrending processing; S203: Three time series signals after detrending processing Normalization was performed to make the mean 0 and the standard deviation 1, resulting in three standardized channel signals. ; S204: A blind source separation algorithm based on second-order statistics is used to analyze the three standardized channel signals. Perform source signal separation; the formula for source signal separation is: ; in, Represents the observed signal vector. ; This represents the separated source signal vector. ; Represents the unmixing matrix; Indicates the transpose symbol; S205: For each separated source signal Frequency domain analysis was performed to calculate its power spectral density, and the component with the most significant periodic peaks within the human heart rate frequency range was selected as the photoplethysmography (PPG) signal. .
[0017] Furthermore, in step S2, the process of filtering out noise from the photoplethysmography signal includes the following steps: S206: The average motion vector within the region of interest for each subject is estimated using the optical flow method, and a reference noise signal related to the motion of each subject is generated. ; S207: filtering the reference noise signal from the photoplethysmogram signal using the adaptive filter to obtain a filtered photoplethysmogram signal; ; wherein the filtering formula is: ; wherein, represents the weight of the adaptive filter.
[0018] Further, in step S2, the photoplethysmogram signal after noise filtering is processed and analyzed to calculate the heart rate and heart rate variability index of each subject, including the following steps: S208: performing fast Fourier transform on the photoplethysmogram signal to obtain a power spectral density; S209: finding the peak frequency of the power spectral density in the preset heart rate frequency range; S210: calculating the heart rate HR according to the peak frequency ; wherein the calculation formula is: ; S211: performing peak detection on the pulse wave signal using a peak detection algorithm based on wavelet transform to obtain a pulse wave peak sequence ; wherein, represents the time point corresponding to the i-th pulse wave peak identified from the photoplethysmogram signal ; represents the time point corresponding to the i-th pulse wave peak identified from the photoplethysmogram signal ; S212: calculating the standard deviation SDNN of all peak-to-peak intervals and the root mean square RMSSD of adjacent peak-to-peak interval differences according to the pulse wave peak sequence ; The calculation formula of the standard deviation SDNN is: ; wherein, N represents the total number of detected pulse wave peaks, represents the i-th peak-to-peak interval, represents the average value of all peak-to-peak intervals; The calculation formula of the root mean square RMSSD is: ; wherein, represents the (i+1)th peak-to-peak interval.
[0019] Further, in step S2, a target detection algorithm based on deep learning is used for multi-target detection of the video image; a lightweight semantic segmentation algorithm is used to separate the region of interest of each testee from the video image; and a multi-target tracking algorithm based on deep learning is used for dynamic tracking of the region of interest of each testee.
[0020] Compared with the prior art, the present application can achieve the following beneficial effects: 1. The present application can simultaneously accurately position and independently track multiple testees by using a target detection algorithm based on deep learning, a lightweight semantic segmentation algorithm, and a multi-target tracking algorithm based on deep learning, thereby realizing real-time, independent, and parallel heart rate and heart rate variability analysis of multiple testees, greatly improving the efficiency of group physiological data acquisition, and meeting the needs of large-scale monitoring in team training, group fitness courses, smart classroom or ward monitoring, and other scenarios.
[0021] 2. The present application uses a blind source separation algorithm to extract pure pulse wave signals from multiple channel signals, combines with an optical flow method to estimate motion vectors in real time and generate reference noise, and uses an adaptive filter to dynamically eliminate motion interference, thereby effectively suppressing motion artifacts caused by head shaking, body displacement, posture changes, and other factors, significantly reducing the damage to rPPG signals, and maintaining the stability and accuracy of measurement in a vigorous exercise scenario, thereby overcoming the pain point of the failure of traditional non-contact methods in a motion scenario.
[0022] 3. The present application not only realizes basic heart rate measurement, but also further realizes accurate calculation of heart rate variability indicators (SDNN, RMSSD), which can quantitatively evaluate the function of the autonomic nervous system, stress level, and fatigue recovery state, and provides more abundant and accurate data dimensions for personalized training guidance and health risk warning.
[0023] 4. The measurement results can be displayed in real time on the screen of the display, so that the coach, medical staff can intuitively and conveniently observe the physiological state changes of each person, and provide strong data support for immediate decision-making. BRIEF DESCRIPTION OF DRAWINGS
[0024] The accompanying drawings, which form a part of the present application, are used to provide further understanding of the present application, and the schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A logic structure schematic diagram of the non-contact multi-person synchronous heart rate measurement system described in the embodiments of the present application; Figure 2A flowchart of a non-contact multi-person synchronous heart rate measurement method according to an embodiment of the present invention.
[0025] Explanation of reference signs: camera 1, processor 2, target detection unit 21, region of interest segmentation unit 22, region of interest tracking unit 23, signal extraction unit 24, noise filtering unit 25, signal calculation unit 26, and display 3. DETAILED DESCRIPTION
[0026] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation on the present invention.
[0027] It should be noted that the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.
[0028] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present invention and simplify the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited by "first", "second", and the like can explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality of" is two or more.
[0029] In the description of the present invention, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", and "connection" should be understood broadly, for example, it can be a fixed connection, or a detachable connection, or an integral connection; it can be a mechanical connection, or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present invention can be understood according to the specific circumstances.
[0030] The present invention will be described in detail below with reference to the drawings and embodiments.
[0031] As Figure 1As shown, the embodiment of the present application provides a non-contact multi-person synchronous heart rate measurement system, which comprises a camera 1, a processor 2 and a display 3. The camera 1 is used to collect video images containing multiple subjects. The processor 2 is used to receive and process the video images collected by the camera 1. The processor 2 comprises a target detection unit 21, a region of interest segmentation unit 22, a region of interest tracking unit 23, a signal extraction unit 24, a noise filtering unit 25 and a signal calculation unit 26. The target detection unit 21 is used to perform multi-target detection on each frame of the video image to identify the positions of each subject in the video image. The region of interest segmentation unit 22 is used to separate the region of interest of each subject from the video image. The region of interest tracking unit 23 is used to dynamically track the separated region of interest of each subject. The signal extraction unit 24 is used to extract a photoplethysmography (PPG) signal from the region of interest of each subject. The noise filtering unit 25 is used to filter the noise in the PPG signal. The signal calculation unit 26 processes and analyzes the PPG signal after noise filtering, calculates the heart rate and heart rate variability index of each subject. The display 3 is used to display the video image processed by the calculation unit 2 in real time, and superimpose the real-time heart rate and heart rate variability index on the image of each subject.
[0032] The camera 1 usually adopts a network camera or a smart camera with high resolution (such as 1080p resolution or higher) and high frame rate (such as 30 fps or higher).
[0033] The calculation unit 2 is the core of the measurement system, usually composed of a high-performance computer, a graphics processing unit (GPU) or a dedicated embedded system, responsible for running the core algorithm of the present application. It is preferred to use a GPU as the calculation unit 2, and the entire algorithm flow (including inference of deep learning model, image processing and signal analysis) is parallelized on the GPU through CUDA, which greatly shortens the single-frame processing time and meets the real-time requirement.
[0034] The target detection unit 21 adopts a deep learning-based target detection algorithm (such as YOLOv5) to perform multi-target detection on the video image.
[0035] The region of interest segmentation unit 22 adopts a lightweight semantic segmentation algorithm (such as STDC) to separate the region of interest of each subject from the video image.
[0036] The region of interest tracking unit 23 adopts a deep learning-based multi-target tracking algorithm (such as DeepSORT) to dynamically track the region of interest of each subject.
[0037] The signal extraction unit 24 comprises a raw signal extraction module, a signal detrending processing module, a signal normalization processing module, a blind source separation module and a pulse signal component selection module.
[0038] The raw signal extraction module is configured to calculate the average value of all pixels in the region of interest of each subject in the red (R), green (G) and blue (B) color channels in each frame t within a time window, to obtain three raw time series signals .
[0039] The signal detrending processing module is configured to perform detrending processing on the three raw time series signals using a smoothing filter.
[0040] The signal normalization processing module is configured to perform normalization processing on the three time series signals after the detrending processing, so that the mean value is 0 and the standard deviation is 1, to obtain three normalized channel signals .
[0041] The blind source separation module is configured to perform source signal separation on the three normalized channel signals using a blind source separation algorithm based on second-order statistics; the formula for source signal separation is: ; wherein represents an observation signal vector, ; represents a separated source signal vector, , corresponding to the channel signals ; represents an unmixing matrix; represents a transposition symbol.
[0042] The pulse signal component selection module performs frequency domain analysis on each separated source signal , calculates its power spectral density, and selects the component with the most significant periodic peak in the human heart rate frequency range as the photoplethysmographic pulse wave signal .
[0043] The noise filtering unit 25 comprises a reference noise generation module and a reference noise filtering module.
[0044] The reference noise generation module is configured to estimate the average motion vector in the region of interest of each subject using an optical flow method, to generate a reference noise signal related to the motion of each subject.
[0045] The reference noise filtering module is configured to use an adaptive filter to filter the reference noise signal from the photoplethysmographic pulse wave signal After filtering, the filtered photoplethysmography (PPG) signal is obtained. The filtering formula is as follows: ; in, The weights of the adaptive filter are represented by T, which indicates the transpose sign.
[0046] The signal calculation unit includes a Fourier transform module, a peak frequency finding module, a heart rate calculation module, a peak detection module, an SDNN index calculation module, and an RMSSD index calculation module.
[0047] The Fourier transform module is used to process photoplethysmography (PPG) signals. Perform a fast Fourier transform to obtain the power spectral density.
[0048] The peak frequency finding module is used to find the peak frequency of the power spectral density within a preset heart rate frequency range. .
[0049] Heart rate calculation module, used to calculate based on peak frequency Calculate heart rate (HR); the calculation formula is: ; The peak detection module is used to detect pulse wave signals using a wavelet transform-based peak detection algorithm. Peak detection was performed to obtain the pulse wave peak sequence. ;in, This indicates the photoplethysmography (PPG) signal. The first one identified in +1 pulse wave peak value corresponding to the time point This indicates the photoplethysmography (PPG) signal. The first one identified in The time point corresponding to the peak value of each pulse wave.
[0050] The SDNN index calculation module is used to calculate based on the pulse wave peak sequence. Calculate the standard deviation of all interpeak periods using SDNN; the formula is: ; Where N represents the total number of detected pulse wave peaks. This represents the interpeak period of the i-th peak. This represents the average value of all interpeak intervals.
[0051] The RMSSD index calculation module is used to calculate the peak pulse wave sequence. Calculate the root mean square (RMSSD) of the interpeak duration difference between adjacent peaks; the formula is as follows: ; wherein, represents the i+1th peak-to-peak interval.
[0052] The above describes in detail the structure of the non-contact multi-person synchronous heart rate measurement system provided by the embodiments of the present application. Corresponding to the measurement system, the embodiments of the present application also provide a non-contact multi-person synchronous heart rate measurement method implemented by using the measurement system.
[0053] As Figure 2 shown, the non-contact multi-person synchronous heart rate measurement method provided by the embodiments of the present application includes the following steps: S1: acquiring video images containing multiple subjects by a camera.
[0054] S2: performing multi-target detection on each frame of the video images by a processor to identify the positions of each subject in the video images, separating the region of interest of each subject from the video images and dynamically tracking the region of interest, and extracting a photoplethysmography signal from the region of interest of each subject, after noise filtering, processing and analyzing the photoplethysmography signal to calculate the heart rate and heart rate variability index of each subject.
[0055] The present application adopts a target detection algorithm based on deep learning to perform multi-target detection on the video images. The target detection algorithm based on deep learning includes but is not limited to the YOLOv5 algorithm, which can locate all the subjects in the video frames in real time and output the coordinates of the bounding box of each subject.
[0056] The present application adopts a lightweight semantic segmentation algorithm to separate the region of interest of each subject from the video images. The lightweight semantic segmentation algorithm includes but is not limited to the STDC algorithm. On the basis of multi-target detection, a lightweight semantic segmentation network is introduced to generate a pixel-level mask for each detected subject, which accurately outlines the contour of the face or upper body and effectively excludes the interference of background pixels. The region of interest is the region covered by the mask, that is, the detection region of the subject.
[0057] In order to maintain stable tracking of the same subject between consecutive video frames, the present application adopts a multi-target tracking algorithm based on deep learning to dynamically track the region of interest of each subject. The lightweight semantic segmentation algorithm includes but is not limited to the DeepSORT algorithm. This algorithm combines the target detection result with a Kalman filter to predict the position of the target in the next frame, and through the calculation of the cosine distance of the deep features (extracted by an independent Re-ID network), it associates the newly detected target with the existing track, effectively solving the ID switching problem caused by occlusion and rapid movement.
[0058] The application adopts a target detection network based on deep learning, training data of which contains crowd images of various postures, clothes and backgrounds, to ensure accurate identification of multiple moving individuals in complex scenes.
[0059] In order to extract a photoplethysmography signal (i.e., rPPG signal) that can be used for HRV analysis from a noisy moving video, the application extracts the signal in the following manner, specifically including the following steps: S201: For each tracked region of interest, calculate the average value of all pixels in the region of interest in the red (R), green (G), and blue (B) color channels in each frame t within a time window, to obtain three original time series signals .
[0060] S202: Use a smoothing filter to perform detrending processing on the three original time series signals to eliminate low-frequency drift caused by slow changes in illumination.
[0061] S203: Perform normalization processing on the three time series signals after detrending processing, so that the mean value of the time series signals is 0 and the standard deviation is 1, to obtain three standardized channel signals . The three standardized channel signals can be regarded as being linearly mixed by multiple independent source signals (including pulse wave signals and noise).
[0062] S204: Use a blind source separation algorithm based on second-order statistics to separate the source signals from the three standardized channel signals ; the formula for source signal separation is: ; wherein represents an observation signal vector ; represents a separated source signal vector ; represents a demixing matrix ; and represents a transpose symbol.
[0063] S205: Perform frequency domain analysis on each separated source signal , calculate its power spectral density, and select the component with the most significant periodic peak value in the human heart rate frequency range as the photoplethysmography signal .
[0064] For example, in a typical heart rate frequency range (0.7-4Hz, corresponding to 42-240 bpm), according to the amplitude, sharpness and signal-to-noise ratio of the spectral peak, the most concentrated and sharpest spectral peak is selected, and the source signal component with the most prominent periodic peak is determined as the pure photoplethysmogram signal .
[0065] S206: Estimate the average motion vector in the region of interest of each subject using the optical flow method to generate a reference noise signal related to the motion of each subject .
[0066] S207: Use an adaptive filter to filter out the reference noise signal from the photoplethysmogram signal , to obtain a filtered photoplethysmogram signal ; wherein the filtering formula is: ; wherein, represents the weight of the adaptive filter, which will be iteratively updated according to the principle of minimizing the mean square error; T represents the transpose symbol.
[0067] In order to further eliminate motion artifacts, the present application introduces an adaptive filter. The average motion vector in the region of interest is estimated by the optical flow method to generate a reference noise signal highly related to the motion . Then, using the LMS (Least Mean Squares) adaptive filter, subtract the noise component related to from , so as to filter out the motion artifacts estimated by the optical flow method as reference noise, and obtain a high-fidelity photoplethysmogram signal , significantly reducing the damage to the rPPG signal, maintaining the stability and accuracy of the measurement in the case of intense motion, and overcoming the pain point problem of the failure of traditional non-contact methods in the motion scene.
[0068] Using the adaptive filter to dynamically eliminate motion interference, the motion artifacts caused by head shaking, body displacement, posture change, etc. are effectively suppressed.
[0069] After obtaining the high-fidelity photoplethysmogram signal , it is processed and analyzed to calculate the heart rate and heart rate variability index of each subject, including the following steps: S208: Perform fast Fourier transform on the photoplethysmogram signal to obtain the power spectral density.
[0070] S209: finding the peak frequency of the power spectral density within a preset heart rate frequency range .
[0071] S210: calculating the heart rate HR according to the peak frequency .
[0072] The calculation formula of the heart rate HR is: .
[0073] S211: performing peak detection on the pulse wave signal using a peak detection algorithm based on wavelet transform to obtain a pulse wave peak sequence ; wherein, denotes the time point corresponding to the i-th pulse wave peak identified from the photoplethysmogram signal , denotes the time point corresponding to the i+1-th pulse wave peak identified from the photoplethysmogram signal .
[0074] Performing peak detection on the pulse wave signal using a peak detection algorithm based on wavelet transform identifies the peak points of each cardiac cycle, and obtains a series of time points at which the peaks occur: ; Calculating the time interval between consecutive peaks, i.e., the peak-to-peak interval (PPI), obtains the pulse wave peak sequence : .
[0075] S212: calculating the heart rate variability index according to the obtained pulse wave peak sequence .
[0076] The heart rate variability index includes the standard deviation SDNN of all peak-to-peak intervals and the root mean square RMSSD of adjacent peak-to-peak interval differences.
[0077] The standard deviation SDNN mainly reflects the overall level of heart rate variability, and its calculation formula is: ; wherein, N represents the total number of detected pulse wave peaks, denotes the i-th peak-to-peak interval, denotes the average value of all peak-to-peak intervals.
[0078] The root mean square RMSSD mainly reflects the activity of the parasympathetic nervous system, and its calculation formula is: ; wherein, represents the i+1th peak-to-peak interval.
[0079] Before calculating the heart rate variability index, the purified pulse wave signal may be subjected to quality assessment. The assessment indicators include signal-to-noise ratio (SNR), signal kurtosis (Kurtosis) and spectral purity. Only when the signal quality score exceeds the preset threshold, the subsequent pulse wave peak sequence extraction and heart rate variability index calculation are performed, and the display result is updated. In this way, it can effectively avoid the occurrence of unreliable and dramatic fluctuations in the heart rate variability index due to low-quality signals.
[0080] On the basis of realizing heart rate measurement, the present application realizes accurate calculation of heart rate variability indexes (SDNN, RMSSD), can quantitatively evaluate the function of the autonomic nervous system, the stress level and the fatigue recovery state, and provides more abundant and more accurate data dimensions for personalized training guidance and health risk warning.
[0081] S3: Real-time display of the video image processed by the calculation unit on the display, and superimposed display of the real-time heart rate and heart rate variability index on the image of each measured person.
[0082] After the heart rate and heart rate variability index are calculated, the calculation result is displayed on the display. Specifically, the calculation unit sends the processed video frame (draws a bounding box on each region of interest and labels the calculated heart rate value and heart rate variability index) to the display for real-time display, so that the coach, medical staff can intuitively and conveniently observe the physiological state changes of each person, providing strong data support for immediate decision-making.
[0083] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions of the present disclosure can be achieved, which are not limited herein.
[0084] The above specific embodiments do not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.
Claims
1. A non-contact multi-person synchronous heart rate measurement system, comprising a camera for acquiring video images containing multiple subjects; characterized in that, The non-contact multi-person synchronous heart rate measurement system also includes a processor and a display; among which, The processor receives and processes video images captured by the camera. The processor includes a target detection unit, a region of interest (ROI) segmentation unit, a ROI tracking unit, a signal extraction unit, a noise filtering unit, and a signal calculation unit. Specifically, the target detection unit performs multi-target detection on each frame of the video image to identify the position of each subject in the video image; the ROI segmentation unit separates the ROI for each subject from the video image; the ROI tracking unit dynamically tracks the separated ROI for each subject; the signal extraction unit extracts photoplethysmography (PPG) signals from the ROI of each subject; the noise filtering unit filters out noise from the PPG signals; and the signal calculation unit processes and analyzes the noise-filtered PPG signals to calculate the heart rate and heart rate variability index for each subject. The display is used to show the video images processed by the computing unit in real time, and to overlay the real-time heart rate and heart rate variability index on the image of each subject.
2. The non-contact multi-person synchronous heart rate measurement system according to claim 1, characterized in that, The signal extraction unit includes: The raw signal extraction module is used to calculate the average value of all pixels in the R, G, and B color channels of the region of interest for each subject in each frame t within the time window, thus obtaining three raw time series signals. ; The signal detrending processing module is used to process three original time series signals using a smoothing filter. Perform detrending processing; The signal normalization processing module is used to normalize the three time series signals after detrending processing. Normalization was performed to make the mean 0 and the standard deviation 1, resulting in three standardized channel signals. ; The blind source separation module is used to perform blind source separation on three standardized channel signals using a second-order statistics-based algorithm. Perform source signal separation; the formula for source signal separation is: ； in, Represents the observed signal vector. ; This represents the separated source signal vector. ; Represents the unmixing matrix; Indicates the transpose symbol; The pulse signal component selection module selects each separated source signal. Frequency domain analysis was performed to calculate its power spectral density, and the component with the most significant periodic peaks within the human heart rate frequency range was selected as the photoplethysmography (PPG) signal. .
3. The non-contact multi-person synchronous heart rate measurement system according to claim 2, characterized in that, The noise filtering unit includes: The reference noise generation module is used to estimate the average motion vector within the region of interest for each subject using the optical flow method, and to generate a reference noise signal related to the motion of each subject. ; The reference noise filtering module is used to filter the reference noise signal using an adaptive filter. From photoplethysmography pulse wave signal After filtering, the filtered photoplethysmography (PPG) signal is obtained. The filtering formula is as follows: ； in, This represents the weights of the adaptive filter.
4. The non-contact multi-person synchronous heart rate measurement system according to claim 3, characterized in that, The signal calculation unit includes: The Fourier transform module is used to analyze photoplethysmography (PPG) signals. Perform a fast Fourier transform to obtain the power spectral density; The peak frequency finding module is used to find the peak frequency of the power spectral density within a preset heart rate frequency range. ; Heart rate calculation module, used to calculate based on peak frequency Calculate heart rate (HR); the calculation formula is: ； The peak detection module is used to detect pulse wave signals using a wavelet transform-based peak detection algorithm. Peak detection was performed to obtain the pulse wave peak sequence. ;in, This indicates the photoplethysmography (PPG) signal. The first one identified in +1 pulse wave peak value corresponding to the time point This indicates the photoplethysmography (PPG) signal. The first one identified in The time point corresponding to the peak value of each pulse wave; The SDNN index calculation module is used to calculate the index based on the pulse wave peak sequence. Calculate the standard deviation of all interpeak periods using SDNN; the formula is: ； Where N represents the total number of detected pulse wave peaks. This represents the interpeak period of the i-th peak. This represents the average value of all interpeak intervals; The RMSSD index calculation module is used to calculate the peak pulse wave sequence. Calculate the root mean square (RMSSD) of the interpeak duration difference between adjacent peaks; the formula is as follows: ； in, This represents the interpeak period of the (i+1)th peak.
5. The non-contact multi-person synchronous heart rate measurement system according to claim 1, characterized in that, The target detection unit uses a deep learning-based target detection algorithm to perform multi-target detection on video images; The Region of Interest (ROI) segmentation unit uses a lightweight semantic segmentation algorithm to separate the ROI of each subject from the video image; The region of interest tracking unit uses a deep learning-based multi-target tracking algorithm to dynamically track the region of interest for each subject.
6. A non-contact multi-person synchronous heart rate measurement method, implemented using the non-contact multi-person synchronous heart rate measurement system according to any one of claims 1 to 5, characterized in that, Includes the following steps: S1: Capture video images containing multiple subjects using a camera; S2: The processor performs multi-target detection on each frame of the video image to identify the position of each subject in the video image, then separates the region of interest of each subject from the video image and dynamically tracks it, and extracts the photoplethysmography signal from the region of interest of each subject. After noise filtering, it is processed and analyzed to calculate the heart rate and heart rate variability index of each subject. S3: The video images processed by the computing unit are displayed in real time on the monitor, and the real-time heart rate and heart rate variability index are superimposed on the image of each subject.
7. The non-contact multi-person synchronous heart rate measurement method according to claim 6, characterized in that, In step S2, the process of extracting the photoplethysmography (PPG) signal from the region of interest of each subject includes the following steps: S201: At each frame t within the time window, calculate the average value of all pixels in the R, G, and B color channels of the region of interest for each subject, thus obtaining three raw time series signals. ; S202: Using a smoothing filter on the three original time series signals Perform detrending processing; S203: Three time series signals after detrending processing Normalization was performed to make the mean 0 and the standard deviation 1, resulting in three standardized channel signals. ; S204: A blind source separation algorithm based on second-order statistics is used to analyze the three standardized channel signals. Perform source signal separation; the formula for source signal separation is: ； in, Represents the observed signal vector. ; This represents the separated source signal vector. ; Represents the unmixing matrix; Indicates the transpose symbol; S205: For each separated source signal Frequency domain analysis was performed to calculate its power spectral density, and the component with the most significant periodic peaks within the human heart rate frequency range was selected as the photoplethysmography (PPG) signal. .
8. The non-contact multi-person synchronous heart rate measurement method according to claim 7, characterized in that, In step S2, the process of filtering noise from the photoplethysmography signal includes the following steps: S206: The average motion vector within the region of interest for each subject is estimated using the optical flow method, and a reference noise signal related to the motion of each subject is generated. ; S207: Use an adaptive filter to filter the reference noise signal From photoplethysmography pulse wave signal After filtering, the filtered photoplethysmography (PPG) signal is obtained. The filtering formula is as follows: ； in, This represents the weights of the adaptive filter.
9. The non-contact multi-person synchronous heart rate measurement method according to claim 7, characterized in that, In step S2, the noise-filtered photoplethysmography (PPG) signal is processed. The process of processing and analyzing data to calculate the heart rate and heart rate variability for each subject includes the following steps: S208: For photoplethysmography (PPG) signals Perform a fast Fourier transform to obtain the power spectral density; S209: Find the peak frequency of the power spectral density within a preset heart rate range. ; S210: Based on peak frequency Calculate heart rate (HR); the calculation formula is: ； S211: A peak detection algorithm based on wavelet transform is used to analyze the pulse wave signal. Peak detection was performed to obtain the pulse wave peak sequence. ;in, This indicates the photoplethysmography (PPG) signal. The first one identified in +1 pulse wave peak value corresponding to the time point This indicates the photoplethysmography (PPG) signal. The first one identified in The time point corresponding to the peak value of each pulse wave; S212: Based on the pulse wave peak sequence Calculate the standard deviation of all peak intervals (SDNN) and the root mean square (RMSSD) of the difference between adjacent peak intervals; The formula for calculating the standard deviation of SDNN is: ； Where N represents the total number of detected pulse wave peaks. This represents the interpeak period of the i-th peak. This represents the average value of all interpeak intervals; The formula for calculating the root mean square (RMSSD) is: ； in, This represents the interpeak period of the (i+1)th peak.
10. The non-contact multi-person synchronous heart rate measurement method according to claim 6, characterized in that, In step S2, a deep learning-based target detection algorithm is used to perform multi-target detection on the video image; a lightweight semantic segmentation algorithm is used to separate the region of interest (ROI) of each subject from the video image; and a deep learning-based multi-target tracking algorithm is used to dynamically track the ROI of each subject.