Non-contact body physiological signal acquisition system and method

Abstract

The invention provides a non-contact body physiological signal acquisition system. The non-contact body physiological signal acquisition system comprises a, a face age group and gender identification function subsystem; b, a non-contact remote photoelectric volume pulse wave function subsystem; and c, a heart rate variability and respiratory rate monitoring function subsystem based on rPPG. The invention further provides a non-contact body physiological signal acquisition method. The non-contact body physiological signal acquisition method comprises the following steps: step A, a face age group and gender identification method; step B, non-contact remote photoelectric volume pulse wave measurement; and step C, realization of a heart rate variability and respiratory rate monitoring function based on rPPG. The defects in the prior art are overcome, and compared with a traditional health monitoring means, non-contact body physiological signal monitoring can also achieve accurate detection without wearing equipment or skin contact. By means of the technology, the problems of cross infection and the like can be effectively avoided, and meanwhile the convenience and comfort of health monitoring are improved.

Description

Technical Field

[0001] The present invention relates to the field of vital sign monitoring and data fusion, and in particular to a non-contact body physiological signal acquisition system and method. Background Art

[0002] At present, the body physiological signal collection devices on the market are divided into several forms: The first type is desktop monitoring equipment, which is used by professional doctors and nurses in hospital departments to monitor the routine physiological characteristics of critically ill patients. For example, the C100 cardiovascular monitor produced by Shenzhen Keman Medical Equipment Co., Ltd.

[0003] The shortcomings of desktop monitoring equipment are also obvious: 1. Poor portability: It is large in size and can usually only be used in fixed locations.

[0004] 2. Complex to use: requires professional operation and is suitable for medical institutions rather than home use.

[0005] 3. High cost: The equipment is expensive and the maintenance cost is also high.

[0006] The second type is wearable monitoring equipment, which can realize early detection and prediction of common diseases and sudden diseases in people with sub-health and latent diseases, but the data collected by the equipment cannot be transmitted in real time, so it cannot realize the function of remote real-time monitoring and can only be used as a fixed medical device in the hospital. For example, the LifeShirt produced by Vivo Metrics in the United States.

[0007] The defects of wearable monitoring devices are also obvious: 1. Limited accuracy: Due to the size and design limitations of the sensor, the data accuracy may be slightly inferior to that of professional equipment.

[0008] 2. Battery life: Frequent monitoring and data transmission may result in limited device battery life.

[0009] 3. Dependence on user behavior: Deviations in wearing method and position may affect the accuracy of the data.

[0010] The third type is portable monitoring equipment, such as the Bioharness portable physiological parameter measurement system of Zephyr in the United States, which embeds sensors into a chest strap. The usage scenarios have been extended from being operated by professional physicians in hospital departments to being used by individuals at home.

[0011] The drawbacks of portable monitoring devices are also obvious: 1. Inconvenient to use: Although portable, it still requires manual operation and may not be as convenient to use as wearable devices.

[0012] 2. Skin contact is required: Some people with sensitive skin may feel severe discomfort.

[0013] 3. Professional operation is required: Some equipment may require certain professional knowledge or training to use correctly. Summary of the invention

[0014] In order to address and solve the above problems, the present invention provides a non-contact body physiological signal acquisition system and method.

[0015] To achieve the above purpose, the technical solution provided by the present invention is as follows: A non-contact body physiological signal acquisition system, comprising: a. The facial age and gender recognition subsystem is used to enhance the intelligence and personalized service capabilities of the non-contact physiological parameter acquisition system. It uses a high-definition camera to capture the user's facial image and automatically identify the individual's gender, male or female, and the age group to which he or she belongs from the facial image; The face age and gender recognition function subsystem includes: data preprocessing module: during image preprocessing, the input image OrigianlImage is converted into a 4D tensor that can be processed by the deep learning model; b. A non-contact remote photoplethysmography functional subsystem for extracting rPPG signals based on facial videos and optimizing rPPG estimation from multiple enhanced videos of different signal frequencies and temporally adjacent videos of similar signal frequencies; the non-contact remote photoplethysmography functional subsystem includes: a video input and preprocessing module, a time segmentation module, a data enhancement module, a signal extraction module, a model optimization module, and a total loss function calculation module; The video input and preprocessing module is used to detect, align and crop the face area in each frame using the MTCNN face detector given a facial video sequence. Time segmentation module: used to segment the aligned video V into multiple segments, each segment contains T frames; data enhancement module: used for spatial enhancement: Apply spatial enhancement, that is, rotate the image by 0°, 90°, 180° and 270°, and flip it horizontally and vertically to obtain multiple positive sample sets. ; Data augmentation processing randomly selects a video segment from the facial video as input, while retaining other segments as neighboring temporal videos of the input; spatial augmentation and learnable frequency augmentation LFA are applied on the input to generate positive and negative video samples respectively; Signal extraction module: used to extract positive and negative samples and They are respectively input into the local rPPG expert aggregation REA module to estimate the corresponding rPPG signal; a local rPPG expert aggregation REA module is designed to extract rPPG signals from positive and negative video samples; Model optimization module: for frequency contrast loss: Between signals in or and Apply frequency contrast loss between them; optimize the rPPG signals generated by the input positive and negative sample videos and the temporal neighborhood videos; Total loss function calculation module: used to calculate the total loss ,The model framework can learn rPPG signals from unlabeled facial videos; c. rPPG-based heart rate variability and respiratory rate monitoring subsystem: used to filter and denoise the obtained rPPG signal, apply a bandpass filter to remove noise and retain the frequency components related to the heart rate, and analyze the power spectral density of the rPPG signal through fast Fourier transform to obtain the maximum peak frequency, that is, the heart rate; The rPPG-based heart rate variability and respiratory rate monitoring subsystem includes: rPPG signal acquisition and filtering module: used to extract the rPPG signal of the face area from the video frame and predict it through the model. The obtained rPPG signal is then processed by a bandpass filter to remove low-frequency and high-frequency noise; Heart rate detection module: used to find the maximum peak frequency in the power spectrum density, and the corresponding frequency is the heart rate; Frequency feature calculation module: used to calculate the frequency feature through the Lomb-Scargle periodogram and extract the low-frequency LF and high-frequency HF power from the IBI sequence; Respiratory rate RR monitoring module: expressed by the high-frequency peak frequency of heart rate variability.

[0016] A non-contact body physiological signal acquisition method comprises the following steps: Step A: A method for identifying the age and gender of a face, using a high-definition camera to capture a user's facial image; automatically identifying the gender of the individual, male or female, and the age group to which the individual belongs from the facial image; The steps include: Step A-1, data preprocessing: During image preprocessing, the input image OrigianlImage is converted into a 4D tensor that can be processed by the deep learning model through a series of steps; these steps include adjusting the image size, performing mean subtraction, RGB channel swapping, and normalization; Step A-2, resize: resize the input image to the fixed size of 256x256 pixels accepted by the model to ensure that the input image size is consistent to meet the structural requirements of the model; Step A-3, mean subtraction: Each pixel value is subtracted from a predefined mean to reduce the impact of illumination changes on the model. The predefined mean of the three BGR channels is [104, 117, 123]: ;

[0017] Where c is the channel index, h and w are the height and width coordinates of the image respectively; RGB channel exchange: The RGB channels of the image need to be exchanged for BGR channels, because some models are trained based on the BGR format; Step A-4, normalization: Normalize the pixel value range of the image to [0, 1], that is, divide each pixel value by 255; normalization can make the model easier to train and reason, and reduce the fluctuation of the value range; ;

[0018] Step A-5, face detection: The preprocessed image is input into the face detection model FaceNet to obtain the face detection result, and the face boxes FaceBoxes are screened out according to the confidence threshold of 0.7; Step A-6, model forward propagation: ; ;

[0019] To train the face detection model FaceNet, a triplet loss function is used. A triplet consists of an anchor image, a positive sample image, and a negative sample image; the goal is to ensure that the distance between the anchor image and the positive sample image is closer than the distance to the negative sample image, and the difference is at least greater than a predefined margin. ; The embedding vector of the anchor image, is the embedding vector of the positive sample image, is the embedding vector of the negative sample image; when the image is input into the face detection model FaceNet, the model will process the image and output the detection result. detections is the detection result output by the model, which is a multidimensional array containing the confidence, position and size information of each detected candidate box; Step A-7, confidence calculation: ; In the detection results, the confidence value indicates how sure the model is that a certain area contains a face. The higher the confidence, the greater the possibility that the area is a face. The confidence value is a value in the detection result, which is located in the i-th detection result in the detections array. Detection box Bounding Box coordinate calculation: ; ; ; ; The coordinates of the detection box are the proportional values ​​output by the model, which need to be multiplied by the width and height of the original image to be converted to actual pixel coordinates. Among them, x1, y1 are the coordinates of the upper left corner of the detection box, and x2, y2 are the coordinates of the lower right corner of the detection box. frameWidth and frameHeight are the width and height of the input image, which are used to convert the relative coordinates of the detection box to absolute pixel coordinates; Step A-8, result filtering: The test result is considered valid only when the confidence exceeds the set threshold of 0.7; Step A-9, Gender Detection: The gender detection model GenderNet is a deep learning model for gender classification. It receives a face image as input and predicts the gender of the person in the image as male or female; Step A-10, Model Architecture: The gender detection model GenderNet is based on a deep convolutional neural network architecture, which includes multiple convolutional layers, pooling layers, and fully connected layers, which gradually extract high-level features of the image. The last layer is the Softmax layer, which outputs two nodes, corresponding to male and female. The Softmax function converts the output of the network into probability values ​​of two categories; by selecting the category corresponding to the node with a larger probability value, the model determines whether the gender in the image is male or female; that is: ; in Yes Category , i.e., the score for males or females; refers to the scores of all possible categories, i.e., males and females, i.e., the sum of the scores of all categories.

[0020] Step A-11, Age Detection: The age detection model AgeNet is also based on a deep convolutional neural network. The last layer of the model, the Softmax layer, outputs a vector containing 8 nodes, each node corresponds to an age group, namely 0-3 years old, 4-77 years old, 8-14 years old, 15-24 years old, 25-37 years old, 38-47 years old, 48-53 years old, 54-100 years old, and each interval represents a classification output; ;

[0021] in The output of the model is The scores of each age group; Refers to the scores of all possible categories (8 categories), that is, the sum of the scores of all categories.

[0022] The following steps are also included: Step B, non-contact remote photoplethysmography measurement: using a frequency-dependent deep convolutional neural network (DNN) model for remote photoplethysmography rPPG signal extraction based on facial videos, the model learns to optimize rPPG estimation from multiple enhanced videos of different signal frequencies and temporally adjacent videos of similar signal frequencies; specifically including: Step B-1, video input and preprocessing: Given a facial video sequence, first use the MTCNN face detector to detect, align and crop the face region in each frame. The aligned video is represented as: ; in represents the Tth frame in the sequence; Step B-2, time segmentation: The aligned video V is segmented into multiple segments, each segment contains T frames. The segment set is represented as ,in Represents the i-th segment in the sequence. A segment is randomly selected as the main input , the remaining fragments are regarded as temporal neighbors and are denoted as ; Step B-3, data enhancement: spatial enhancement: Apply spatial enhancement, that is, rotate the image by 0°, 90°, 180° and 270°, and flip it horizontally and vertically to obtain multiple positive sample sets. , spatial enhancement does not affect the intrinsic rPPG signal in the sample, that is: ; Step B-4, Learnable Frequency Enhancement LFA: The LFA module modulates The rPPG signal in the negative sample set is generated , the rPPG signal frequency of the negative sample is different: ; in is the frequency ratio applied by the LFA module; Step B-5, signal extraction: positive and negative samples and They are respectively input into the local rPPG expert aggregation REA module to estimate the corresponding rPPG signal: ; in, and ; Step B-6, Model Optimization: Frequency Contrast Loss: Between signals in or and Apply frequency contrast loss between: ; Step B-7, frequency ratio consistency loss: This loss is used to constrain and its temporal neighbors have similar rPPG signal frequencies, namely: ; Frequency consistency loss across videos: ; Step B-8, total loss calculation: ; in, , , is the weight factor; through the above stages, the model framework can learn rPPG signals from unlabeled facial videos.

[0023] The following steps are also included: Step C, realization of heart rate variability and respiratory rate monitoring function based on rPPG: This monitoring function is mainly realized by filtering and denoising the obtained remote photoplethysmography rPPG signal; a bandpass filter can be applied to remove noise and retain the frequency components related to the heart rate, and the power spectrum density of the rPPG signal is analyzed by fast Fourier transform to obtain the maximum peak frequency, that is, the heart rate; heart rate variability includes three attributes, namely low frequency LF, high frequency HF and LF / HF ratio, and the three attribute indicators of HRV can be calculated by analyzing the heart beat interval IBI sequence, and the respiratory frequency is related to LF; Specifically include: Step C-1, acquisition and filtering of rPPG signals: extract the rPPG signal of the face area from the video frame and predict it through the model; the obtained rPPG signal is then processed by a bandpass filter to remove low-frequency and high-frequency noise; the frequency range of the bandpass filter is set to 0.6 Hz to 4 Hz. The filtered rPPG signal is expressed as: ; Step C-2, heart rate calculation: The power spectrum density PSD of the rPPG signal is analyzed based on the fast Fourier transform FFT; the specific calculation is as follows: Fourier transform: Calculate the FFT of the rPPG signal to obtain the frequency domain signal ; Represents the Fourier transform operation, which is a mathematical method for converting a time domain signal into a frequency domain signal. It converts a signal from a time-varying representation (time domain) to a representation of frequency components (frequency domain).

[0024] Power spectral density PSD: Calculate the power spectral density of a frequency domain signal: ; Step C-3, heart rate detection: Find the maximum peak frequency in the power spectrum density, and the corresponding frequency is the heart rate. The formula for calculating the heart rate is: ; Among them, HR is the heart rate, the unit is bpm, 0.67 Hz and 3.33 Hz correspond to 40 bpm and 200 bpm respectively, which are the commonly used heart rate frequency ranges; The frequency domain characteristics of heart rate variability (HRV) mainly include low-frequency (LF) and high-frequency (HF) power and their ratio; The IBI is the time interval between consecutive heart beat peaks. ; The calculation formula of IBI is: ; in, It is The location of the heart beat peak, It is and The interval between peaks; Step C-4, calculation of frequency characteristics: The frequency characteristics are calculated by Lomb-Scargle periodogram, and the low frequency LF and high frequency HF power are extracted from the IBI sequence; the specific steps are as follows: Step C-4i, spectrum estimation: Spectrum analysis of the IBI sequence is performed using the Lomb-Scargle method to obtain the frequency and power spectrum ; ; Step C-4ii, Low and High Frequency Power: Select the power in the low frequency 0.04 Hz - 0.15 Hz and high frequency 0.15 Hz - 0.4 Hz regions within the frequency range.

[0025] ; ; Step C-4iii, spectrum peak frequency: Find the frequency peak corresponding to the highest power in the low-frequency and high-frequency regions respectively: ; ; Step C-4iiiii, frequency domain features: The low frequency peak frequency is , reflects the dominant frequency of the heart rate variability signal in the low-frequency range; The high frequency peak frequency is , reflects the main frequency of the heart rate variability signal in the high frequency range; The low frequency power is , low-frequency power is usually used to measure the overall activity level of the sympathetic and parasympathetic nervous systems; The high frequency power is , which is often used to measure the activity of the parasympathetic nervous system; Normalize the low-frequency power and high-frequency power: ; ; Step C-5, respiratory rate RR monitoring: The respiratory rate can be represented by the high-frequency peak frequency of the heart rate variability.

[0026] The present invention has the following beneficial effects at the application level: The present invention overcomes the defects of the prior art. Compared with traditional health monitoring methods, non-contact body physiological signal monitoring does not require wearable equipment or contact with the skin, and can also achieve accurate detection. This technology can effectively avoid problems such as cross-infection, while improving the convenience and comfort of health monitoring. It can achieve a high degree of automation, making the collection, processing and analysis of monitoring data more convenient and efficient. By using technologies such as big data analysis and artificial intelligence, various health indicators of the body can be accurately identified, and more comprehensive health analysis and risk assessment services can be provided. It also has the advantage of real-time monitoring, which can better monitor the progression of the disease and the treatment effect, and can also help people predict and prevent health problems in a timely manner. At the same time, this technology is not only suitable for personal use, but also has broad application prospects, such as hospitals, highways, sports events and other scenarios.

[0027] In addition, in terms of scalability and compatibility, the present invention can be seamlessly connected with other health management systems or smart devices to form a complete health monitoring ecosystem. At the same time, it can be flexibly adjusted according to specific application scenarios and is suitable for different groups of people, such as the elderly, patients with chronic diseases or high-intensity athletes. In terms of energy saving and environmental protection, due to its non-contact design, it does not require a large amount of energy consumption, reduces the need for battery use and replacement, has the characteristics of low power consumption, and is in line with the concept of green environmental protection. In terms of efficiency and multi-tasking, the present invention can not only monitor multiple physiological parameters in real time, but also perform data analysis and risk assessment at the same time, saving user time and improving the efficiency of diagnosis and treatment. At the same time, in terms of cost, compared with traditional vital sign monitoring equipment, the present invention reduces hardware costs, and the use of big data and artificial intelligence technology also reduces long-term operation and maintenance costs, and has higher economy.

[0028] The present invention has the following beneficial effects at the technical level: 1) Multi-ROI region extraction improves robustness: Based on the detection of rPPG signals, the present invention selects multiple regions of interest (ROIs) on the skin for signal extraction, which can effectively reduce the error caused by environmental factors (such as uneven lighting, occlusion, etc.) in a single ROI. Through multi-region signal fusion processing, the interference of artifacts and noise is reduced, and the robustness of heart rate and other physiological parameter detection is significantly improved.

[0029] 2) Signal enhancement and noise suppression: After extracting rPPG signals in multiple ROIs, the credibility of the signal is further enhanced by averaging processing, while eliminating noise caused by local skin movement, micro-expression changes, light reflection, etc. This algorithm design makes the final extracted physiological signal more stable and less error-prone, ensuring high-precision measurement in different usage scenarios.

[0030] 3) Combination of time domain and frequency domain analysis: This algorithm not only relies on time domain information in the signal processing stage, but also analyzes the frequency domain information through Fourier transform to accurately extract the frequency characteristics of the target physiological signal, such as heart rate, etc. This dual analysis method improves the accuracy of signal extraction, especially in motion and non-stationary scenes, it can still maintain high detection performance.

[0031] 4) Multi-frame fusion enhances signal quality: By fusing and calculating the signals of multiple time frames, the stability and accuracy of the rPPG signal are further improved. Even if the signal may fluctuate slightly in a short period of time, the fusion of multiple frames can effectively smooth out these fluctuations and ensure the consistency and continuity of the output signal.

[0032] 5) Real-time and parallel computing optimization: The present invention adopts parallel processing technology, which can reduce the computing time while maintaining high computing accuracy when performing multiple ROI signal extraction and fusion calculation, thereby realizing real-time monitoring of physiological signals. This is of great significance for application scenarios that require continuous monitoring, such as hospitals or sports venues.

[0033] 6) Algorithm adaptability and scene optimization: The algorithm has scene adaptability and can automatically adjust signal extraction parameters according to different lighting conditions and skin characteristics (such as color, texture, etc.), ensuring that reliable rPPG signals can be obtained in a variety of environments. Through this adaptive algorithm design, the system can be widely used in different populations and scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the system framework of the present invention.

[0035] Figure 2 It is a schematic diagram of the model framework of the present invention.

[0036] Figure 3 It is a schematic diagram of heart rate HR, heart rate variability (high frequency peak frequency, low frequency peak frequency, high frequency power, high frequency power) and breathing frequency based on rPPG wave extraction of the present invention. DETAILED DESCRIPTION

[0037] According to the attached Figure 1-3 And embodiments are used to illustrate the specific implementation of the present invention: Embodiment 1

[0038] A non-contact body physiological signal acquisition system, characterized by comprising: a. The facial age and gender recognition subsystem is used to enhance the intelligence and personalized service capabilities of the non-contact physiological parameter acquisition system. It uses a high-definition camera to capture the user's facial image and automatically identify the individual's gender, male or female, and the age group to which they belong from the facial image; The face age and gender recognition function subsystem includes: data preprocessing module: during image preprocessing, the input image OrigianlImage is converted into a 4D tensor that can be processed by the deep learning model; b. A non-contact remote photoplethysmography functional subsystem for extracting rPPG signals based on facial videos and optimizing rPPG estimation from multiple enhanced videos of different signal frequencies and temporally adjacent videos of similar signal frequencies; the non-contact remote photoplethysmography functional subsystem includes: a video input and preprocessing module, a time segmentation module, a data enhancement module, a signal extraction module, a model optimization module, and a total loss function calculation module; The video input and preprocessing module is used to detect, align and crop the face area in each frame using the MTCNN face detector given a facial video sequence. Time segmentation module: used to segment the aligned video V into multiple segments, each segment contains T frames; data enhancement module: used for spatial enhancement: Apply spatial enhancement, that is, rotate the image by 0°, 90°, 180° and 270°, and flip it horizontally and vertically to obtain multiple positive sample sets. ; Data augmentation processing randomly selects a video segment from the facial video as input, while retaining other segments as neighboring temporal videos of the input; spatial augmentation and learnable frequency augmentation LFA are applied on the input to generate positive and negative video samples respectively; Signal extraction module: used to extract positive and negative samples and They are respectively input into the local rPPG expert aggregation REA module to estimate the corresponding rPPG signal; a local rPPG expert aggregation REA module is designed to extract rPPG signals from positive and negative video samples; Model optimization module: for frequency contrast loss: Between signals in or and Apply frequency contrast loss between them; optimize the rPPG signals generated by the input positive and negative sample videos and the temporal neighborhood videos; Total loss function calculation module: used to calculate the total loss ,The model framework can learn rPPG signals from unlabeled facial videos; c. rPPG-based heart rate variability and respiratory rate monitoring subsystem: used to filter and denoise the obtained rPPG signal, apply a bandpass filter to remove noise and retain the frequency components related to the heart rate, and analyze the power spectral density of the rPPG signal through fast Fourier transform to obtain the maximum peak frequency, that is, the heart rate; The rPPG-based heart rate variability and respiratory rate monitoring subsystem includes: rPPG signal acquisition and filtering module: used to extract the rPPG signal of the face area from the video frame and predict it through the model. The obtained rPPG signal is then processed by a bandpass filter to remove low-frequency and high-frequency noise; Heart rate detection module: used to find the maximum peak frequency in the power spectrum density, and the corresponding frequency is the heart rate; Frequency feature calculation module: used to calculate the frequency feature through the Lomb-Scargle periodogram and extract the low-frequency LF and high-frequency HF power from the IBI sequence; Respiratory rate RR monitoring module: expressed by the high-frequency peak frequency of heart rate variability. Embodiment 2

[0039] 1. Implementation of face age and gender recognition function: 1.1 Brief Introduction The face age and gender recognition function can enhance the intelligence and personalized service capabilities of the non-contact physiological parameter collection system. Compared with traditional physiological monitoring equipment, non-contact equipment with face recognition function can provide more accurate health monitoring by combining personal information without the user's awareness.

[0040] The main implementation methods are: using a high-definition camera to capture the user's facial image, and secondly, automatically identifying the individual's gender ("male" or "female") and age group (such as 0-2 years old, 4-6 years old, etc.) from the facial image.

[0041] 1.2 Technical implementation plan: Data preprocessing: During image preprocessing, the input image (OrigianlImage) needs to go through a series of steps to be converted into a 4D tensor that can be processed by the deep learning model. These steps include adjusting the image size, performing mean subtraction, RGB channel swapping, and normalization.

[0042] Resizing: Resize the input image to a fixed size of 256x256 pixels accepted by the model to ensure that the input image size is consistent to fit the structural requirements of the model.

[0043] Mean subtraction: Each pixel value is subtracted from a predefined mean value to reduce the impact of illumination changes on the model. The predefined mean of the three BGR channels is [104, 117, 123]: ; Where c is the channel index, h and w are the height and width coordinates of the image, respectively.

[0044] RGB channel swap: The RGB channels of the image need to be swapped to BGR channels, because some models are trained based on the BGR format.

[0045] Normalization: Normalize the pixel value range of the image to [0, 1], that is, divide each pixel value by 255. Normalization makes the model easier to train and reason about, and reduces the fluctuation of the value range.

[0046] ; Face detection: The preprocessed image is input into the face detection model FaceNet to obtain the face detection result, and the face boxes FaceBoxes are screened out according to the confidence threshold of 0.7.

[0047] Model forward propagation: ; ; To train the face detection model FaceNet, a triplet loss function is used. A triplet consists of an anchor image, a positive sample image, and a negative sample image. The goal is to ensure that the distance between the anchor image and the positive sample image is closer than the distance to the negative sample image, and the difference is at least greater than a predefined margin. . The embedding vector of the anchor image, is the embedding vector of the positive sample image, is the embedding vector of the negative sample image. When the image is input into the face detection model FaceNet, the model processes the image and outputs the detection result. detections is the detection result output by the model, which is a multidimensional array containing the confidence, position and size information of each detected candidate box.

[0048] Confidence calculation: ; In the detection results, the confidence value indicates how sure the model is that a certain area contains a face. The higher the confidence, the more likely the area is a face. Confidence is a value in the detection result, which is located in the i-th detection result in the detections array.

[0049] Detection box Bounding Box coordinate calculation: ; ; ; ; The coordinates of the detection box are the proportional values ​​output by the model, which need to be multiplied by the width and height of the original image to be converted to actual pixel coordinates. Among them, x1, y1 are the coordinates of the upper left corner of the detection box, and x2, y2 are the coordinates of the lower right corner of the detection box. frameWidth and frameHeight are the width and height of the input image, which are used to convert the relative coordinates of the detection box to absolute pixel coordinates.

[0050] Result filtering: The detection result is considered valid only when the confidence exceeds the set threshold of 0.7.

[0051] Gender Detection: GenderNet is a deep learning model for gender classification. It receives a face image as input and predicts the gender of the person in the image as male or female.

[0052] Model architecture: The gender detection model GenderNet is based on a deep convolutional neural network architecture, which contains multiple convolutional layers, pooling layers, and fully connected layers to gradually extract high-level features of the image. The last layer is the Softmax layer, which outputs two nodes, corresponding to male and female. The Softmax function converts the output of the network into probability values ​​of two categories. By selecting the category corresponding to the node with a larger probability value, the model determines whether the gender in the image is "male" or "female". That is: ; in Yes Category score of (male or female); Refers to the scores of all possible categories (male and female), that is, the sum of the scores of all categories.

[0053] Age detection: The age detection model AgeNet is also based on a deep convolutional neural network. The last layer of the model, the Softmax layer, outputs a vector containing 8 nodes, each node corresponds to an age range (such as 0-3 years old, 4-7 years old, 8-14 years old, 15-24 years old, 25-37 years old, 38-47 years old, 48-53 years old, 54-100 years old). Each interval represents a classification output. ; in The output of the model is The scores of each age group; Refers to the scores of all possible categories (8 categories), that is, the sum of the scores of all categories.

[0054] The final input and output model of the face age and gender recognition function is as follows Figure 1 As shown: Figure 1The system framework of the present invention consists of three parts: a face detection model FaceNet for identifying face areas, a gender detection model GenderNet for identifying gender, and an age detection model AgeNet for identifying age.

[0055] 2. Non-contact measurement method - remote photoelectric volume pulse wave function realization: 2.1 Brief Introduction Remote photoplethysmography (rPPG) is a technology that uses optical sensors to detect changes in blood flow on the skin surface. It is often used to non-invasively measure physiological parameters such as heart rate and blood pressure. rPPG extracts pulse wave information related to cardiac activity by analyzing the light signal reflected by the skin of the face. We designed a novel frequency-dependent deep convolutional neural network (DNN) model for rPPG signal extraction based on facial videos. The model learns to optimize rPPG estimation from multiple enhanced videos of different signal frequencies and temporally adjacent videos of similar signal frequencies. It has six main stages: video input and preprocessing, temporal segmentation, data augmentation, signal extraction, model optimization, and total loss function calculation.

[0056] 2.2 Technical Implementation Solution Video input and preprocessing: Given a facial video sequence, we first use the MTCNN face detector to detect, align, and crop the face region in each frame. The aligned video is represented as: ; in Represents the Tth frame in the sequence.

[0057] Time segmentation: Align the video is divided into multiple segments, each segment contains T frames. The segment set is represented as ,in Represents the i-th segment in the sequence. A segment is randomly selected as the main input , the remaining fragments are regarded as temporal neighbors and are denoted as ; Data enhancement: spatial enhancement: Apply spatial enhancement, that is, rotate the image by 0°, 90°, 180° and 270°, and flip it horizontally and vertically to obtain multiple positive sample sets. , spatial enhancement does not affect the intrinsic rPPG signal in the sample, that is: ; Learnable frequency enhancement LFA: The LFA module modulates The rPPG signal in the negative sample set is generated , the rPPG signal frequency of the negative sample is different: ; in is the frequency ratio applied by the LFA module.

[0058] Signal extraction: positive and negative samples and They are respectively input into the local rPPG expert aggregation RE module to estimate the corresponding rPPG signal: ; in, and .

[0059] Model optimization: frequency contrast loss: Between signals in or and Apply frequency contrast loss between: ; Frequency ratio consistency loss: This loss is used to constrain and its temporal neighbors have similar rPPG signal frequencies, namely: ; Frequency consistency loss across videos: ; Total loss calculation: ; in, , , is the weight factor.

[0060] Through the above stages, the model framework can learn rPPG signals from unlabeled facial videos. The model overview is as follows: Figure 2 shown.

[0061] Figure 2 The model framework of the present invention includes three main stages. 1) Data augmentation: A video segment is randomly selected from the facial video as input, while other segments are retained as the neighboring temporal videos of the input. Spatial augmentation and learnable frequency augmentation LFA are applied on the input to generate positive and negative video samples respectively. 2) Signal extraction: A local rPPG expert aggregation REA module is designed to extract rPPG signals from positive and negative video samples. 3) Model optimization: The rPPG signals generated by the input positive and negative sample videos and the temporal neighbor videos are optimized.

[0062] 3. Implementation of heart rate, heart rate variability and respiratory rate monitoring functions based on rPPG 3.1 Brief Introduction This monitoring function is mainly achieved by filtering and denoising the obtained rPPG signal. A bandpass filter (usually between 0.5Hz and 4Hz) can be applied to remove noise and retain the frequency components related to the heart rate, and the power spectral density of the rPPG signal is analyzed by fast Fourier transform to obtain the maximum peak frequency, that is, the heart rate. Heart rate variability includes three attributes, namely low frequency (LF), high frequency HF and LF / HF ratio. The three attribute indicators of HRV can be calculated by analyzing the interbeat interval (IBI) sequence, and the respiratory rate is related to LF.

[0063] 3.2 Technical Implementation Solution Acquisition and filtering of rPPG signals: The rPPG signal of the face area is extracted from the video frame and predicted by the model. The obtained rPPG signal is then processed by a bandpass filter to remove low-frequency and high-frequency noise. The frequency range of the bandpass filter is set to 0.6 Hz to 4 Hz. The filtered rPPG signal is expressed as: ; The calculation of heart rate is based on the analysis of the power spectral density (PSD) of the rPPG signal using the Fast Fourier Transform (FFT). The specific calculation is as follows: Fourier transform: Calculate the FFT of the rPPG signal to obtain the frequency domain signal ; Represents the Fourier transform operation, which is a mathematical method for converting a time domain signal into a frequency domain signal. It converts a signal from a time-varying representation (time domain) to a representation of frequency components (frequency domain).

[0064] Power Spectral Density (PSD): Calculates the power spectral density of a frequency domain signal ; Heart rate detection: Find the maximum peak frequency in the power spectrum density, and the corresponding frequency is the heart rate. The formula for calculating the heart rate is: ; Among them, HR is the heart rate in bpm (beats per minute). 0.67 Hz and 3.33 Hz correspond to 40 bpm and 200 bpm respectively, which are the commonly used heart rate frequency ranges.

[0065] The frequency domain characteristics of heart rate variability (HRV) mainly include low frequency (LF) and high frequency HF power and their ratio.

[0066] Interbeat interval (IBI) is the time interval between consecutive heart beat peaks. ;

[0067] The calculation formula of IBI is: ; in, It is The location of the heart beat peak, It is and The interval between peaks.

[0068] Calculation of frequency characteristics: The frequency characteristics are calculated by Lomb-Scargle periodogram, and low frequency (LF) and high frequency HF power are extracted from the IBI sequence. The specific steps are as follows: i. Spectrum estimation: The Lomb-Scargle method is used to analyze the spectrum of the IBI sequence and obtain the frequency and power spectrum ; ; ii. Low and High Frequency Power: Select the power in the low frequency (0.04 Hz - 0.15 Hz) and high frequency (0.15Hz - 0.4 Hz) regions within the frequency range.

[0069] ; ; iii. Spectrum peak frequency: Find the frequency peak corresponding to the highest power in the low-frequency and high-frequency regions respectively.

[0070] ; ; iv. Frequency domain characteristics: The low frequency peak frequency is , reflects the dominant frequency of the heart rate variability signal in the low frequency range. It is usually related to the activity of the sympathetic and parasympathetic nervous systems and is an indicator of heart rate regulation.

[0071] The high frequency peak frequency is , reflects the main frequency of the heart rate variability signal in the high frequency range, which is usually related to the activity of the parasympathetic nervous system (especially the vagus nerve). The frequency corresponding to the high frequency peak frequency usually matches the respiratory rate.

[0072] The low frequency power is , low-frequency power is often used to measure the overall activity level of the sympathetic and parasympathetic nervous systems. Higher low-frequency power indicates increased activity in the sympathetic nervous system, which is often associated with blood pressure fluctuations and stress responses.

[0073] The high frequency power is , which is often used to measure the activity of the parasympathetic nervous system. Higher high-frequency power indicates a stronger influence of the parasympathetic nervous system on cardiac activity and is often associated with a state of rest or relaxation.

[0074] In order to reduce the influence of individual differences and signal strength, highlight the relative activity of sympathetic and parasympathetic nerves, and eliminate the influence of environment and measurement conditions, low-frequency power and high-frequency power are usually normalized: ; ; The low frequency / high frequency ratio (LF / HF) is an important indicator in the frequency domain analysis of heart rate variability. This ratio is often used to assess the relative balance between sympathetic and parasympathetic activity in the autonomic nervous system. Usually, a low LF / HF ratio (usually < 1) can be interpreted as the dominance of the parasympathetic nervous system, which is usually associated with a state of relaxation, quiet rest, or recovery. A moderate LF / HF ratio (usually between 1 and 3) can be interpreted as a more balanced balance between the sympathetic and parasympathetic nervous systems. This is generally considered a "normal" or healthy state of autonomic balance. A high LF / HF ratio (usually > 3) can be interpreted as the sympathetic nervous system may be dominant, which may be associated with stress, anxiety, physical activity, or sympathetic nervous system hyperactivity.

[0075] Respiratory rate RR monitoring: Because respiratory activity can directly affect the rhythm of the heart, this phenomenon is called respiratory sinus arrhythmia (RSA). At different stages of the respiratory cycle, the heart rate will naturally fluctuate, and this fluctuation is concentrated in the high-frequency HF range of the HRV spectrum. Therefore, the respiratory rate can be represented by the high-frequency peak frequency of the heart rate variability.

[0076] Based on this, the flowchart of the technical implementation is as follows Figure 3 As shown: Figure 3 It is the heart rate HR, heart rate variability (high frequency peak frequency, low frequency peak frequency, high frequency power, high frequency power) and breathing frequency extracted based on rPPG waves of the present invention.

[0077] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only preferred examples of the invention and are not intended to limit the present invention. Without departing from the novel spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A non-contact body physiological signal acquisition system, characterized in that include: a. The facial age and gender recognition subsystem is used to enhance the intelligence and personalized service capabilities of the non-contact physiological parameter acquisition system. It uses a high-definition camera to capture the user's facial image and automatically identify the individual's gender, male or female, and the age group to which he or she belongs from the facial image; The face age and gender recognition function subsystem includes: data preprocessing module: during image preprocessing, the input image OrigianlImage is converted into a 4D tensor that can be processed by the deep learning model; b. A non-contact remote photoplethysmography functional subsystem for extracting rPPG signals based on facial videos and optimizing rPPG estimation from multiple enhanced videos of different signal frequencies and temporally adjacent videos of similar signal frequencies; the non-contact remote photoplethysmography functional subsystem includes: a video input and preprocessing module, a time segmentation module, a data enhancement module, a signal extraction module, a model optimization module, and a total loss function calculation module; The video input and preprocessing module is used to detect, align and crop the face area in each frame using the MTCNN face detector given a facial video sequence. Time segmentation module: used to segment the aligned video V into multiple segments, each segment contains Frame; data enhancement module: used for spatial enhancement: Apply spatial enhancement, that is, rotate the image by 0°, 90°, 180° and 270°, and flip it horizontally and vertically to obtain multiple positive sample sets. ; Data augmentation processing randomly selects a video segment from the facial video as input, while retaining other segments as neighboring temporal videos of the input; spatial augmentation and learnable frequency augmentation LFA are applied on the input to generate positive and negative video samples respectively; Signal extraction module: used to extract positive and negative samples and They are respectively input into the local rPPG expert aggregation REA module to estimate the corresponding rPPG signal; a local rPPG expert aggregation REA module is designed to extract rPPG signals from positive and negative video samples; Model optimization module: for frequency contrast loss: Between signals in or and Apply frequency contrast loss between them; optimize the rPPG signals generated by inputting positive and negative sample videos and temporal neighborhood videos; Total loss function calculation module: used to calculate the total loss , the model framework can learn rPPG signals from unlabeled facial videos; c. rPPG-based heart rate variability and respiratory rate monitoring subsystem: used to filter and denoise the obtained rPPG signal, apply a bandpass filter to remove noise and retain the frequency components related to the heart rate, and analyze the power spectrum density of the rPPG signal through fast Fourier transform to obtain the maximum peak frequency, that is, the heart rate; The rPPG-based heart rate variability and respiratory rate monitoring subsystem includes: rPPG signal acquisition and filtering module: used to extract the rPPG signal of the face area from the video frame and predict it through the model. The obtained rPPG signal is then processed by a bandpass filter to remove low-frequency and high-frequency noise; Heart rate detection module: used to find the maximum peak frequency in the power spectrum density, and the corresponding frequency is the heart rate; Frequency feature calculation module: used to calculate the frequency feature through the Lomb-Scargle periodogram and extract the low-frequency LF and high-frequency HF power from the IBI sequence; Respiratory rate RR monitoring module: expressed by the high-frequency peak frequency of heart rate variability.

2. A non-contact body physiological signal acquisition method, characterized in that The steps include: Step A: A method for identifying the age and gender of a face, using a high-definition camera to capture a user's facial image; automatically identifying the gender of the individual, male or female, and the age group to which the individual belongs from the facial image; The steps include: Step A-1, data preprocessing: During image preprocessing, the input image OrigianlImage is converted into a 4D tensor that can be processed by the deep learning model through a series of steps; these steps include adjusting the image size, performing mean subtraction, RGB channel swapping, and normalization; Step A-2, resize: resize the input image to the fixed size of 256x256 pixels accepted by the model to ensure that the input image size is consistent to meet the structural requirements of the model; Step A-3, mean subtraction: A predefined mean is subtracted from each pixel value to reduce the impact of illumination changes on the model; the predefined mean of the three BGR channels is [104, 117, 123]: ； Where c is the channel index, h and w are the height and width coordinates of the image respectively; RGB channel exchange: The RGB channels of the image need to be exchanged for BGR channels, because some models are trained based on the BGR format; Step A-4, normalization: Normalize the pixel value range of the image to [0, 1], that is, divide each pixel value by 255; normalization can make the model easier to train and reason, and reduce the fluctuation of the value range; ； Step A-5, face detection: The preprocessed image is input into the face detection model FaceNet to obtain the face detection result, and the face boxes FaceBoxes are screened out according to the confidence threshold of 0.7; Step A-6, model forward propagation: ； ； To train the face detection model FaceNet, a triplet loss function is used; a triplet consists of an anchor image, a positive sample image, and a negative sample image; the goal is to ensure that the distance between the anchor image and the positive sample image is closer than the distance to the negative sample image, and the difference is at least greater than a predefined margin ; The embedding vector of the anchor image, is the embedding vector of the positive sample image, is the embedding vector of the negative sample image; when the image is input into the face detection model FaceNet, the model will process the image and output the detection result; detections is the detection result output by the model, which is a multidimensional array containing the confidence, position and size information of each detected candidate box; Step A-7, confidence calculation: ； In the detection results, the confidence value indicates the model's degree of certainty that a certain area contains a face. The higher the confidence, the greater the possibility that the area is a face. Confidence is a value in the detection result, which is located in the i-th detection result of the detections array. Detection box Bounding Box coordinate calculation: ； ； ； ； The coordinates of the detection box are the proportional values ​​output by the model, which need to be multiplied by the width and height of the original image to be converted into actual pixel coordinates. x1, y1 are the coordinates of the upper left corner of the detection box, and x2, y2 are the coordinates of the lower right corner of the detection box. frameWidth and frameHeight are the width and height of the input image, which are used to convert the relative coordinates of the detection box into absolute pixel coordinates. Step A-8, result filtering: The test result is considered valid only when the confidence exceeds the set threshold of 0.7; Step A-9, Gender Detection: The gender detection model GenderNet is a deep learning model for gender classification; it receives a face image as input and predicts the gender of the person in the image as male or female; Step A-10, Model Architecture: The gender detection model GenderNet is based on a deep convolutional neural network architecture, which includes multiple convolutional layers, pooling layers, and fully connected layers to gradually extract high-level features of the image. The last layer is the Softmax layer, which outputs two nodes, corresponding to male and female respectively. The Softmax function converts the output of the network into probability values ​​of two categories. By selecting the category corresponding to the node with a larger probability value, the model determines whether the gender in the image is male or female; that is: ； in Yes Category , i.e., the score for males or females; refers to the scores of all possible categories, i.e., males and females, i.e., the sum of the scores of all categories; Step A-11, age detection: The age detection model AgeNet is also based on a deep convolutional neural network. The last layer of the model, the Softmax layer, outputs a vector containing 8 nodes, each node corresponds to an age group, namely 0-3 years old, 4-7 years old, 8-14 years old, 15-24 years old, 25-37 years old, 38-47 years old, 48-53 years old, 54-100 years old, and each interval represents a classification output; that is, ； in The output of the model is The scores of each age group; Refers to all possible categories, that is, the scores of 8 categories, that is, the sum of the scores of all categories.

3. The non-contact body physiological signal acquisition method according to claim 2, characterized in that The following steps are also included: Step B, non-contact remote photoplethysmography measurement: using a frequency-dependent deep convolutional neural network (DNN) model for remote photoplethysmography rPPG signal extraction based on facial videos, the model learns to optimize rPPG estimation from multiple enhanced videos of different signal frequencies and temporally adjacent videos of similar signal frequencies; specifically including: Step B-1, video input and preprocessing: Given a facial video sequence, first use the MTCNN face detector to detect, align and crop the face region in each frame; the aligned video is represented as: ； in represents the Tth frame in the sequence; Step B-2, time segmentation: The aligned video V is segmented into multiple segments, each segment contains T frames; the segment set is represented as ,in represents the i-th segment in the sequence; a segment is randomly selected as the main input , the remaining fragments are regarded as temporal neighbors and are denoted as ; Step B-3, data enhancement: spatial enhancement: Apply spatial enhancement, that is, rotate the image by 0°, 90°, 180° and 270°, and flip it horizontally and vertically to obtain multiple positive sample sets. , spatial enhancement does not affect the intrinsic rPPG signal in the sample, that is: ； Step B-4, learnable frequency enhancement LF: The LFA module modulates The rPPG signal in the negative sample set is generated , the rPPG signal frequency of the negative sample is different: ； in is the frequency ratio applied by the LFA module; Step B-5, signal extraction: positive and negative samples and They are respectively input into the local rPPG expert aggregation REA module to estimate the corresponding rPPG signal: ； in, and ; Step B-6, Model Optimization: Frequency Contrast Loss: Between signals in or and Apply frequency contrast loss between: ； Step B-7, frequency ratio consistency loss: This loss is used to constrain and its temporal neighbors have similar rPPG signal frequencies, namely: ； Frequency consistency loss across videos: ； Step B-8, total loss calculation: ； in, , , is the weight factor; through the above stages, the model framework can learn rPPG signals from unlabeled facial videos.

4. The non-contact body physiological signal acquisition method according to claim 2 or 3, characterized in that The following steps are also included: Step C, implementation of heart rate variability and respiratory rate monitoring function based on rPPG: This monitoring function is mainly achieved by filtering and denoising the obtained remote photoplethysmography rPPG signal; a bandpass filter can be applied to remove noise and retain the frequency components related to the heart rate, and the power spectral density of the rPPG signal is analyzed by fast Fourier transform to obtain the maximum peak frequency, i.e., the heart rate; heart rate variability includes three attributes, i.e., low frequency LF, high frequency HF and LF / HF ratio, and the three attribute indicators of HRV can be calculated by analyzing the IBI sequence of cardiac intervals, and the respiratory rate is related to LF; Specifically include: Step C-1, acquisition and filtering of rPPG signals: extract the rPPG signal of the face area from the video frame and predict it through the model; the obtained rPPG signal is then processed by a bandpass filter to remove low-frequency and high-frequency noise; the frequency range of the bandpass filter is set to 0.6 Hz to 4 Hz; the filtered rPPG signal is expressed as: ； in, Use Butterworth filter to achieve bandpass filtering. The Butterworth filter is characterized by smooth passband response, no ripples, and can better maintain the signal shape than other filters; Step C-2, heart rate calculation: The power spectral density PSD of the rPPG signal is analyzed based on the fast Fourier transform FFT; the specific calculation is as follows: Fourier transform: Calculate the FFT of the rPPG signal to obtain the frequency domain signal ； Represents the Fourier transform operation, which is a mathematical method for converting a time domain signal into a frequency domain signal. It converts a signal from a time-varying representation (time domain) to a representation of frequency components (frequency domain); Power Spectral Density PSD: Calculates the power spectral density of a frequency domain signal ； Step C-3, heart rate detection: Find the maximum peak frequency in the power spectrum density, and the corresponding frequency is the heart rate; the calculation formula for the heart rate is: ； Among them, HR is the heart rate, the unit is bpm, 0.67 Hz and 3.33 Hz correspond to 40 bpm and 200 bpm respectively, which are the commonly used heart rate frequency ranges; The frequency domain characteristics of heart rate variability (HRV) mainly include low-frequency (LF) and high-frequency (HF) power and their ratio; The IBI is the time interval between consecutive heart beat peaks, usually at a known sampling rate. Calculated under the condition of; The calculation formula of IBI is: ； in, It is The location of the heart rate peak, It is and The interval between peaks; Step C-4, calculation of frequency characteristics: The frequency characteristics are calculated by Lomb-Scargle periodogram, and the low frequency LF and high frequency HF power are extracted from the IBI sequence; the specific steps are as follows: Step C-4i, spectrum estimation: Spectrum analysis of the IBI sequence is performed using the Lomb-Scargle method to obtain the frequency and power spectrum ; ； Step C-4ii, low frequency and high frequency power: select the power in the low frequency 0.04 Hz - 0.15 Hz and high frequency 0.15 Hz - 0.4 Hz regions within the frequency range; ； ； Step C-4iii, spectrum peak frequency: Find the frequency peak corresponding to the highest power in the low-frequency and high-frequency regions respectively: ； ； Step C-4iiiii, frequency domain features: The low frequency peak frequency is , reflects the dominant frequency of the heart rate variability signal in the low-frequency range; The high frequency peak frequency is , reflects the main frequency of the heart rate variability signal in the high frequency range; The low frequency power is , low-frequency power is usually used to measure the overall activity level of the sympathetic and parasympathetic nervous systems; The high frequency power is , which is often used to measure the activity of the parasympathetic nervous system; Normalize the low-frequency power and high-frequency power: ； ； Step C-5, respiratory rate RR monitoring: The respiratory rate can be represented by the high-frequency peak frequency of the heart rate variability.

Images