Health monitoring method, device, electronic device and storage medium

Abstract

The present invention relates to the field of artificial intelligence technology, and provides a health monitoring method, device, electronic device and storage medium, wherein the method comprises: performing face detection on a currently acquired video frame to obtain a face region image of the current frame; performing key point detection on the face region image to obtain face key points; determining a face local region from the face region image based on the face key points; aligning the face local region of the current frame with the face local region of the previous frame based on the target key points in the face region image, and applying the aligned face local region for health monitoring to obtain health indicators. The present invention can suppress the influence of too many interfering pixels on feature extraction by determining the face local region from the face region image based on the detected face key points, and can significantly alleviate the influence of pixel misalignment by aligning the face local region of the current frame with the previous frame, thereby improving the accuracy of health monitoring.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a health monitoring method, device, electronic device and storage medium. Background Art

[0002] Currently, AI-powered health monitoring technology is gaining widespread attention and application in fields such as security, smart healthcare, and intelligent cockpits. This technology, leveraging advanced algorithms and sensors, enables real-time monitoring and assessment of individual health status. Especially in specialized scenarios (such as the cockpit), facial video-based health monitoring can proactively reject non-living subjects and quickly and cost-effectively identify abnormal changes in the subject's heart rate, respiration, and other data, thereby safeguarding their lives.

[0003] Health monitoring methods based on facial videos primarily utilize deep learning neural networks to mine continuous frames of facial images and assess an individual's health status by predicting data such as heart rate and respiration. However, in scenarios where the subject cannot actively cooperate for extended periods of time (such as driving), this method suffers from large prediction deviations. This is because in non-cooperative scenarios, due to the continuous movement of the face, pixels in the same position may correspond to different facial regions in different frames. This large deviation in the position of the pixels before and after can seriously affect the prediction results of the neural network. Therefore, how to achieve accurate and stable health monitoring in non-cooperative scenarios has become an urgent problem to be solved in the current field of artificial intelligence health monitoring. Summary of the Invention

[0004] The present invention provides a health monitoring method, device, electronic device and storage medium to solve the defect that in non-cooperative health monitoring scenarios, facial movement causes large prediction deviation, which affects the health monitoring effect.

[0005] The present invention provides a health monitoring method, comprising:

[0006] Perform face detection on the currently captured video frame to obtain the face area image of the current frame;

[0007] Performing key point detection on the face area image to obtain face key points;

[0008] Determining a local face region from the face region image based on the face key points;

[0009] Based on the target key points in the face area image, the local face area of ​​the current frame is aligned with the local face area of ​​the previous frame, and the aligned local face area is used for health monitoring to obtain a health indicator.

[0010] According to a health monitoring method provided by the present invention, performing key point detection on the face region image to obtain face key points includes:

[0011] Performing liveness detection on the face region image based on a liveness detection model to obtain a detection result, wherein the liveness detection model is a classification network trained using live face images as positive samples and non-live false face images and / or background images as negative samples;

[0012] When the detection result indicates that there is a living body, key point detection is performed on the face area image to obtain face key points.

[0013] According to a health monitoring method provided by the present invention, determining a local face area from the face area image based on the face key points includes:

[0014] In a case where the face region image includes a plurality of faces, determining the face key points of the subject from the face key points based on the face identification of the subject;

[0015] Based on the facial key points of the detected object, a local facial area is determined from the facial area image.

[0016] According to a health monitoring method provided by the present invention, determining a local facial area from the facial area image based on the facial key points of the subject includes:

[0017] In a case where the face region image includes a left face region and a right face region of the subject, based on the confidences of the facial key points of the subject, respectively determining a mean confidence value of all facial key points in the left face region and a mean confidence value of all facial key points in the right face region;

[0018] A side area with a higher confidence mean value is selected from the left face area and the right face area as the local face area.

[0019] According to a health monitoring method provided by the present invention, the health monitoring of the aligned local face area is performed to obtain health indicators, including:

[0020] When the aligned local face region is a single-channel image, the aligned local face region is used for health monitoring based on a single-channel source network to obtain a health indicator;

[0021] In the case where the aligned local face region is a multi-channel image, the aligned local face region is used for health monitoring based on a multi-channel source network to obtain a health indicator.

[0022] According to a health monitoring method provided by the present invention, when the health indicator includes a heart rate indicator and / or a respiratory indicator, the health monitoring is performed on the aligned local face area to obtain the health indicator, including:

[0023] Applying the aligned local face area to perform health monitoring to obtain indicator data of the current frame, wherein the indicator data includes heart rate data and / or respiration data;

[0024] Denoising the indicator data based on a sparse matrix to obtain a denoised prediction value of the current frame, and storing the denoised prediction value of the current frame in a preset queue, wherein the denoised prediction value includes a heart rate prediction value and / or a respiration prediction value, and the sparse matrix is ​​initialized using the sequence number of the current frame as a parameter;

[0025] Drawing a heart rate waveform diagram based on the heart rate prediction value, and / or drawing a respiration waveform diagram based on the respiration prediction value;

[0026] In the case that the denoised prediction values ​​stored in the preset queue are full of cycles, the denoised prediction values ​​stored in the preset queue are comprehensively processed to obtain the health indicator.

[0027] According to a health monitoring method provided by the present invention, denoising the indicator data based on a sparse matrix to obtain a denoised prediction value of the current frame includes:

[0028] Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame;

[0029] Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set;

[0030] Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

[0031] According to a health monitoring method provided by the present invention, the comprehensive processing of the denoised prediction values ​​stored in the preset queue to obtain the health index includes:

[0032] Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values;

[0033] Performing power spectrum density calculation based on the processed predicted value to obtain a calculation result;

[0034] Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

[0035] The present invention also provides a health monitoring device, comprising:

[0036] A face detection unit is used to perform face detection on the currently acquired video frame to obtain a face area image of the current frame;

[0037] A key point detection unit, configured to perform key point detection on the face region image to obtain face key points;

[0038] A region determining unit, configured to determine a local face region from the face region image based on the face key points;

[0039] A health monitoring unit is used to align the local face area of ​​the current frame with the local face area of ​​the previous frame based on the target key points in the face area image, and apply the aligned local face area to perform health monitoring to obtain health indicators.

[0040] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the health monitoring method described above is implemented.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements any of the above-described health monitoring methods when executed by a processor.

[0042] The present invention also provides a computer program product, comprising a computer program, which implements any of the above-mentioned health monitoring methods when executed by a processor.

[0043] The health monitoring method, device, electronic device, and storage medium provided by the present invention accurately extract a face region image from the currently captured video frame through face detection technology, and further locate the key points of the face in the face region image using key point detection technology, thereby providing a solid foundation for subsequent health monitoring. Based on the detected face key points, the local face region is determined from the face region image, thereby suppressing the influence of excessive interfering pixels on feature extraction. The target key points in the face region image are used to align the local face region of the current frame with the local face region of the previous frame, which can significantly alleviate the influence of pixel misalignment. By performing health monitoring on the aligned local face region, the deep features of the face can be more accurately captured, thereby reducing the prediction bias caused by face movement and improving the accuracy and effectiveness of health monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 is a flow chart of the health monitoring method provided by the present invention;

[0046] Figure 2 It is a schematic diagram of the key point area provided by the present invention;

[0047] Figure 3 Schematic diagram of key point correction provided by the present invention;

[0048] Figure 4 1 is a flow chart of the non-cooperative end-to-end health monitoring method combined with face liveness detection provided by the present invention;

[0049] Figure 5 is a schematic diagram of the process of post-processing of heart rate and / or respiratory index data provided by the present invention;

[0050] Figure 6 is a schematic structural diagram of the health monitoring device provided by the present invention;

[0051] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0053] Current approaches to acquiring health data are largely based on active human interaction, and fall into two main categories: a) Attaching sensors or wearing instruments, such as blood pressure monitors or respirators, to the subject's chest or wrist to collect health data like heart rate, blood pressure, and respiration. Comprehensive data analysis then provides a glimpse into health status. b) Health monitoring technology based on facial video continuously captures facial frames and uses deep learning neural networks to mine deep features from pixel changes between previous and subsequent facial frames to predict health data like heart rate and respiration.

[0054] However, from the perspective of the data input source of health monitoring technology, although high-precision health data can be obtained by using sensors attached to the subject, or using equipment such as blood pressure monitors and respirators, its applicable scenarios are greatly limited, especially in scenarios where the subject cannot actively cooperate for a long time, such as driving scenarios, because it is not possible to require every driver to carry an instrument.

[0055] Health monitoring solutions based on facial videos also have certain drawbacks in scenarios where the subject cannot actively cooperate for long periods of time. In this solution, the core method is to use a deep learning neural network to extract deep features based on the changes in pixels corresponding to the same face in consecutive frames. These deep features are used to extract common features from different samples with the same output, finding discriminative high-dimensional information, thereby predicting the corresponding health indicators. However, in actual applications, due to the forward and backward movement of the face in consecutive frames, the facial positions corresponding to the same pixel position are not completely aligned. That is, the same pixel coordinate may be the tip of the nose in the previous frame, but a pixel in the cheek area in the next frame. When performing health monitoring in non-cooperative scenarios, facial movement can easily lead to many such large deviations in the forward and backward pixel positions of the face image, thus affecting the network prediction effect.

[0056] In this regard, the present invention provides a health monitoring method. By utilizing key point technology, a local face area is cut out from a face area image based on the detected face key points, which can suppress the influence of excessive interference pixels on feature extraction. By utilizing the target key points in the face area image, the local face area of ​​the current frame is further aligned with the previous frame, and the aligned local face area is used for health monitoring. This can alleviate the influence of pixel misalignment and reduce the prediction deviation caused by face movement, thereby overcoming the above-mentioned defects.

[0057] Figure 1 This is a flow chart of the health monitoring method provided by the present invention, such as Figure 1 As shown, the method includes:

[0058] Step 110: perform face detection on the currently captured video frame to obtain a face region image of the current frame.

[0059] Specifically, when performing health monitoring, a trained face detection model can be selected and applied to the currently captured video frame to identify and extract the face region, thereby obtaining a face region image of the current frame. Here, the face region image of the current frame refers to the image of the face portion identified and extracted from the currently captured video frame using the face detection algorithm.

[0060] It is understandable that there are many types of existing face detection models, and a suitable model can be selected according to the specific scenario and actual needs. For example, the face detection model can be MTCNN (Multi-task Cascaded Convolutional Networks), Faster R-CNN (Faster Region-based Convolutional Neural Network), SSD (Single Shot MultiBox Detector), etc. The embodiment of the present invention does not make specific limitations on this.

[0061] Step 120: perform key point detection on the face region image to obtain face key points.

[0062] Specifically, after locating the facial region image, a facial landmark detection algorithm can be used to further detect key points on the face. Here, facial landmark detection refers to further locating key facial features within the identified facial region image, thereby obtaining facial landmarks. Facial landmarks are characteristic points in a facial image with clear location and semantic information. For example, facial landmarks can include key locations such as eyebrows, eyes, nose, mouth, and facial contours.

[0063] It is understood that facial landmark detection can be achieved by building a deep learning model (such as a convolutional neural network) to directly predict the coordinates of facial landmarks. During training, the model learns to extract features from facial region images and predict the locations of landmarks based on these features. Facial landmark detection can also be achieved by modeling each landmark as a heat map, which is not specifically limited in this embodiment of the present invention.

[0064] Step 130: Determine a local facial region from the facial region image based on the facial key points.

[0065] Specifically, after locating each key point in the facial region image, these key points can be used to further define the local area of ​​the face. For example, the left or right facial region can be defined by connecting specific key points, and the facial region on one side can be used as the local facial region. The cheek region can also be defined by connecting specific key points, and the cheek region can be used as the local facial region. The mouth region can also be defined by connecting key points such as the nose tip and the corners of the mouth, and this area can also be used as the local facial region.

[0066] After defining the local facial region, it can be cropped from the original facial region image. Here, a local facial region refers to a specific portion or area within a facial image defined by key points. By identifying and cropping the local facial region from the facial region image, we can prevent invalid pixels from interfering with subsequent health monitoring, thereby more accurately capturing subtle facial changes.

[0067] It is understandable that, considering that the video frames collected in non-cooperative scenarios may only contain the side face of the subject, that is, there will only be one local area, therefore, when using the side face or cheek area as the local face area for health monitoring, in order to be applicable to monitoring in both frontal and side face scenarios, a local face area is used as input during network training. When the determined local face area includes both the left face area and the right face area, the better side can be selected from the two sides of the face area by comprehensively considering the confidence of the key points. Specifically, the mean confidence of all key points of each side face can be calculated and compared. A high mean confidence value indicates that this side of the face is better for feature extraction, thereby selecting the local area of ​​the corresponding side face for network prediction.

[0068] Step 140 , based on the target key points in the face region image, align the face local region of the current frame with the face local region of the previous frame, and use the aligned face local region for health monitoring to obtain health indicators.

[0069] Specifically, after determining the local area of ​​the face, in order to reduce the impact of large misalignment of facial pixels in adjacent frames caused by facial motion, key point correction technology can be used to align the facial images of adjacent frames. Here, key point correction technology is a technology in the field of facial image processing, which aims to achieve facial alignment by detecting and locating target key points (such as corners of the eyes, corners of the mouth, tip of the nose, etc.) in the face area image. It should be understood that the target key points in the face area image refer to specific facial feature points in the face area image that are used for alignment and calibration. The target key points are usually the more stable and easily recognizable parts of the facial features, which can serve as a reference for face alignment.

[0070] Specifically, when using key point correction technology to align facial images of adjacent frames, it can be achieved through the following steps: first, find the corresponding target key points in the facial area images (or local facial areas) of the current frame and the previous frame, and match them. This can be achieved by calculating metrics such as Euclidean distance and cosine similarity between the key points; based on the matched key points, calculate a transformation matrix (such as affine transformation, perspective transformation, etc.) to transform the local facial area of ​​the current frame to a position aligned with the previous frame; use the calculated transformation matrix to transform the local facial area of ​​the current frame to obtain the aligned local facial area.

[0071] It's understood that an aligned facial region refers to a region in the current frame that, after the alignment steps, is aligned or similar in position to the previous frame's. This alignment process helps reduce jitter caused by facial movement (such as blinking, smiling, and turning the head), thereby providing more accurate data for subsequent health monitoring.

[0072] After achieving face alignment using keypoint correction technology, the aligned facial regions can be fed into a deep neural network for health prediction, resulting in various health indicators. Health indicators here refer to quantitative or qualitative metrics used to assess an individual's health status. For example, health indicators can include heart rate, respiratory rate, blood pressure, body temperature, etc.

[0073] The method provided by the embodiment of the present invention uses face detection technology to accurately extract a face area image from the currently captured video frame, and uses key point detection technology to further locate the key points of the face in the face area image, which can provide a solid foundation for subsequent health monitoring. Based on the detected face key points, the local face area is determined from the face area image, thereby suppressing the influence of too many interfering pixels on feature extraction. The target key points in the face area image are used to align the local face area of ​​the current frame with the local face area of ​​the previous frame, which can significantly alleviate the influence of pixel misalignment. By performing health monitoring on the aligned local face area, the deep features of the face can be captured more accurately, thereby reducing the prediction deviation caused by face movement and improving the accuracy and effectiveness of health monitoring.

[0074] Based on any of the above embodiments, step 120 specifically includes:

[0075] Step 121: Perform liveness detection on the face area image based on a liveness detection model to obtain a detection result. The liveness detection model is a classification network trained using live face images as positive samples and non-live false face images and / or background images as negative samples.

[0076] It should be noted that in health monitoring solutions based on face videos, in order to improve the accuracy and effectiveness of monitoring, it is usually necessary to avoid attack detection of non-living faces. In response to the problem of non-living suppression, most of the current solutions adopt traditional face liveness detection technology, that is, by detecting the distance changes of key points of the face, identifying actions such as blinking, opening the mouth, nodding, and shaking the head. Although this method of performing liveness detection by triggering corresponding actions by the subject is effective, it requires the subject to actively cooperate with the trigger. In order to be applicable to non-cooperative scenarios, the embodiment of the present invention proposes a non-active cooperative liveness detection solution, which can perform health monitoring in scenarios where the subject is unconscious and non-cooperative. This will have certain application effects in scenarios such as long-distance driving and micro-monitoring during interrogation.

[0077] Specifically, after performing face detection on the currently captured video frame, liveness detection can be further performed on the detected face area. Compared to face detection, liveness detection is a simple binary classification network used to distinguish between real face images and fake, non-live face images or other forged images. It does not require learning the task of facial frame coordinate regression and has good classification results.

[0078] When performing liveness detection on a face region image, the face region image can be input into a pre-trained liveness detection model, and the model is used to perform classification prediction on the face region image to obtain a detection result. Here, the detection result refers to the output obtained by the liveness detection model after performing liveness detection on the input human body region image. It can be a binary classification result, for example, the detection result can be "live" or "non-live". The liveness detection model can perform liveness confidence prediction on the face region image output by the face detection. If it is greater than a set threshold, the detection result can be determined to be live, otherwise the detection result is determined to be non-live. It should be understood that when the detection result is live, it indicates that the input is a real face image, and subsequent pre-processing such as face key point detection and face alignment can be performed at this time; when the detection result is non-live, it indicates that the input is a fake face image or other forged image, and the system can directly jump to the next frame for reasoning.

[0079] It is understandable that before executing step 121, the liveness detection model can be pre-trained through the following steps: first, a large number of live face images are collected as positive samples, and a large number of non-live fake face images and / or actual common background images are collected as negative samples; image processing technology is used to extract features in the image, and the extracted features are input into the deep learning model. Through iterative training, the model can accurately distinguish between positive samples and negative samples, thereby training the liveness detection model.

[0080] Step 122: If the detection result indicates that a living body exists, key point detection is performed on the face region image to obtain face key points.

[0081] Specifically, when the detection result is a living body, it indicates that there is a living body in the input face area image, that is, the input image is a real face image. In this case, the face area image can be detected by a face key point detection algorithm to locate the face key points.

[0082] Based on any of the above embodiments, step 130 specifically includes:

[0083] Step 131, when the face region image includes multiple faces, determining the face key points of the subject from the face key points based on the face identifier of the subject;

[0084] Step 132: determining a local facial region from the facial region image based on the facial key points of the subject.

[0085] It should be noted that, considering that there is still a common interference problem in actual scenarios, some health data (such as heart rate, respiration, etc.) are often collected after a certain period of data, and then uniformly post-processed and filtered for analysis. However, when there are multiple people in the application scenario, because the measured object moves, it may cause the input network prediction to be not the same face during the detection cycle. It is also possible that the face frame of the measured object contains other invalid pixels or even the partial face area of ​​someone else. Most current health monitoring methods have not proposed an effective solution to solve the problem of network prediction errors caused by erroneous face input and the inclusion of too many interfering pixels. In this regard, the present invention proposes a multi-target tracking solution, and uses key point prediction to cut out the local area of ​​the face as the network input, thereby effectively reducing the interference of invalid pixels or other people's facial pixels.

[0086] Specifically, multi-target tracking is achieved by binding the facial identifier of each object to their respective periodic homologous inputs. In a non-cooperative health monitoring scenario, the videos, images, and extracted facial key points, local facial areas, and facial features collected for each object will be bound to the facial identifier of each object, so that when the neural network is input for periodic analysis and prediction, it can be ensured that the predicted health indicators are for the same specified object (i.e., the object being measured), thereby avoiding the problem of network prediction errors caused by erroneous facial input. It should be understood that facial identification refers to a feature or code that can uniquely identify the face of each object, such as a facial feature vector, a face ID, etc.

[0087] Specifically, because each subject's facial region image and extracted facial key points are bound to the subject's facial identifier, the subject's facial key points can be matched from all extracted facial key points based on the subject's facial identifier. Subsequently, based on the subject's facial key points, a local facial region of the subject can be extracted from the facial region image, allowing subsequent use of this local facial region for health monitoring of the subject.

[0088] Furthermore, step 132 specifically includes:

[0089] Step 1321: When the face region image includes a left face region and a right face region of the subject, based on the confidence levels of the facial key points of the subject, respectively determine the mean confidence level of all facial key points in the left face region and the mean confidence level of all facial key points in the right face region.

[0090] Step 1322: Select the area with a higher confidence mean from the left face area and the right face area as the local face area.

[0091] Specifically, in actual application scenarios, the problem of interference between multiple faces can easily lead to large network prediction errors. For example, in a cockpit scenario, the face frame of the front row sometimes includes part of the face area of ​​the back row. Figure 2 is a schematic diagram of the key point area interception provided by the present invention, such as Figure 2 As shown in the figure, the face frame of face A (the red frame on the left in the figure) contains part of the face area of ​​face B (the blue frame on the left in the figure). If the face frame of face A is directly used for health monitoring prediction, it will cause incorrect prediction because it contains too many features of face B. Therefore, the embodiment of the present invention proposes to use the effective areas of the left and right cheeks as input, that is, Figure 2 As shown in the two red rectangular areas on the right side of the figure, the right cheek area is obtained by connecting the key point output numbers 1 and 49, and the left cheek area is obtained by connecting the key points 15 and 53. Here, when performing facial key point detection, each key point will be marked with a serial number. By connecting the key points with the specified serial number, the left cheek area and the right cheek area can be determined, so that the cheek area can be used as a local area of ​​the face for health monitoring prediction. It should be understood that the key points of the above-mentioned face A and face B can be bound and distinguished by their respective face identifiers, or they can be distinguished according to the ratio of the face frame area of ​​face A and face B, and the object with the larger face area can be selected.

[0092] It should be noted that, considering that there are two local facial regions, one on the left and one on the right, when the face is viewed from the front, but only one local facial region when viewed from the side, the network training uses a single local facial region as input. When the facial region image includes both the left and right facial regions, it is necessary to preferentially select one of the two facial regions as input. The embodiment of the present invention proposes a scheme that combines the confidence of key points to optimize the selection between the two local facial regions. The mean confidence values ​​of all key points on each side of the face are compared, and the side facial region with the higher mean confidence value is selected for network prediction.

[0093] Specifically, after locating the facial region image, a keypoint detection algorithm can be used to detect keypoints on the face. For each detected keypoint, the algorithm typically outputs a confidence value, indicating the accuracy of the keypoint's location. Based on the symmetry of the face, the face can be divided into a left-side facial region and a right-side facial region, and keypoints are assigned to the corresponding regions. After the regions are divided, all keypoints within each side face region are screened out, and the confidence values ​​of these keypoints are averaged to obtain the mean confidence value for that side face region. A high mean value indicates that this side of the face is better for feature extraction, so the local region of the corresponding side face (such as the cheek area) can be selected for network prediction.

[0094] In addition, in order to reduce the impact of jitter deviation caused by facial movement, the embodiment of the present invention also adopts the use of key point correction technology to align adjacent frames of facial images, thereby reducing the impact of large misalignment of adjacent frames of facial images. Figure 3 It is a schematic diagram of the key point correction provided by the present invention, such as Figure 3 As shown, the black dots in the left image represent the coordinates of the facial key points in the current frame, and the white dots represent the coordinates of the facial key points in the previous frame. The right image shows the coordinate position change between the current frame and the previous frame after the key point correction technology. It can be seen that the deviation of the uncorrected pixels on the left is large, while the deviation of the pixels in adjacent frames is small after correction. The correction scheme can reduce the prediction error caused by motion. It should be understood that after completing the key point correction and achieving face image alignment, the aligned local face area can be used for health monitoring prediction. The local face area can be the side face area or the cheek area defined by specific key points in the side face area. This embodiment of the present invention does not specifically limit this.

[0095] Based on any of the above embodiments, in step 140, the health monitoring of the aligned local facial area to obtain health indicators includes:

[0096] Step 141: When the aligned local face region is a single-channel image, health monitoring is performed on the aligned local face region based on a single-channel source network to obtain a health indicator.

[0097] Step 142 , when the aligned local face region is a multi-channel image, health monitoring is performed on the aligned local face region based on a multi-channel source network to obtain health indicators.

[0098] It should be noted that in video-based health monitoring solutions, the current mainstream images are divided into RGB and infrared imaging. The former is composed of three channels: red, green, and blue, while the latter is a single-channel image. Because deep learning models generally require inputs to have the same number of channels, for example, most models are designed for RGB images. Therefore, infrared imaging is usually copied into three channels before entering the model to match the number of channels of the RGB image. Although the single channel has the same number of channels as the RGB image after copying and can be used as a unified input for model training, the imaging effects of the two are quite different. In order to fully improve the accuracy of health monitoring predictions, the algorithm level needs to adopt a larger and deeper network to extract deep common features, which is often larger than the network designed only for RGB or single-channel data.

[0099] However, larger and deeper networks also mean higher computational complexity and longer inference times, which will affect the real-time performance of algorithm reasoning, especially in systems that require rapid response. To address this, the present invention adopts separate single- and multi-channel reasoning and a unified data post-processing solution. Specifically, networks trained with corresponding data types are used to perform inference and prediction on input data from different channels, and a unified post-processing solution is used for comprehensive analysis to obtain the output of various health monitoring indicators. This ensures that the real-time performance of reasoning is not as time-consuming as that of large networks trained with mixed training, and does not affect the accuracy of algorithm predictions.

[0100] Specifically, network training uses different data sources, primarily dividing infrared and RGB images, and training separate health monitoring network models: single-channel source networks and multi-channel source networks. This allows for inference and prediction based on the number of channels in the input image during deployment. A unified post-processing solution is then employed to obtain the final health indicators. A single-channel source network refers to a deep learning network designed specifically for processing single-channel images (such as infrared images) and performing health monitoring; a multi-channel source network refers to a deep learning network designed specifically for processing multi-channel images (such as RGB images) and performing health monitoring.

[0101] Based on any of the above embodiments, when the health indicator includes a heart rate indicator and / or a breathing indicator, in step 140, step 141, or step 142, the health monitoring of the aligned local face area is performed to obtain the health indicator, including:

[0102] Step 1401 : Apply the aligned local face area to perform health monitoring to obtain indicator data of the current frame, where the indicator data includes heart rate data and / or respiration data.

[0103] Specifically, by distinguishing the number of channels of the input image and selecting the corresponding single-channel source network or multi-channel source network for inference and prediction, the index data of the current frame can be obtained. If the predicted health indicators are body temperature, blood pressure, etc., after obtaining the index data of the current frame, it can be output and displayed. However, when the health indicators include heart rate indicators and / or respiratory indicators, it is usually necessary to collect data for a certain period of time for analysis and processing before the final output indicators can be obtained. Therefore, for this type of indicator, the embodiment of the present invention proposes a unified post-processing solution, that is, real-time synchronous update of the heart rate waveform and / or respiratory waveform, and smoothing of the data within the period, which can effectively reduce the problem of large fluctuations in the overall output caused by single-frame output errors, thereby improving the accuracy of period prediction. The post-processing solution will be introduced in detail below.

[0104] Step 1402: Denoise the indicator data based on a sparse matrix to obtain a denoised prediction value of the current frame, and store the denoised prediction value of the current frame in a preset queue. The denoised prediction value includes a heart rate prediction value and / or a breathing prediction value. The sparse matrix is ​​initialized with the serial number of the current frame as a parameter.

[0105] Specifically, after obtaining the predicted value output by the single-channel source network or the multi-channel source network (i.e., the indicator data of the current frame), it can be denoised using the sparse matrix updated in real time to obtain the denoised predicted value of the current frame.

[0106] Furthermore, step 1402 specifically includes:

[0107] Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame;

[0108] Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set;

[0109] Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

[0110] Specifically, denoising is achieved by using the cumulative integral set processing composed of the real-time updated sparse matrix D and the network output. Assume that the network continuous frame prediction result is , sum up the integral to get , follow-up And so on. Assume that the current frame is Frame, the real-time updated sparse matrix is ​​used The frame number of the frame is initialized as a parameter Matrix , its formula is as follows:

[0111]

[0112] in, is a A diagonal matrix with all diagonal elements set to 1. To find the inverse matrix function, is the amplification factor, which is set to 100 in the embodiment of the present invention. It is a special The "diagonal matrix" is filled diagonally with the values ​​1, -2, and 1 as shown below:

[0113]

[0114] Then, use The sparse matrix and The cumulative integral set of Multiplying them, we get The denoising prediction value set of , the last value in the set is the denoising prediction value of the current frame.

[0115] Step 1403: draw a heart rate waveform diagram based on the heart rate prediction value, and / or draw a respiration waveform diagram based on the respiration prediction value.

[0116] Specifically, after obtaining the denoised prediction value for the current frame, it can be directly plotted as a point value in the real-time waveform, thereby obtaining a real-time heart rate and respiration waveform. It should be understood that each time sparse denoising is performed, the prediction value for each frame is updated, so the updated value of a certain number of frames can be selected to adjust the waveform to make the waveform smoother.

[0117] Step 1404 : When the denoised prediction values ​​stored in the preset queue are full of cycles, the denoised prediction values ​​stored in the preset queue are comprehensively processed to obtain the health indicator.

[0118] Specifically, for the cumulative integral set , taking the sparse matrix After denoising, the denoising prediction value of the current frame is obtained , save it to the preset queue When the queue After the full cycle, the elements stored in the queue reach Frame (the embodiment of the present invention sets is 180 frames), which will be used as a cycle for prediction and subsequent comprehensive processing operations to obtain the final heart rate and respiratory output indicators.

[0119] Furthermore, step 1404 specifically includes:

[0120] Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values;

[0121] Performing power spectrum density calculation based on the processed predicted value to obtain a calculation result;

[0122] Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

[0123] Specifically, when the queue After the stored elements are fully cycled, first Butterworth filtering is used for smoothing, which provides a smooth transition between the pass and cutoff frequencies, eliminating sharp edges. Next, conventional power spectral density calculations are performed. Power spectral density (PSD) plays an important role in heart rate signal analysis, particularly in the study of heart rate variability (HRV). HRV refers to the variation in the intervals between consecutive heartbeats, reflecting the regulation of cardiac activity by the autonomic nervous system. Therefore, PSD reveals the energy distribution of HRV signals across different frequency ranges, providing a quantitative means for understanding the heart's autonomic regulatory function.

[0124] The specific calculation process is: After Butterworth filtering and smoothing, we get , the power spectrum density can be calculated by Fourier transform. In the embodiment of the present invention, the Fourier transform adopts a 256-cycle length, so the first 180 elements are used Fill, the rest are initialized with 0, and then Fourier transform is performed to obtain a periodic set containing complex elements , a total of 256 output sets, then directly retain the head and tail values, and the middle 254 pairs of elements before and after as the real and imaginary parts of the complex number (for example As the real part, As the imaginary part) and perform the modulo operation to obtain the set , this set is the obtained power spectrum density output, in which each element value is the output of heart rate or breathing index at different frequencies. Then, using the frequency range of heart rate or breathing For example, the heart rate frequency range can be Set to 1.0, Set to 2.5, using this frequency range to Filter and get the corresponding output Finally, The maximum value is the final heart rate or breathing value.

[0125] Table 1 shows the test results of the unified post-processing solution of the present invention. A heart rate nail monitor was worn during the system experience. The data comparison between the two is shown in the following table:

[0126] Table 1 Test results

[0127]

[0128] Based on Table 1 above, it can be seen that among the 23 test records, only two have large differences in the predicted output amplitudes. This shows that the non-cooperative health monitoring method provided by the embodiment of the present invention has a certain degree of accuracy.

[0129] Based on any of the above-mentioned embodiments, the present invention provides a non-cooperative, end-to-end health monitoring method combined with facial liveness detection. To reduce hardware costs and computing resource requirements, and to facilitate deployment and application, the present invention implements health monitoring by simply integrating with existing monitoring systems, eliminating the need for additional hardware. It utilizes liveness detection technology within a deep learning algorithm to meet the requirements of non-cooperative scenarios. It also employs multi-target tracking technology to predict periodic, homologous inputs bound to multiple face IDs. It also employs local region selection of key points and face correction to reduce interference from invalid or other facial pixels and mitigate the effects of pixel misalignment. It also employs a separation of single- and multi-channel input data and a unified post-processing analysis scheme for network prediction, effectively improving the algorithm's prediction accuracy for cross-domain input sources. Finally, to improve the algorithm's prediction accuracy, the present invention proposes a rigorous post-processing scheme for continuously output indicator signals, implementing real-time updates and filtering and smoothing of the network prediction output to effectively reduce the impact of single-frame errors. It also proposes a solution for real-time display of heart rate and respiratory waveforms.

[0130] Figure 4 : is a flow chart of the non-cooperative end-to-end health monitoring method for combined face liveness detection provided by the present invention, such as Figure 4 As shown, the method includes:

[0131] In step S1, face detection is first performed on the currently captured video frame, followed by liveness detection of the detected face area. Liveness detection, compared to face detection, is a simpler binary classification network. It predicts the liveness confidence of the face area image output by face detection. If the confidence exceeds a set threshold, subsequent preprocessing such as face correction is performed. Otherwise, the system skips to the next frame for inference.

[0132] Step S2: Based on the facial key point technology, key point detection is performed on the facial area image. The key point technology predicts the positions of key areas such as eyebrows, eyes, nose, mouth, and facial contour in the facial image to obtain facial key points.

[0133] In step S3, in real-world scenarios, interference between multiple faces can easily lead to large prediction errors. Based on multi-target face tracking technology, each face ID is bound to a corresponding periodic homologous input. Keypoint region extraction technology is also employed, with the left or right cheek region serving as the valid region for health monitoring prediction. Furthermore, this embodiment of the present invention utilizes keypoint correction technology to align adjacent facial frames to mitigate the impact of significant misalignment between adjacent frames.

[0134] In step S4, the number of channels of the input source (i.e., input image) is distinguished, the corresponding network model is selected for inference and prediction according to the number of channels of the input source, and then a unified post-processing solution is adopted.

[0135] The unified post-processing solution proposed in the embodiment of the present invention is a detailed and efficient data smoothing solution, which can greatly reduce the problem of large overall output fluctuations caused by single-frame output errors. It also proposes a solution for real-time display of heart rate and respiratory waveforms. The core operation is to adopt a continuous value sparse denoising and real-time updating solution for the heart rate and respiratory indicators predicted by the network, so as to obtain the heart rate and respiratory variability characteristic values, and retain the output of each frame. The heart rate and respiratory characteristic value queues after a cycle are smoothed by Butterworth filtering, and then quantized by a power spectral density solution, so as to obtain the final heart rate and respiratory output indicators of this cycle.

[0136] Figure 5 : is a flow chart of post-processing of heart rate and / or respiratory index data provided by the present invention, such as Figure 5 As shown in the figure, the specific processing process is divided into denoising and smoothing. The operation above the queue filling is predictive denoising, which plots the real-time indicator waveform. The operation below the queue filling is the unified smoothing and filtering of the denoised prediction values ​​for a cycle to obtain the corresponding health indicator value. Specifically, after obtaining the network output prediction result, it can be cached in a queue. Then, a cumulative integral set consisting of a real-time updated sparse matrix and the network output is used for denoising, resulting in the denoised prediction value for the current frame. This is directly plotted as a point value on the real-time waveform to obtain the real-time heart rate and respiration waveform. Simultaneously, the denoised prediction value for the current frame is cached in another queue. When the queue is full for a cycle, that is, when the elements stored in the queue reach a preset number of frames (e.g., 180 frames), the elements in the queue are used as a cycle prediction and subsequent filtering operations are performed. During the filtering process, Butterworth filtering is first used for smoothing, followed by dequantization using the power spectral density, and finally filtering using the heart rate or respiration range to obtain the final heart rate and respiration output indicators for that cycle.

[0137] It is understandable that the continuous frame prediction output optimization scheme of health indicators and the real-time waveform drawing are lacking in the current deep learning neural network health detection scheme. The embodiment of the present invention combines traditional signal processing algorithms and statistical analysis based on changes in heart rate and respiratory cycle frequency to propose an effective heart rate and respiratory periodic data post-processing scheme. It uses dynamic adjustment of sparse matrices and historical data denoising to draw heart rate and respiratory waveforms in real time, thereby ensuring the continuity of the peak and trough periodic changes of the drawn waveform. At the same time, it also proposes to use Butterworth filtering operation to smooth the periodic output data. Then, with the power spectral density quantization scheme as the core, combined with the frequency range of heart rate and respiratory cycle changes, the quantization results are filtered for maximum value to obtain the final output.

[0138] The method provided in an embodiment of the present invention adopts a scheme for health monitoring prediction based on the correction of key points in adjacent frames and the local effective area of ​​the cheek, which can suppress the influence of excessive interference pixels on network feature extraction. At the same time, a set of periodic indicator optimization schemes based on power spectral density quantization is formulated to effectively improve the accuracy of health indicator prediction.

[0139] The health monitoring device provided by the present invention is described below. The health monitoring device described below and the health monitoring method described above can be referenced to each other.

[0140] Based on any of the above embodiments, Figure 6 is a schematic diagram of the structure of the health monitoring device provided by the present invention, such as Figure 6 As shown, the device includes:

[0141] The face detection unit 610 is used to perform face detection on the currently captured video frame to obtain a face region image of the current frame;

[0142] A key point detection unit 620 is used to perform key point detection on the face region image to obtain face key points;

[0143] A region determining unit 630 is configured to determine a local face region from the face region image based on the face key points;

[0144] The health monitoring unit 640 is used to align the local face area of ​​the current frame with the local face area of ​​the previous frame based on the target key points in the face area image, and apply the aligned local face area to perform health monitoring to obtain health indicators.

[0145] The device provided by the embodiment of the present invention uses face detection technology to accurately extract a face area image from the currently captured video frame, and uses key point detection technology to further locate the key points of the face in the face area image, which can provide a solid foundation for subsequent health monitoring. Based on the detected face key points, the local face area is determined from the face area image, thereby suppressing the influence of too many interfering pixels on feature extraction. The target key points in the face area image are used to align the local face area of ​​the current frame with the local face area of ​​the previous frame, which can significantly alleviate the influence of pixel misalignment. By performing health monitoring on the aligned local face area, the deep features of the face can be captured more accurately, thereby reducing the prediction deviation caused by face movement and improving the accuracy and effectiveness of health monitoring.

[0146] Based on any of the above embodiments, the key point detection unit 620 is specifically configured to:

[0147] Performing liveness detection on the face region image based on a liveness detection model to obtain a detection result, wherein the liveness detection model is a classification network trained using live face images as positive samples and non-live false face images and / or background images as negative samples;

[0148] When the detection result indicates that there is a living body, key point detection is performed on the face area image to obtain face key points.

[0149] Based on any of the above embodiments, the region determining unit 630 includes:

[0150] a key point determination subunit, configured to determine the facial key points of the subject from the facial key points based on the face identifier of the subject when the face region image includes multiple faces;

[0151] The local area determination subunit is used to determine the local face area from the face area image based on the face key points of the object being measured.

[0152] Based on any of the above embodiments, the local area determination subunit is specifically configured to:

[0153] In a case where the face region image includes a left face region and a right face region of the subject, based on the confidences of the facial key points of the subject, respectively determining a mean confidence value of all facial key points in the left face region and a mean confidence value of all facial key points in the right face region;

[0154] A side area with a higher confidence mean value is selected from the left face area and the right face area as the local face area.

[0155] Based on any of the above embodiments, the health monitoring unit 640 includes:

[0156] a single-channel monitoring unit configured to, when the aligned local face region is a single-channel image, perform health monitoring on the aligned local face region based on a single-channel source network to obtain a health indicator;

[0157] The multi-channel monitoring unit is used to apply the aligned local face area to perform health monitoring based on the multi-channel source network to obtain health indicators when the aligned local face area is a multi-channel image.

[0158] Based on any of the above embodiments, when the health indicator includes a heart rate indicator and / or a breathing indicator, the health monitoring unit 640 includes:

[0159] a prediction subunit, configured to perform health monitoring on the aligned local face region to obtain indicator data of a current frame, wherein the indicator data includes heart rate data and / or respiration data;

[0160] a denoising subunit, configured to denoise the indicator data based on a sparse matrix to obtain a denoised predicted value of the current frame, and store the denoised predicted value of the current frame in a preset queue, wherein the denoised predicted value includes a heart rate predicted value and / or a respiration predicted value, and the sparse matrix is ​​initialized using the sequence number of the current frame as a parameter;

[0161] a drawing subunit, configured to draw a heart rate waveform diagram based on the heart rate prediction value, and / or draw a respiration waveform diagram based on the respiration prediction value;

[0162] The post-processing subunit is used to perform comprehensive processing on the denoised prediction values ​​stored in the preset queue to obtain the health indicator when the denoised prediction values ​​stored in the preset queue are full of cycles.

[0163] Based on any of the above embodiments, the denoising subunit is specifically configured to:

[0164] Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame;

[0165] Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set;

[0166] Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

[0167] Based on any of the above embodiments, the post-processing subunit is specifically configured to:

[0168] Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values;

[0169] Performing power spectrum density calculation based on the processed predicted value to obtain a calculation result;

[0170] Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

[0171] Figure 7 An example of a physical structure diagram of an electronic device is shown below. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a health monitoring method, which includes: performing face detection on the currently acquired video frame to obtain a face region image of the current frame; performing key point detection on the face region image to obtain face key points; determining a face local region from the face region image based on the face key points; aligning the face local region of the current frame with the face local region of the previous frame based on the target key points in the face region image, and applying the aligned face local region to perform health monitoring to obtain a health indicator.

[0172] Furthermore, the logic instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the relevant art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0173] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the health monitoring method provided by the above methods, which includes: performing face detection on the currently acquired video frame to obtain a face area image of the current frame; performing key point detection on the face area image to obtain face key points; based on the face key points, determining a local face area from the face area image; based on the target key points in the face area image, aligning the local face area of ​​the current frame with the local face area of ​​the previous frame, and applying the aligned local face area to perform health monitoring to obtain health indicators.

[0174] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the health monitoring method provided by the above-mentioned methods, the method comprising: performing face detection on the currently acquired video frame to obtain a face area image of the current frame; performing key point detection on the face area image to obtain face key points; based on the face key points, determining a local face area from the face area image; based on the target key points in the face area image, aligning the local face area of ​​the current frame with the local face area of ​​the previous frame, and applying the aligned local face area to perform health monitoring to obtain health indicators.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0176] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A health monitoring method, characterized in that: include: Perform face detection on the currently acquired video frame to obtain the face area image of the current frame; Performing key point detection on the face area image to obtain face key points; Based on the facial key points, determining a local facial region from the facial region image; Based on the target key points in the face region image, aligning the face local region of the current frame with the face local region of the previous frame, and applying the aligned face local region to perform health monitoring to obtain a health index; In the case where the health indicator includes a heart rate indicator and / or a breathing indicator, the health monitoring is performed on the aligned local face area to obtain the health indicator, including: Applying the aligned local face area to perform health monitoring to obtain indicator data of the current frame; Based on the sparse matrix, the indicator data is denoised to obtain a denoised prediction value of the current frame, and the denoised prediction value of the current frame is stored in a preset queue; When the denoised prediction values ​​stored in the preset queue are full of cycles, the denoised prediction values ​​stored in the preset queue are comprehensively processed to obtain the health indicator; The step of denoising the indicator data based on the sparse matrix to obtain a denoised prediction value of the current frame includes: Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame; Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set; Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

2. The health monitoring method according to claim 1, characterized in that: The step of performing key point detection on the face region image to obtain face key points includes: Based on a liveness detection model, performing liveness detection on the face region image to obtain a detection result, wherein the liveness detection model is a classification network trained by using live face images as positive samples and non-live false face images and / or background images as negative samples; When the detection result indicates that there is a living body, key point detection is performed on the face area image to obtain face key points.

3. The health monitoring method according to claim 1, characterized in that: The step of determining a local face region from the face region image based on the face key points includes: In the case where the face region image includes a plurality of faces, based on the face identification of the subject, determining the face key points of the subject from the face key points; Based on the facial key points of the detected object, a local facial area is determined from the facial area image.

4. The health monitoring method according to claim 3, characterized in that: The method of determining a local face area from the face area image based on the face key points of the object under test includes: In a case where the face region image includes a left face region and a right face region of the subject, based on the confidences of the face key points of the subject, respectively determining the mean confidence values ​​of all face key points in the left face region and the mean confidence values ​​of all face key points in the right face region; A side area with a higher confidence mean value is selected from the left side face area and the right side face area as the local face area.

5. The health monitoring method according to claim 1, characterized in that: The application performs health monitoring on the aligned local face area to obtain health indicators, including: In the case where the aligned local face region is a single-channel image, based on a single-channel source network, the aligned local face region is used for health monitoring to obtain a health index; In the case where the aligned local face region is a multi-channel image, the aligned local face region is used for health monitoring based on a multi-channel source network to obtain a health indicator.

6. The health monitoring method according to any one of claims 1 to 5, characterized in that: The indicator data includes heart rate data and / or breathing data, the denoising prediction value includes a heart rate prediction value and / or a breathing prediction value, the sparse matrix is ​​initialized using the sequence number of the current frame as a parameter, and the method further includes: A heart rate waveform graph is drawn based on the heart rate prediction value, and / or a respiration waveform graph is drawn based on the respiration prediction value.

7. The health monitoring method according to any one of claims 1 to 5, characterized in that: The comprehensive processing of the denoised prediction values ​​stored in the preset queue to obtain the health indicator includes: Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values; Based on the processed predicted value, power spectrum density calculation is performed to obtain a calculation result; Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

8. A health monitoring device, characterized in that: include: A face detection unit is used to perform face detection on the currently acquired video frame to obtain a face region image of the current frame; A key point detection unit, used to perform key point detection on the face area image to obtain face key points; A region determination unit, configured to determine a local face region from the face region image based on the face key points; A health monitoring unit, configured to align a local face region of a current frame with a local face region of a previous frame based on target key points in the face region image, and perform health monitoring using the aligned local face region to obtain a health index; In the case where the health indicator includes a heart rate indicator and / or a breathing indicator, the health monitoring is performed on the aligned local face area to obtain the health indicator, including: Applying the aligned local face area to perform health monitoring to obtain indicator data of the current frame; Based on the sparse matrix, the indicator data is denoised to obtain a denoised prediction value of the current frame, and the denoised prediction value of the current frame is stored in a preset queue; When the denoised prediction values ​​stored in the preset queue are full of cycles, the denoised prediction values ​​stored in the preset queue are comprehensively processed to obtain the health indicator; The step of denoising the indicator data based on the sparse matrix to obtain a denoised prediction value of the current frame includes: Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame; Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set; Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the health monitoring method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the health monitoring method according to any one of claims 1 to 7 is implemented.

Images