Health monitoring method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a health monitoring method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the face detection of a currently collected video frame, and obtaining a face region image of the current frame; carrying out key point detection on the face region image to obtain face key points; determining a face local area from the face area image based on the face key points; and based on the target key point in the face region image, aligning the face local region of the current frame with the face local region of the previous frame, and performing health monitoring by using the aligned face local region to obtain a health index. According to the method, the face local area is determined from the face area image based on the detected face key points, the influence of excessive interference pixels on feature extraction can be suppressed, the influence caused by pixel dislocation can be remarkably relieved by aligning the face local area of the current frame with the previous frame, and thus the accuracy of health monitoring is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a health monitoring method, device, electronic device and storage medium. Background Art

[0002] At present, artificial intelligence health monitoring technology is increasingly gaining widespread attention and application in the fields of security, smart medical care, smart cockpits, etc. This technology can achieve real-time monitoring and evaluation of individual health status through advanced algorithms and sensor equipment. Especially in special scenarios (such as cockpits), by adopting health monitoring methods based on face videos, it can actively reject non-living objects, quickly and cheaply gain insight into abnormal data changes such as heart rate and breathing of the measured object, thereby playing a role in protecting life safety.

[0003] The health monitoring method based on face video mainly uses deep learning neural networks to mine continuous frames of face images and evaluate the individual's health status by predicting heart rate, breathing and other data. However, in scenarios where the subject cannot actively cooperate for a long time (such as driving scenarios), this method has the defect of large prediction deviation. This is because in non-cooperative scenarios, due to the continuous movement of the face, pixels in the same position may correspond to different face areas in different frames. This problem of large deviation in the position of front and back pixels will seriously affect the prediction effect of the neural network. Therefore, how to achieve accurate and stable health monitoring in non-cooperative scenarios has become an urgent problem to be solved in the current field of artificial intelligence health monitoring. Summary of the invention

[0004] The present invention provides a health monitoring method, device, electronic device and storage medium, which are used to solve the defect that in a non-cooperative health monitoring scenario, the prediction deviation is large due to face movement, which affects the health monitoring effect.

[0005] The present invention provides a health monitoring method, comprising: Perform face detection on the currently acquired video frame to obtain the face area image of the current frame; Performing key point detection on the face area image to obtain face key points; Based on the facial key points, determining a local facial region from the facial region image; Based on the target key points in the face area image, the face local area of ​​the current frame is aligned with the face local area of ​​the previous frame, and the aligned face local area is used for health monitoring to obtain a health index.

[0006] According to a health monitoring method provided by the present invention, the key point detection is performed on the face area image to obtain the face key points, including: Based on a liveness detection model, performing liveness detection on the face region image to obtain a detection result, wherein the liveness detection model is a classification network trained by using live face images as positive samples and non-live false face images and / or background images as negative samples; When the detection result indicates that there is a living body, key point detection is performed on the face area image to obtain face key points.

[0007] According to a health monitoring method provided by the present invention, the method of determining a local face area from the face area image based on the face key points includes: In the case where the face region image includes a plurality of faces, based on the face identification of the subject, determining the face key points of the subject from the face key points; Based on the facial key points of the detected object, a local facial area is determined from the facial area image.

[0008] According to a health monitoring method provided by the present invention, the method of determining a local face area from the face area image based on the face key points of the subject under test comprises: In a case where the face region image includes a left face region and a right face region of the subject, based on the confidences of the face key points of the subject, respectively determining the mean confidence values ​​of all face key points in the left face region and the mean confidence values ​​of all face key points in the right face region; A side area with a higher confidence mean value is selected from the left side face area and the right side face area as the local face area.

[0009] According to a health monitoring method provided by the present invention, the health monitoring is performed on the aligned local face area to obtain health indicators, including: In the case where the aligned local face region is a single-channel image, based on a single-channel source network, the aligned local face region is used for health monitoring to obtain a health index; In the case where the aligned local face region is a multi-channel image, the aligned local face region is used for health monitoring based on a multi-channel source network to obtain a health indicator.

[0010] According to a health monitoring method provided by the present invention, when the health index includes a heart rate index and / or a breathing index, the health monitoring is performed on the aligned local face area to obtain the health index, including: Applying the aligned local face area to perform health monitoring to obtain indicator data of the current frame, wherein the indicator data includes heart rate data and / or breathing data; Based on a sparse matrix, the indicator data is denoised to obtain a denoised prediction value of the current frame, and the denoised prediction value of the current frame is stored in a preset queue, wherein the denoised prediction value includes a heart rate prediction value and / or a breathing prediction value, and the sparse matrix is ​​initialized using the sequence number of the current frame as a parameter; Draw a heart rate waveform graph based on the heart rate prediction value, and / or draw a respiration waveform graph based on the respiration prediction value; When the denoising prediction values ​​stored in the preset queue are full of cycles, the denoising prediction values ​​stored in the preset queue are comprehensively processed to obtain the health indicator.

[0011] According to a health monitoring method provided by the present invention, the indicator data is denoised based on a sparse matrix to obtain a denoised prediction value of the current frame, including: Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame; Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set; Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

[0012] According to a health monitoring method provided by the present invention, the denoised prediction values ​​stored in the preset queue are comprehensively processed to obtain the health index, including: Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values; Based on the processed predicted value, power spectrum density calculation is performed to obtain a calculation result; Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

[0013] The present invention also provides a health monitoring device, comprising: A face detection unit is used to perform face detection on the currently acquired video frame to obtain a face region image of the current frame; A key point detection unit, used to perform key point detection on the face area image to obtain face key points; A region determination unit, configured to determine a local face region from the face region image based on the face key points; The health monitoring unit is used to align the local face area of ​​the current frame with the local face area of ​​the previous frame based on the target key points in the face area image, and apply the aligned local face area to perform health monitoring to obtain health indicators.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned health monitoring methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described health monitoring methods.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the health monitoring method described above is implemented.

[0017] The health monitoring method, device, electronic device and storage medium provided by the present invention can accurately extract the face area image from the currently collected video frame through face detection technology, and further locate the key points of the face in the face area image by using key point detection technology, which can provide a solid foundation for subsequent health monitoring. Based on the detected face key points, the local face area is determined from the face area image, thereby suppressing the influence of too many interfering pixels on feature extraction, and using the target key points in the face area image to align the local face area of ​​the current frame with the local face area of ​​the previous frame, which can significantly alleviate the influence of pixel misalignment. By performing health monitoring on the aligned local face area, the deep features of the face can be captured more accurately, thereby reducing the prediction deviation caused by face movement and improving the accuracy and effectiveness of health monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 is a flow chart of the health monitoring method provided by the present invention; Figure 2 It is a schematic diagram of the key point area interception provided by the present invention; Figure 3 It is a schematic diagram of key point correction provided by the present invention; Figure 4 It is a flow chart of the non-cooperative end-to-end health monitoring method combined with face liveness detection provided by the present invention; Figure 5 It is a schematic diagram of the process of post-processing of heart rate and / or breathing index data provided by the present invention; Figure 6 is a schematic diagram of the structure of the health monitoring device provided by the present invention; Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The current solutions for obtaining health data are basically human-initiated cooperation, which are mainly divided into the following two solutions: a. Attach sensors or wear instruments on the chest or wrist of the subject, such as a sphygmomanometer or respirator, to collect health data such as heart rate, blood pressure and respiration, and then conduct comprehensive data analysis to give health status. b. Health monitoring technology based on face video, which continuously collects face frames and uses deep learning neural networks to mine the deep features of pixel changes of previous and subsequent face frames to predict health data such as heart rate and respiration.

[0022] However, from the perspective of the data input source of health monitoring technology, although high-precision health data can be obtained by using sensors attached to the subject, or using equipment such as blood pressure monitors and respirators, its applicable scenarios are greatly limited, especially in scenarios where the subject cannot actively cooperate for a long time, such as driving scenarios, because it is not possible to require every driver to carry an instrument.

[0023] The health monitoring solution based on face video also has certain defects in scenarios where the subject cannot actively cooperate for a long time. In this solution, the core method is to use a deep learning neural network to mine deep features based on the changes in pixels corresponding to the same face in consecutive frames, and use these deep features of different samples with the same output to extract their common features and find high-dimensional information with discriminative features, so as to predict the corresponding health indicators. However, in actual applications, due to the forward and backward movement of faces in consecutive frames, the face positions corresponding to the pixels in the same position cannot be completely aligned, that is, the same pixel coordinate may be the tip of the nose in the previous frame, but may be a pixel in the cheek area in the next frame. When performing health monitoring in non-cooperative scenarios, face movement can easily lead to many such large deviations in the forward and backward pixel positions in the face image, thus affecting the network prediction effect.

[0024] In this regard, the present invention provides a health monitoring method, which utilizes key point technology to cut out a local face area from a face area image based on detected face key points, thereby suppressing the influence of excessive interference pixels on feature extraction. By utilizing the target key points in the face area image, the local face area of ​​the current frame is further aligned with the previous frame, and the aligned local face area is used for health monitoring, which can alleviate the influence of pixel misalignment and reduce the prediction deviation caused by face movement, thereby overcoming the above-mentioned defects.

[0025] Figure 1 is a flow chart of the health monitoring method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, performing face detection on the currently captured video frame to obtain a face region image of the current frame.

[0026] Specifically, when performing health monitoring, a trained face detection model can be selected and applied to the currently acquired video frame to identify and extract the face area, thereby obtaining the face area image of the current frame. Here, the face area image of the current frame refers to the image of the face part identified and extracted from the currently acquired video frame by the face detection algorithm.

[0027] It is understandable that there are many types of existing face detection models, and a suitable model can be selected according to specific scenarios and actual needs. For example, the face detection model can be MTCNN (Multi-task Cascaded Convolutional Networks), Faster R-CNN (Faster Region-based Convolutional Neural Network), SSD (Single Shot MultiBox Detector), etc. The embodiment of the present invention does not make specific limitations on this.

[0028] Step 120: perform key point detection on the face area image to obtain face key points.

[0029] Specifically, after locating the face region image, the face key point detection algorithm can be further used to detect the key points on the face. Here, face key point detection refers to further locating the key feature points of the face in the determined face region image, so as to obtain the face key points. Face key points refer to some feature points in the face image with clear position and semantic information. For example, face key points can include key positions such as eyebrows, eyes, nose, mouth and facial contour.

[0030] It is understandable that facial key point detection can be achieved by constructing a deep learning model (such as a convolutional neural network) to directly predict the coordinates of facial key points. During the training process, the model will learn to extract features from the face area image and predict the location of the key points based on these features. Facial key point detection can also be achieved by modeling each key point into a heat map, which is not specifically limited in the embodiments of the present invention.

[0031] Step 130: determining a local face region from the face region image based on the face key points.

[0032] Specifically, after locating each key point in the face region image, the local region of the face can be further defined based on these key points. For example, the left face region or the right face region can be defined by connecting specific key points, and the face region on one side can be used as the local face region; the cheek region can also be defined by connecting specific key points, and the cheek region can be used as the local face region; the mouth region can also be defined by connecting key points such as the nose tip and the corners of the mouth, and this region can also be used as the local face region.

[0033] After defining the local area of ​​the face, the local area of ​​the face can be cropped from the original face area image. Here, the local area of ​​the face refers to a specific part or area defined by key points in the face image. By determining and cropping the local area of ​​the face from the face area image, it is possible to avoid other invalid pixels from interfering with subsequent health monitoring, thereby more accurately capturing subtle changes in the face.

[0034] It is understandable that, considering that the video frames collected in non-cooperative scenarios may only contain the side face of the subject, that is, there will only be one local area. Therefore, when the side face or cheek area is used as the local face area for health monitoring, in order to be applicable to monitoring in both frontal and side face scenarios, a local face area is used as input during network training. When the determined local face area includes both the left face area and the right face area, the better side can be selected from the two sides of the face area by comprehensively considering the confidence of the key points. Specifically, the mean confidence of all key points of each side face can be calculated and compared. A high mean confidence value indicates that this side of the face is better for feature extraction, thereby selecting the local area of ​​the corresponding side face for network prediction.

[0035] Step 140, based on the target key points in the face area image, align the local face area of ​​the current frame with the local face area of ​​the previous frame, and use the aligned local face area to perform health monitoring to obtain health indicators.

[0036] Specifically, after determining the local area of ​​the face, in order to reduce the impact of large misalignment of facial pixels in adjacent frames caused by facial movement, key point correction technology can be used to align adjacent frames of face images. Here, key point correction technology is a technology in the field of face image processing, which aims to achieve face alignment by detecting and locating target key points (such as corners of the eyes, corners of the mouth, tip of the nose, etc.) in the face area image. It should be understood that the target key points in the face area image refer to specific facial feature points in the face area image that are used for alignment and calibration. The target key points are usually the more stable and easily recognizable parts of the facial features, which can be used as a reference for face alignment.

[0037] Specifically, when using key point correction technology to align facial images of adjacent frames, it can be achieved through the following steps: first, in the facial area images (or local facial areas) of the current frame and the previous frame, find the corresponding target key points and match them. This can be achieved by calculating metrics such as Euclidean distance and cosine similarity between the key points; based on the matched key points, calculate a transformation matrix (such as affine transformation, perspective transformation, etc.) to transform the local facial area of ​​the current frame to a position aligned with the previous frame; use the calculated transformation matrix to transform the local facial area of ​​the current frame to obtain the aligned local facial area.

[0038] It can be understood that the aligned face local area means that after the above alignment steps, the face local area of ​​the current frame and the face local area of ​​the previous frame are consistent or close in position. This alignment process helps to reduce the impact of jitter deviation caused by face movement (such as blinking, smiling, turning the head, etc.), thereby providing more accurate data for subsequent health monitoring.

[0039] After the face is aligned using key point correction technology, the aligned face local area can be input into the deep neural network for health prediction, thereby obtaining various health indicators. Here, health indicators refer to quantitative or qualitative indicators used to evaluate the health status of an individual. For example, health indicators can include heart rate, respiratory rate, blood pressure, body temperature, etc.

[0040] The method provided by the embodiment of the present invention uses face detection technology to accurately extract the face area image from the currently collected video frame, and uses key point detection technology to further locate the key points of the face in the face area image, which can provide a solid foundation for subsequent health monitoring. Based on the detected face key points, the local face area is determined from the face area image, thereby suppressing the influence of too many interfering pixels on feature extraction. The target key points in the face area image are used to align the local face area of ​​the current frame with the local face area of ​​the previous frame, which can significantly alleviate the influence of pixel misalignment. By performing health monitoring on the aligned local face area, the deep features of the face can be captured more accurately, thereby reducing the prediction deviation caused by face movement and improving the accuracy and effectiveness of health monitoring.

[0041] Based on any of the above embodiments, step 120 specifically includes: Step 121, based on the liveness detection model, perform liveness detection on the face area image to obtain a detection result, wherein the liveness detection model is a classification network trained by using live face images as positive samples and non-live false face images and / or background images as negative samples.

[0042] It should be noted that in health monitoring solutions based on face videos, in order to improve the accuracy and effectiveness of monitoring, it is usually necessary to avoid attack detection of non-living faces. In response to the problem of non-living suppression, most solutions currently adopt traditional face liveness detection technology, that is, by detecting the distance changes of key points of the face and identifying actions such as blinking, opening the mouth, nodding, and shaking the head. Although this method of performing liveness detection by triggering corresponding actions by the object under test is effective, it requires the object under test to actively cooperate with the trigger. In order to be applicable to non-cooperative scenarios, an embodiment of the present invention proposes a non-active cooperative liveness detection solution, which can perform health monitoring in scenarios where the object under test is unconscious and non-cooperative, which will have certain application effects in scenarios such as long-distance driving and micro-monitoring during interrogation.

[0043] Specifically, after performing face detection on the currently captured video frame, liveness detection can be further performed on the detected face area. Compared with face detection, liveness detection is a simple two-classification network used to distinguish real face images from non-live fake face images or other forged images. It does not require task learning for face frame coordinate regression and has a good classification effect.

[0044] When performing liveness detection on a face region image, the face region image can be input into a pre-trained liveness detection model, and the face region image can be classified and predicted by the model to obtain a detection result. Here, the detection result refers to the output obtained by the liveness detection model after performing liveness detection on the input human body region image, which can be a binary classification result, for example, the detection result can be "live" or "non-live". The liveness detection model can predict the liveness confidence of the face region image output by the face detection, and the detection result can be determined to be live if it is greater than a set threshold, otherwise the detection result is determined to be non-live. It should be understood that when the detection result is liveness, it indicates that the input is a real face image, and subsequent preprocessing such as face key point detection and face alignment can be performed at this time; when the detection result is non-liveness, it indicates that the input is a fake face image or other forged image, and the system can be directly jumped to the next frame for reasoning.

[0045] It is understandable that before executing step 121, the liveness detection model can be pre-trained through the following steps: first, a large number of live face images are collected as positive samples, and a large number of non-live fake face images and / or actual common background images are collected as negative samples; image processing technology is used to extract features in the image, and the extracted features are input into the deep learning model, and through iterative training, the model can accurately distinguish between positive samples and negative samples, thereby training the liveness detection model.

[0046] Step 122, when the detection result indicates that there is a living body, key point detection is performed on the face area image to obtain face key points.

[0047] Specifically, when the detection result is a living body, it indicates that there is a living body in the input face area image, that is, the input image is a real face image. In this case, the face area image can be detected by a face key point detection algorithm to locate the face key points.

[0048] Based on any of the above embodiments, step 130 specifically includes: Step 131, when the face region image includes a plurality of faces, based on the face identifier of the subject, determining the face key points of the subject from the face key points; Step 132: determining a local face region from the face region image based on the face key points of the object being measured.

[0049] It should be noted that there is a common interference problem in actual scenarios, because some health data (such as heart rate, breathing, etc.) are often uniformly post-processed and filtered and analyzed after collecting data for a certain period of time. However, when there are multiple people in the application scenario, because the object being measured moves, it may cause the input network prediction to be not the same face during the detection cycle. It may also appear that the face frame of the object being measured contains other invalid pixels or even the partial face area of ​​others. However, most of the current health monitoring methods have not proposed an effective solution to solve the problem of network prediction errors caused by erroneous face input and too many interfering pixels. In this regard, the present invention proposes a multi-target tracking solution, and cuts out the local area of ​​the face as the network input through key point prediction, thereby effectively reducing the interference of invalid pixels or other people's facial pixels.

[0050] Specifically, multi-target tracking is achieved by binding the facial identifier of each object to their respective periodic homologous inputs. In a non-cooperative health monitoring scenario, the videos, images, and extracted facial key points, local facial areas, facial features, etc. collected for each object will be bound to the facial identifier of each object, so that when the neural network is input for periodic analysis and prediction, it can be ensured that the predicted health indicators are for the same specified object (i.e., the object being measured), thereby avoiding the problem of network prediction errors caused by erroneous facial input. It should be understood that a facial identifier refers to a feature or code that can uniquely identify the face of each object, such as a facial feature vector, a face ID, etc.

[0051] Specifically, since the face region image and the extracted face key points of each object are bound to the face identifier of the object, the face key points of the object can be matched from all the extracted face key points according to the face identifier of the object. Subsequently, according to the face key points of the object, the local face region of the object can be cut out from the face region image, so that the local face region can be used to perform health monitoring on the object in the future.

[0052] Furthermore, step 132 specifically includes: Step 1321, when the face region image includes the left face region and the right face region of the subject, based on the confidence of the face key points of the subject, respectively determine the mean confidence value of all the face key points in the left face region and the mean confidence value of all the face key points in the right face region; Step 1322: Select a side region with a higher confidence mean value from the left face region and the right face region as the local face region.

[0053] Specifically, in actual application scenarios, the problem of interference between multiple faces can easily lead to large network prediction errors. For example, in a cockpit scenario, the face frame of the front row sometimes includes part of the face area of ​​the back row. Figure 2 is a schematic diagram of the key point area interception provided by the present invention, such as Figure 2 As shown in the figure, the face frame of face A (the red frame on the left in the figure) contains part of face B's area (the blue frame on the left in the figure). If the face frame of face A is used directly for health monitoring prediction, it will cause incorrect prediction because it contains too many features of face B. Therefore, the embodiment of the present invention proposes to use the effective areas of the left and right cheeks as input, that is, Figure 2 As shown in the two red rectangular areas on the right, the right cheek area is obtained by connecting the key point output serial numbers 1 and 49, and the left cheek area is obtained by connecting the serial numbers 15 and 53. Here, when performing facial key point detection, each key point will be marked with a serial number. By connecting the key points with specified serial numbers, the left cheek area and the right cheek area can be determined, so that the cheek area can be used as a local area of ​​the face for health monitoring prediction. It should be understood that the key points of the above-mentioned face A and face B can be bound and distinguished by their respective face identifiers, or they can be distinguished according to the ratio of the face frame area of ​​face A and face B, and the object with a larger face area can be selected.

[0054] It should be noted that, considering that there are two local face regions on the left and right sides when the face is facing forward, but there is only one local face region when the face is in profile, one local region is used as input during network training. When the face region image includes both the left face region and the right face region, it is necessary to select one side of the face regions on both sides as input. The embodiment of the present invention proposes a scheme of comprehensive key point confidence for the optimal selection between the two local face regions, compares the mean values ​​of all key point confidences of each side face, and selects the side face region with a higher mean confidence value for network prediction.

[0055] Specifically, after locating the face area image, the key point detection algorithm can be used to detect the key points on the face. For each detected key point, the algorithm usually outputs a confidence value, indicating the accuracy of the key point position. According to the symmetry of the face, the face can be divided into the left face area and the right face area, and the key points are assigned to the corresponding areas. After the area is divided, all the key points in each side face area are screened out, and the confidence values ​​of these key points are averaged to obtain the confidence mean of the side face area. A high mean value indicates that this side of the face is better for feature extraction, so the local area of ​​the corresponding side face (such as the cheek area) can be selected for network prediction.

[0056] In addition, in order to reduce the impact of jitter deviation caused by face movement, the embodiment of the present invention also adopts the use of key point correction technology to align adjacent frames of face images, thereby reducing the impact of large misalignment of adjacent frames of face images. Figure 3 It is a schematic diagram of the key point correction provided by the present invention, such as Figure 3 As shown, the black dots in the left image represent the coordinates of the key points of the face in the current frame, the white dots represent the coordinates of the key points of the face in the previous frame, and the right image is the change in coordinate position between the current frame and the previous frame obtained after the key point correction technology. It can be seen that the deviation of the uncorrected pixels on the left is large, while the deviation of the pixels of adjacent frames is small after correction. The correction scheme can reduce the prediction error caused by motion. It should be understood that after completing the key point correction and realizing the alignment of the face picture, the aligned local face area can be used for health monitoring prediction. The local face area can be a side face area or a cheek area defined by specific key points in the side face area. The embodiment of the present invention does not specifically limit this.

[0057] Based on any of the above embodiments, in step 140, the health monitoring of the aligned local face area is performed to obtain the health index, including: Step 141, when the aligned local face region is a single-channel image, based on a single-channel source network, the aligned local face region is used for health monitoring to obtain a health index; Step 142, when the aligned local face region is a multi-channel image, based on a multi-channel source network, the aligned local face region is used for health monitoring to obtain a health indicator.

[0058] It should be noted that in video-based health monitoring solutions, the current mainstream images are divided into RGB and infrared imaging. The former is composed of three channels: red, green, and blue, while the latter is a single-channel image. Since deep learning models usually require the input to have the same number of channels, for example, most models are designed for RGB images, infrared imaging is usually copied into three channels before entering the model to match the number of channels of RGB images. Although the single channel has the same number of channels as the RGB image after copying and can be used as a unified input for model training, the imaging effects of the two are quite different. In order to fully improve the accuracy of health monitoring predictions, a larger and deeper network is needed at the algorithm level to extract deep common features, which is often larger than a network designed only for RGB or single-channel data.

[0059] However, a larger and deeper network also means higher computational complexity and longer reasoning time, which will affect the real-time performance of algorithm reasoning, especially in systems that require rapid response, which will not be able to meet the needs. In this regard, the embodiment of the present invention adopts single and multi-channel separate reasoning and a unified data post-processing solution, that is, the network trained with the corresponding data type is used for reasoning and prediction of input data from different channels, and a unified post-processing solution is used for comprehensive analysis to obtain the output of various health monitoring indicators, which ensures that the real-time performance of reasoning will not be as time-consuming as that of a large network with mixed training, and will not affect the accuracy of algorithm prediction.

[0060] Specifically, different data sources are used during network training, and infrared images and RGB images are mainly divided, and their respective health monitoring network models, namely single-channel source network and multi-channel source network, are trained separately. In actual deployment, the corresponding network model is directly selected according to the number of channels of the input image for inference and prediction, and then a unified post-processing scheme is adopted to obtain the final health index. Here, the single-channel source network refers to a deep learning network designed for processing single-channel images (such as infrared images) and performing health monitoring; the multi-channel source network refers to a deep learning network designed for processing multi-channel images (such as RGB images) and performing health monitoring.

[0061] Based on any of the above embodiments, when the health index includes a heart rate index and / or a breathing index, in step 140, step 141 or step 142, the health monitoring is performed on the aligned local face area to obtain the health index, including: Step 1401, applying the aligned local face area to perform health monitoring to obtain indicator data of the current frame, wherein the indicator data includes heart rate data and / or breathing data.

[0062] Specifically, by distinguishing the number of channels of the input image and selecting the corresponding single-channel source network or multi-channel source network for inference and prediction, the index data of the current frame can be obtained. If the predicted health indicators are body temperature, blood pressure, etc., after obtaining the index data of the current frame, it can be output and displayed. However, when the health indicators include heart rate indicators and / or respiratory indicators, it is usually necessary to collect data for a certain period of time for analysis and processing before the final output indicators can be obtained. Therefore, for this type of indicator, an embodiment of the present invention proposes a unified post-processing scheme, that is, real-time synchronous update of the heart rate waveform and / or respiratory waveform, and smoothing of the data within the period, which can effectively reduce the problem of large overall output fluctuations caused by single-frame output errors, thereby improving the accuracy of period prediction. The post-processing scheme will be introduced in detail below.

[0063] Step 1402, based on the sparse matrix, denoise the indicator data to obtain a denoised prediction value of the current frame, and store the denoised prediction value of the current frame in a preset queue, wherein the denoised prediction value includes a heart rate prediction value and / or a breathing prediction value, and the sparse matrix is ​​initialized using the sequence number of the current frame as a parameter.

[0064] Specifically, after obtaining the predicted value output by the single-channel source network or the multi-channel source network (ie, the indicator data of the current frame), it can be denoised using a sparse matrix that is updated in real time to obtain a denoised predicted value of the current frame.

[0065] Further, step 1402 specifically includes: Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame; Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set; Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

[0066] Specifically, denoising is achieved by using the real-time updated sparse matrix D and the cumulative integral set processing composed of the network output. Assume that the network continuous frame prediction result is , sum up the integral to get , follow-up And so on. Assume that the current frame is Frame, the real-time updated sparse matrix is ​​used The frame number of the frame is initialized as a parameter The matrix , the formula is as follows:

[0067] in, is a A diagonal matrix with all diagonal elements being 1. To find the inverse matrix function, is the amplification factor, which is set to 100 in the embodiment of the present invention. is a special The "diagonal matrix" is filled diagonally with the values ​​1, -2, and 1 as shown below:

[0068] Then, use The sparse matrix and The cumulative integral set of Multiplying them, we get The denoising prediction value set of , the last value in the set is the denoising prediction value of the current frame.

[0069] Step 1403: draw a heart rate waveform graph based on the heart rate prediction value, and / or draw a respiration waveform graph based on the respiration prediction value.

[0070] Specifically, after obtaining the denoising prediction value of the current frame, it can be directly drawn as the point value of the real-time waveform, thereby obtaining a real-time heart rate and breathing waveform. It should be understood that each time sparse denoising is performed, the prediction value of each frame will be updated, so that the updated value of a certain number of frames can be selected to adjust the waveform to make the waveform smoother.

[0071] Step 1404, when the denoised prediction values ​​stored in the preset queue are full of cycles, the denoised prediction values ​​stored in the preset queue are comprehensively processed to obtain the health indicator.

[0072] Specifically, for the cumulative integral set , taking the sparse matrix After denoising, the denoising prediction value of the current frame is obtained , save it to the preset queue When the queue After a full cycle, the elements stored in the queue reach Frame (the embodiment of the present invention sets 180 frames), which will be used as a cycle prediction and subsequent comprehensive processing operations to obtain the final heart rate and respiratory output indicators.

[0073] Furthermore, step 1404 specifically includes: Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values; Based on the processed predicted value, power spectrum density calculation is performed to obtain a calculation result; Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

[0074] Specifically, when the queue After the stored elements are fully cycled, first Butterworth filtering is used for smoothing. This step provides a smooth transition between the pass frequency and the cutoff frequency without "sharp" edge data. Then, conventional power spectral density calculation is performed. Power spectral density (PSD) plays an important role in heart rate signal analysis, especially in the study of heart rate variability (HRV). HRV refers to the change in the interval between consecutive heartbeats, which reflects the regulation of cardiac activity by the autonomic nervous system. Therefore, PSD can reveal the energy distribution of HRV signals in different frequency ranges, thus providing a quantitative means to understand the autonomic regulatory function of the heart.

[0075] The specific calculation process is: After Butterworth filtering and smoothing, we get , the power spectrum density can be calculated by Fourier transform. In the embodiment of the present invention, the Fourier transform adopts a 256-cycle length, so the first 180 elements are used Fill, the rest are initialized with 0, and then Fourier transform is performed to obtain a periodic set containing complex elements , a total of 256 output sets, then directly retain the head and tail values, and the middle 254 pairs of elements are used as the real and imaginary parts of the complex number (for example As the real part, As the imaginary part), we can obtain the set , this set is the obtained power spectrum density output, in which each element value is the output of heart rate or breathing index at different frequencies. Then, using the frequency range of heart rate or breathing routine For example, the heart rate frequency range can be Set to 1.0, Set to 2.5, using this frequency range Filter and get the corresponding output Finally, The maximum value is the final heart rate or breathing value.

[0076] Table 1 is the test result of the unified post-processing solution of the present invention. A heart rate nail monitor was worn during the system experience. The data comparison between the two is shown in the following table: Table 1 Test results

[0077] Based on the above Table 1, it can be seen that among the 23 test records, only two have large differences in predicted output amplitudes, which shows that the non-cooperative health monitoring method provided by the embodiment of the present invention has a certain degree of accuracy.

[0078] Based on any of the above embodiments, the embodiment of the present invention provides a non-cooperative end-to-end health monitoring method combined with face liveness detection. In order to reduce the cost of hardware facilities, computing resource requirements and facilitate deployment and application, the embodiment of the present invention realizes health monitoring by simply combining with the current monitoring system without the need for additional hardware equipment. Liveness detection technology is adopted in the deep learning algorithm to meet the needs of non-cooperative scenarios, and multi-target tracking technology is adopted to complete the periodic homologous input prediction of multiple face IDs bound to each other, and to adopt key point local area selection and face correction to reduce the interference of invalid or other people's facial pixels, and alleviate the impact of pixel dislocation; at the same time, single and multi-channel input data separation and network prediction unified post-processing analysis scheme are adopted to effectively improve the prediction accuracy of the algorithm under cross-domain input sources. Finally, in order to improve the prediction accuracy of the algorithm, the embodiment of the present invention proposes a set of rigorous continuous output indicator signal post-processing schemes, and adopts real-time update and filtering and smoothing processing for the network prediction output, thereby effectively reducing the impact of single-frame errors, and at the same time proposes a scheme for how to display heart rate and respiratory waveforms in real time.

[0079] Figure 4 is a flow chart of the non-cooperative end-to-end health monitoring method combined with face liveness detection provided by the present invention, such as Figure 4 As shown, the method includes: Step S1, first perform face detection on the currently acquired video frame, and then perform liveness detection on the detected face area. Compared with face detection, liveness detection is a simple two-classification network. It predicts the liveness confidence of the face area image output by face detection. If it is greater than the set threshold, it performs subsequent face correction and other preprocessing. Otherwise, it jumps to the next frame for system reasoning.

[0080] Step S2, based on the face key point technology, key point detection is performed on the face area image. The key point technology predicts the positions of key areas such as eyebrows, eyes, nose, mouth, facial contour, etc. in the face image to obtain face key points.

[0081] Step S3, in actual scenarios, the problem of multiple faces interfering with each other can easily lead to large prediction errors. Based on the face multi-target tracking technology, each face ID is bound to the corresponding periodic homologous input. At the same time, the key point area interception technology is used to use the left cheek area or the right cheek area as the effective area for health monitoring prediction. In addition, the embodiment of the present invention also uses the key point correction technology to align the face images of adjacent frames to reduce the impact of the large misalignment of the face images of adjacent frames.

[0082] In step S4, the number of channels of the input source (i.e., the input image) is distinguished, and the corresponding network model is selected for inference and prediction according to the number of channels of the input source, and then a unified post-processing solution is adopted.

[0083] The unified post-processing solution proposed in the embodiment of the present invention is a detailed and efficient data smoothing processing solution, which can greatly reduce the problem of large overall output fluctuations caused by single-frame output errors. At the same time, a solution for how to display heart rate and respiratory waveforms in real time is also proposed. The core operation is to adopt a continuous value sparse denoising and real-time updating solution for the heart rate and respiratory indicators predicted by the network, so as to obtain the heart rate and respiratory variability characteristic values, and retain the output of each frame. After the heart rate and respiratory characteristic value queues filled with a cycle are smoothed by Butterworth filtering, the power spectral density solution is adopted for quantization processing, so as to obtain the final heart rate and respiratory output indicators of this cycle.

[0084] Figure 5 : is a schematic diagram of the process of post-processing the heart rate and / or breathing index data provided by the present invention, such as Figure 5 As shown, the specific processing process is divided into denoising and smoothing. The queue is full and the operation above is predictive denoising, drawing a real-time indicator waveform. The queue is full and the operation below is to uniformly smooth and filter the denoised prediction value of one cycle to obtain the corresponding health indicator value. Specifically, after obtaining the prediction result of the network output, the result can be cached in the queue, and then the cumulative integral set composed of the real-time updated sparse matrix and the network output is used for denoising to obtain the denoised prediction value of the current frame, which is directly drawn as the point value of the real-time waveform to obtain the real-time heart rate and breathing waveform. At the same time, the denoised prediction value of the current frame is cached in another queue. When the queue is full of a cycle, that is, the elements stored in the queue reach the preset number of frames (such as 180 frames), the elements in the queue can be used as a cycle prediction and subsequent filtering operations can be performed. In the filtering operation process, the Butterworth filter is first smoothed, and then the power spectrum density is used for dequantization. Finally, the comment range of the heart rate or breathing is used for filtering to obtain the final heart rate and breathing output indicators of this cycle.

[0085] It is understandable that the continuous frame prediction output optimization scheme of health indicators and the real-time waveform drawing are lacking in the current deep learning neural network health detection scheme. The embodiment of the present invention combines traditional signal processing algorithms and statistical analysis based on changes in heart rate and respiratory cycle frequency to propose an effective heart rate and respiratory periodic data post-processing scheme. It uses dynamic adjustment of sparse matrices and historical data denoising to draw heart rate and respiratory waveforms in real time, thereby ensuring the continuity of the peak and trough periodic changes of the drawn waveform. At the same time, it also proposes to use Butterworth filtering operation to smooth the periodic output data, and then use the power spectral density quantization scheme as the core, combined with the frequency range of heart rate and respiratory cycle changes, to perform maximum value filtering on the quantization results to obtain the final output.

[0086] The method provided in an embodiment of the present invention adopts a scheme for health monitoring prediction based on correction of key points of adjacent frames and local effective areas of the cheek, which can suppress the influence of excessive interference pixels on network feature extraction. At the same time, a set of periodic indicator optimization schemes based on power spectral density quantization is formulated to effectively improve the accuracy of health indicator prediction.

[0087] The health monitoring device provided by the present invention is described below. The health monitoring device described below and the health monitoring method described above can be referenced to each other.

[0088] Based on any of the above embodiments, Figure 6 is a schematic diagram of the structure of the health monitoring device provided by the present invention, such as Figure 6 As shown, the device comprises: The face detection unit 610 is used to perform face detection on the currently acquired video frame to obtain a face region image of the current frame; A key point detection unit 620 is used to perform key point detection on the face area image to obtain face key points; A region determination unit 630, configured to determine a local face region from the face region image based on the face key points; The health monitoring unit 640 is used to align the local face area of ​​the current frame with the local face area of ​​the previous frame based on the target key points in the face area image, and apply the aligned local face area to perform health monitoring to obtain health indicators.

[0089] The device provided by the embodiment of the present invention uses face detection technology to accurately extract the face area image from the currently collected video frame, and uses key point detection technology to further locate the key points of the face in the face area image, which can provide a solid foundation for subsequent health monitoring. Based on the detected face key points, the local face area is determined from the face area image, thereby suppressing the influence of too many interfering pixels on feature extraction. The target key points in the face area image are used to align the local face area of ​​the current frame with the local face area of ​​the previous frame, which can significantly alleviate the influence of pixel misalignment. By performing health monitoring on the aligned local face area, the deep features of the face can be captured more accurately, thereby reducing the prediction deviation caused by face movement and improving the accuracy and effectiveness of health monitoring.

[0090] Based on any of the above embodiments, the key point detection unit 620 is specifically used for: Based on a liveness detection model, performing liveness detection on the face region image to obtain a detection result, wherein the liveness detection model is a classification network trained by using live face images as positive samples and non-live false face images and / or background images as negative samples; When the detection result indicates that there is a living body, key point detection is performed on the face area image to obtain face key points.

[0091] Based on any of the above embodiments, the region determining unit 630 includes: A key point determination subunit, configured to determine the face key points of the subject under test from the face key points based on the face identification of the subject under test when the face region image includes multiple faces; The local area determination subunit is used to determine the local face area from the face area image based on the face key points of the object being measured.

[0092] Based on any of the above embodiments, the local area determination subunit is specifically used for: In a case where the face region image includes a left face region and a right face region of the subject, based on the confidences of the face key points of the subject, respectively determining the mean confidence values ​​of all face key points in the left face region and the mean confidence values ​​of all face key points in the right face region; A side area with a higher confidence mean value is selected from the left side face area and the right side face area as the local face area.

[0093] Based on any of the above embodiments, the health monitoring unit 640 includes: A single-channel monitoring unit, configured to, when the aligned local face region is a single-channel image, perform health monitoring on the aligned local face region based on a single-channel source network to obtain a health indicator; The multi-channel monitoring unit is used to perform health monitoring on the aligned local face area based on a multi-channel source network to obtain health indicators when the aligned local face area is a multi-channel image.

[0094] Based on any of the above embodiments, when the health indicator includes a heart rate indicator and / or a breathing indicator, the health monitoring unit 640 includes: A prediction subunit, configured to use the aligned local face area to perform health monitoring to obtain indicator data of a current frame, wherein the indicator data includes heart rate data and / or respiration data; A denoising subunit, configured to denoise the indicator data based on a sparse matrix to obtain a denoised prediction value of the current frame, and store the denoised prediction value of the current frame in a preset queue, wherein the denoised prediction value includes a heart rate prediction value and / or a breathing prediction value, and the sparse matrix is ​​initialized using the sequence number of the current frame as a parameter; A drawing subunit, used for drawing a heart rate waveform graph based on the heart rate prediction value, and / or drawing a breathing waveform graph based on the breathing prediction value; The post-processing subunit is used to perform comprehensive processing on the denoised prediction values ​​stored in the preset queue to obtain the health indicator when the denoised prediction values ​​stored in the preset queue are full of cycles.

[0095] Based on any of the above embodiments, the denoising subunit is specifically used for: Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame; Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set; Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

[0096] Based on any of the above embodiments, the post-processing subunit is specifically used for: Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values; Based on the processed predicted value, power spectrum density calculation is performed to obtain a calculation result; Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

[0097] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the health monitoring method, which includes: performing face detection on the currently acquired video frame to obtain a face area image of the current frame; performing key point detection on the face area image to obtain face key points; based on the face key points, determining a face local area from the face area image; based on the target key points in the face area image, aligning the face local area of ​​the current frame with the face local area of ​​the previous frame, and applying the aligned face local area to perform health monitoring to obtain health indicators.

[0098] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the health monitoring method provided by the above methods, which includes: performing face detection on the currently acquired video frame to obtain a face area image of the current frame; performing key point detection on the face area image to obtain face key points; based on the face key points, determining a local face area from the face area image; based on the target key points in the face area image, aligning the local face area of ​​the current frame with the local face area of ​​the previous frame, and applying the aligned local face area to perform health monitoring to obtain health indicators.

[0100] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the health monitoring method provided by the above-mentioned methods, the method comprising: performing face detection on a currently acquired video frame to obtain a face area image of the current frame; performing key point detection on the face area image to obtain face key points; based on the face key points, determining a local face area from the face area image; based on the target key points in the face area image, aligning the local face area of ​​the current frame with the local face area of ​​the previous frame, and applying the aligned local face area to perform health monitoring to obtain health indicators.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A health monitoring method, characterized in that: include: Perform face detection on the currently acquired video frame to obtain the face area image of the current frame; Performing key point detection on the face area image to obtain face key points; Based on the facial key points, determining a local facial region from the facial region image; Based on the target key points in the face area image, the face local area of ​​the current frame is aligned with the face local area of ​​the previous frame, and the aligned face local area is used for health monitoring to obtain a health index.

2. The health monitoring method according to claim 1, characterized in that: The step of performing key point detection on the face region image to obtain face key points includes: Based on a liveness detection model, performing liveness detection on the face region image to obtain a detection result, wherein the liveness detection model is a classification network trained by using live face images as positive samples and non-live false face images and / or background images as negative samples; When the detection result indicates that there is a living body, key point detection is performed on the face area image to obtain face key points.

3. The health monitoring method according to claim 1, characterized in that: The step of determining a local face region from the face region image based on the face key points includes: In the case where the face region image includes a plurality of faces, based on the face identification of the subject, determining the face key points of the subject from the face key points; Based on the facial key points of the detected object, a local facial area is determined from the facial area image.

4. The health monitoring method according to claim 3, characterized in that: The method of determining a local face area from the face area image based on the face key points of the object under test includes: In a case where the face region image includes a left face region and a right face region of the subject, based on the confidences of the face key points of the subject, respectively determining the mean confidence values ​​of all face key points in the left face region and the mean confidence values ​​of all face key points in the right face region; A side area with a higher confidence mean value is selected from the left side face area and the right side face area as the local face area.

5. The health monitoring method according to claim 1, characterized in that: The application performs health monitoring on the aligned local face area to obtain health indicators, including: In the case where the aligned local face region is a single-channel image, based on a single-channel source network, the aligned local face region is used for health monitoring to obtain a health index; In the case where the aligned local face region is a multi-channel image, the aligned local face region is used for health monitoring based on a multi-channel source network to obtain a health indicator.

6. The health monitoring method according to any one of claims 1 to 5, characterized in that: In the case where the health indicator includes a heart rate indicator and / or a breathing indicator, the health monitoring is performed on the aligned local face area to obtain the health indicator, including: Applying the aligned local face area to perform health monitoring to obtain indicator data of the current frame, wherein the indicator data includes heart rate data and / or breathing data; Based on a sparse matrix, the indicator data is denoised to obtain a denoised prediction value of the current frame, and the denoised prediction value of the current frame is stored in a preset queue, wherein the denoised prediction value includes a heart rate prediction value and / or a breathing prediction value, and the sparse matrix is ​​initialized using the sequence number of the current frame as a parameter; Draw a heart rate waveform graph based on the heart rate prediction value, and / or draw a respiration waveform graph based on the respiration prediction value; When the denoising prediction values ​​stored in the preset queue are full of cycles, the denoising prediction values ​​stored in the preset queue are comprehensively processed to obtain the health indicator.

7. The health monitoring method according to claim 6, characterized in that: The step of denoising the indicator data based on the sparse matrix to obtain a denoised prediction value of the current frame includes: Determine a cumulative integral set based on the indicator data of the current frame and the indicator data of each historical frame; Multiplying the sparse matrix by the accumulated integral set to obtain a denoised prediction value set; Based on the denoising prediction value set, a denoising prediction value of the current frame is determined.

8. The health monitoring method according to claim 6, characterized in that: The comprehensive processing of the denoised prediction values ​​stored in the preset queue to obtain the health indicator includes: Performing filtering and smoothing processing on the denoised prediction values ​​stored in the preset queue to obtain processed prediction values; Based on the processed predicted value, power spectrum density calculation is performed to obtain a calculation result; Based on a preset frequency range, the calculation result is filtered to obtain the heart rate index and / or the breathing index.

9. A health monitoring device, characterized in that: include: A face detection unit is used to perform face detection on the currently acquired video frame to obtain a face region image of the current frame; A key point detection unit, used to perform key point detection on the face area image to obtain face key points; A region determination unit, configured to determine a local face region from the face region image based on the face key points; The health monitoring unit is used to align the local face area of ​​the current frame with the local face area of ​​the previous frame based on the target key points in the face area image, and apply the aligned local face area to perform health monitoring to obtain health indicators.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the health monitoring method according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the health monitoring method according to any one of claims 1 to 8 is implemented.

Images