Non-contact blood pressure monitoring system, readable storage medium and electronic device

Abstract

The application provides a non-contact blood pressure monitoring system, a readable storage medium and an electronic device, and relates to the technical field of non-contact blood pressure monitoring.The application carries out spatial filtering and time domain filtering on collected data, effectively extracts color information hidden in face and palm videos, constructs a feature representation block of each column representing a fixed area of pixel value change over time, and each row represents a complete RIS sequence at different times, and cooperates with the attention mechanism designed in the application to realize PPG signal enhancement; meanwhile, the network pays attention to the relationship between channels, time sequence information and spatial importance distribution, so that the network can more effectively learn blood pressure related features, overcome the shortcomings of existing research, and be more accurate than other deep learning models.

Description

Technical Field

[0001] This invention relates to the field of non-contact blood pressure monitoring technology, specifically to a non-contact blood pressure monitoring system, a readable storage medium, and an electronic device. Background Technology

[0002] Blood pressure is an important physiological parameter of the human body, which can reflect the heart's pumping function, peripheral vascular resistance, and blood volume. Its measurement is the main method for diagnosing diseases such as hypertension.

[0003] Most existing non-contact blood pressure estimation methods are based on PPG features combined with individual features. The main methods for PPG feature extraction are waveform analysis and deep learning.

[0004] However, due to the complexity of factors affecting blood pressure, waveform analysis cannot obtain deep-seated correlation features hidden in the region of interest. Furthermore, conventional deep learning methods are ill-suited for blood pressure problems, exhibiting weak learning capabilities for blood pressure-related features. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a non-contact blood pressure monitoring system that solves the problem that existing deep learning-based non-contact blood pressure estimation methods have weak learning capabilities for blood pressure-related features.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] Firstly, a non-contact blood pressure monitoring system is provided, the system comprising:

[0010] The target image acquisition module is used to acquire target images of the region of interest.

[0011] The data preprocessing module is used to generate feature representation blocks containing blood pressure-related information based on the target image;

[0012] The blood pressure estimation module is used to take the feature representation block and individual basic attributes as input to the trained blood pressure estimation model to obtain the blood pressure estimate.

[0013] Furthermore, the target image acquisition module includes:

[0014] Video capture unit, used to capture visible light images;

[0015] The region of interest detection unit is used to identify the region of interest from the visible light image and obtain the target image of the region of interest;

[0016] The image extraction unit is used to convert the target image of the region of interest into a single-frame image.

[0017] Furthermore, the regions of interest are the nose, forehead, and palm areas.

[0018] Furthermore, the data preprocessing module includes:

[0019] The image stitching unit is used to stitch the nose region image and the forehead region image to obtain a face region image;

[0020] The spatial filtering unit is used to perform Gaussian pyramid processing on the face region image and the palm region image respectively, and take the top two layers of the image to obtain the feature maps RIS1 and RIS2 of the face region image and the feature maps RIS3 and RIS4 of the palm region image.

[0021] The feature representation block generation unit is used to straighten the feature maps RIS1, RIS2, RIS3, and RIS4 respectively, and stitch the video frames row by row to obtain feature representation blocks TSM1, TSM2, TSM3, and TSM4. Each column of the feature representation block represents the change of pixel values ​​in a fixed region over time, and each row represents a complete RIS sequence at different times.

[0022] The time-domain filtering unit is used to apply Butterworth filters to TSM1, TSM2, TSM3, and TSM4 to remove noise outside the preset frequency band;

[0023] The feature representation block supplement unit is used to construct two more feature representation blocks, TSM5 and TSM6, based on feature representation blocks TSM1, TSM2, TSM3, and TSM4, where TSM5 = TSM1 - TSM3 and TSM6 = TSM2 - TSM4.

[0024] Furthermore, the blood pressure estimation model is a deep regression network based on an attention module.

[0025] Furthermore, the attention module reduces the channel dimension of the features to 1 / k of the input through the first fully connected layer, and upgrades the features to the original dimension through the second fully connected layer, obtaining channel attention maps, temporal attention maps, and spatial attention maps. The generated attention maps are multiplied by the original input and finally added together to obtain the final output attention weight map.

[0026] Furthermore, the deep regression network is constructed using ResNet as its backbone.

[0027] Furthermore, the blood pressure estimation module also includes:

[0028] Blood pressure classification unit, used to classify blood pressure according to blood pressure classification standards.

[0029] In a second aspect, a computer-readable storage medium is provided that stores a computer program for non-contact blood pressure monitoring, wherein the computer program causes a computer to perform the following steps:

[0030] Acquire target images of the region of interest;

[0031] Generate a feature representation block containing blood pressure-related information based on the target image;

[0032] The feature representation blocks and basic individual attributes are used as inputs to the trained blood pressure estimation model to obtain blood pressure estimates.

[0033] Thirdly, an electronic device is provided, comprising:

[0034] One or more processors;

[0035] Memory; and

[0036] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including steps for performing the following:

[0037] Acquire target images of the region of interest;

[0038] Generate a feature representation block containing blood pressure-related information based on the target image;

[0039] The feature representation blocks and basic individual attributes are used as inputs to the trained blood pressure estimation model to obtain blood pressure estimates.

[0040] (III) Beneficial Effects

[0041] This invention provides a non-contact blood pressure monitoring system. Compared with the prior art, it has the following advantages:

[0042] This invention performs spatial and temporal filtering on the collected data to effectively extract color information hidden in facial and hand videos. It constructs a feature representation block where each column represents the change of pixel values ​​in a fixed region over time, and each row represents a complete RIS sequence at different times. Combined with the attention mechanism designed in this invention, it achieves PPG signal enhancement. At the same time, it makes the network focus on the relationships between channels, temporal information, and spatial importance distribution, enabling the network to learn blood pressure-related features more effectively, overcoming the shortcomings of existing research, and is more accurate than other deep learning models. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a system block diagram of Embodiment 1 of the present invention;

[0045] Figure 2 This is a schematic diagram of the process for generating a feature representation block containing blood pressure-related information in Embodiment 1 of the present invention;

[0046] Figure 3 This is a schematic diagram of the Gaussian pyramid processing in Embodiment 1 of the present invention;

[0047] Figure 4 This is a schematic diagram of the attention module principle in Embodiment 1 of the present invention;

[0048] Figure 5 This is a schematic diagram of the blood pressure estimation model in Embodiment 1 of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] This application provides a non-contact blood pressure monitoring system, which solves the problem that existing deep learning-based non-contact blood pressure estimation methods have weak learning ability for blood pressure-related features.

[0051] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows:

[0052] This invention performs spatial and temporal filtering on the collected data to effectively extract color information hidden in facial and hand videos. It constructs a feature representation block where each column represents the change of pixel values ​​in a fixed region over time, and each row represents a complete RIS sequence at different times. Combined with the attention mechanism designed in this invention, it achieves PPG signal enhancement. At the same time, it makes the network focus on the relationships between channels, temporal information, and spatial importance distribution, enabling the network to learn blood pressure-related features more effectively, overcoming the shortcomings of existing research, and is more accurate than other deep learning models.

[0053] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0054] Example 1:

[0055] like Figure 1 As shown, the present invention provides a non-contact blood pressure monitoring system, the system comprising:

[0056] The target image acquisition module is used to acquire target images of the region of interest.

[0057] The data preprocessing module is used to generate feature representation blocks containing blood pressure-related information based on the target image;

[0058] The blood pressure estimation module is used to take the feature representation block and individual basic attributes as input to the trained blood pressure estimation model to obtain the blood pressure estimate.

[0059] The beneficial effects of this embodiment are:

[0060] 1) In this embodiment of the invention, spatial and temporal filtering is performed on the collected data to effectively extract color information hidden in facial and hand videos. A feature representation block is constructed in which each column represents the change of pixel values ​​of a fixed region over time, and each row represents a complete RIS sequence at different times. Combined with the attention mechanism designed in this invention, PPG signal enhancement is achieved. At the same time, the network focuses on the relationship between channels, temporal information and spatial importance distribution, so that the network can learn blood pressure-related features more effectively, overcome the shortcomings of existing research, and is more accurate than other deep learning models.

[0061] The implementation process of the embodiments of the present invention will be described in detail below:

[0062] S1. Acquire target images of the region of interest through the target image acquisition module.

[0063] Specifically, a video acquisition unit can be used to capture visible light images; the video acquisition unit is a visible light camera; for example, it can capture 12 consecutive seconds of video containing a face and a hand. Then, a region of interest (ROI) detection unit identifies the ROI from the visible light images to obtain the target image of the ROI. For example, because the forehead and nose are rich in blood vessels, they reflect rPPG signals more clearly than other facial areas. Since PTT information can be obtained by the difference in PPG between two different parts of the body, the hand area video is chosen as another source of PPG signal. The ROI detection unit can then use a YOLOv3 detector to calibrate and extract the forehead, nose, and hand areas as ROIs in the video. Finally, an image extraction unit converts the target image of the ROI into a single frame image as input data for the data preprocessing stage.

[0064] S2. The data preprocessing module generates a feature representation block containing blood pressure-related information based on the target image.

[0065] Specifically, such as Figure 2 As shown, firstly, the image stitching unit can be used to stitch the nose region image ROI1 and the forehead region image ROI2 to obtain the face region image ROI3 = (h, w, c), where h, w, and c represent the height, width, and number of channels, respectively. Then, the spatial filtering unit performs Gaussian pyramid processing on the face region image ROI3 and the palm region image ROI4 = (h, w, c) respectively, and the top two layers of the image are taken from each. Taking ROI3 as an example, the detailed process is as follows... Figure 3 As shown, feature maps RIS1 and RIS2 (Region of Interest Sequence) of the face region image are obtained, with shapes of (h / 8, w / 8, c) and (h / 4, w / 4, c) respectively. Similarly, feature maps RIS3 and RIS4 of the palm region image can also be obtained.

[0066] Subsequently, the feature maps RIS1, RIS2, RIS3, and RIS4 are straightened using the feature representation block generation unit, and the video frames are stitched together row by row to obtain the feature representation blocks TSM1, TSM2, TSM3, and TSM4 (Temporal Spatial Feature Map). Each column of the feature representation block represents the change of pixel values ​​in a fixed region over time, and each row represents a complete RIS sequence at different times.

[0067] Since more attention needs to be paid to these rows and columns to obtain information on time changes and spatial importance distribution, and the obtained feature representation blocks contain noise such as illumination and motion, it is necessary to use the time domain filtering unit to apply Butterworth filters to TSM1, TSM2, TSM3, and TSM4 to remove noise outside the preset frequency band; the preset frequency band can be set to [0.75, 2.5] Hz.

[0068] In addition, in order to obtain feature representation blocks rich in PTT information, two more feature representation blocks TSM5 and TSM6 are constructed based on feature representation blocks TSM1, TSM2, TSM3, and TSM4 using feature representation block supplementation units, where TSM5 = TSM1 - TSM3 and TSM6 = TSM2 - TSM4.

[0069] Finally, six multi-scale feature representation blocks are obtained from the input video data and used as input to the blood pressure estimation network.

[0070] S3. Using the blood pressure estimation module, the feature representation block and individual basic attributes are used as inputs to the trained blood pressure estimation model to obtain the blood pressure estimate.

[0071] To enhance meaningful channel, temporal, and spatial information, focus on inter-channel relationships, temporal information, and spatial importance, and suppress irrelevant information, this embodiment designs an attention module BPAB that significantly improves blood pressure estimation performance with only a limited number of parameters added.

[0072] like Figure 4 As shown, the attention module reduces the channel dimension of the features to 1 / k of the input through the first fully connected layer, and upgrades the features to the original dimension through the second fully connected layer, obtaining the channel attention map Mc(F), the temporal attention map Mt(F), and the spatial attention map Ms(F). The generated attention maps are multiplied by the original input and finally added together to obtain the final output attention weight map F'.

[0073] The corresponding equations are as follows:

[0074]

[0075]

[0076]

[0077]

[0078] This indicates element-wise multiplication. This indicates element-wise addition.

[0079] In practice, the input to the blood pressure estimation model is the multi-scale feature representation block obtained after preprocessing and basic individual attributes (such as gender, age, weight, height, and BMI). Since ResNet can solve the gradient vanishing problem in deep networks, and residual learning has powerful semantic feature extraction and ensemble capabilities, the network design is as follows: Figure 5 As shown, this embodiment uses a regression network based on the ResNet backbone combined with the aforementioned attention mechanism to construct a blood pressure estimation model, which outputs systolic and diastolic blood pressure to achieve continuous blood pressure estimation. Furthermore, blood pressure grading can be further implemented based on blood pressure grading standards.

[0080] Model performance validation:

[0081] As shown in Table 1, the design is compared with popular attention modules SE-block and CBAM on a self-collected dataset based on mean absolute error (MAE), root mean square error (RMSE), and correlation index (Pearson correlation coefficient ρ). The performance of the designed temporal attention branch and spatial attention branch is also compared.

[0082] Table 1

[0083]

[0084] Among them, BPAB T This indicates that, based on SE, only a temporal attention branch is added; BPAB S This means that, based on SE, only a spatial attention branch is added.

[0085] As can be seen from the table above, the performance of the attention module BPAB designed based on the attention module SE-block in this embodiment has been greatly improved, and it can extract blood pressure-related features more effectively.

[0086] Example 2:

[0087] A computer-readable storage medium storing a computer program for non-contact blood pressure monitoring, wherein the computer program causes a computer to perform the following steps:

[0088] Acquire target images of the region of interest;

[0089] Generate a feature representation block containing blood pressure-related information based on the target image;

[0090] The feature representation blocks and basic individual attributes are used as inputs to the trained blood pressure estimation model to obtain blood pressure estimates.

[0091] Example 3:

[0092] An electronic device, comprising:

[0093] One or more processors;

[0094] Memory; and

[0095] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including steps for performing the following:

[0096] Acquire target images of the region of interest;

[0097] Generate a feature representation block containing blood pressure-related information based on the target image;

[0098] The feature representation blocks and basic individual attributes are used as inputs to the trained blood pressure estimation model to obtain blood pressure estimates.

[0099] It is understood that the computer-readable storage medium and electronic device provided in the embodiments of the present invention correspond to the above-described non-contact blood pressure monitoring system. The explanations, examples, and beneficial effects of the relevant contents can be referred to the corresponding contents in the non-contact blood pressure monitoring system, and will not be repeated here.

[0100] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0101] 1) This invention performs spatial and temporal filtering on the collected data to effectively extract color information hidden in facial and hand videos. It constructs a feature representation block where each column represents the change of pixel values ​​in a fixed region over time, and each row represents a complete RIS sequence at different times. Combined with the attention mechanism designed in this invention, it achieves PPG signal enhancement. At the same time, it makes the network pay attention to the relationship between channels, temporal information and spatial importance distribution, so that the network can learn blood pressure-related features more effectively, overcome the shortcomings of existing research, and is more accurate than other deep learning models.

[0102] 2) This invention further classifies the estimated blood pressure values ​​in the blood pressure regression task into levels to achieve blood pressure classification.

[0103] It should be noted that, through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the prior art, can be embodied in the form of software products. These computer software products can be stored in computer-readable storage media, such as ROM / RAM, magnetic disks, optical disks, etc., and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain portions of the embodiments. In this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A non-contact blood pressure monitoring system, characterized in that, The system includes: The target image acquisition module is used to acquire target images of the region of interest. The data preprocessing module is used to generate feature representation blocks containing blood pressure-related information based on the target image; The blood pressure estimation module is used to take the feature representation block and individual basic attributes as input to the trained blood pressure estimation model to obtain the blood pressure estimate. The regions of interest are the nose, forehead, and palm areas; The data preprocessing module includes: The image stitching unit is used to stitch the nose region image and the forehead region image to obtain a face region image; The spatial filtering unit performs Gaussian pyramid processing on the face region image and the hand region image respectively, and extracts the top two layers of the image to obtain the feature map of the face region image. and and feature maps of the palm area. and ; Feature representation block generation unit, used for feature map , , , The video frames are straightened individually and then stitched together row by row to obtain feature representation blocks. , , , In this block, each column represents the change of pixel values ​​in a fixed region over time, and each row represents a complete feature at a different time. RIS sequence; The time-domain filtering unit is used for... , , , Apply a Butterworth filter to remove noise outside the preset frequency band; Feature representation block supplementation unit, used for feature representation blocks , , , Construct two more feature representation blocks and ,in, , Six multi-scale feature representation blocks are obtained from the input video data and used as input to the blood pressure estimation network. The preset frequency band is set to [0.75, 2.5] Hz; face region image. ( h,w,c ),in, These represent the height, width, and number of channels, respectively. The blood pressure estimation model is a deep regression network based on an attention module. The attention module reduces the channel dimension of the features to 1 / k of the input through the first fully connected layer, and upgrades the features to the original dimension through the second fully connected layer, resulting in a channel attention map, a temporal attention map, and a spatial attention map. The generated attention maps are multiplied by the original input and then summed to obtain the final output attention weight map. The deep regression network is constructed using ResNet as its backbone.

2. The non-contact blood pressure monitoring system as described in claim 1, characterized in that, The target image acquisition module includes: Video capture unit, used to capture visible light images; The region of interest detection unit is used to identify the region of interest from the visible light image and obtain the target image of the region of interest; The image extraction unit is used to convert the target image of the region of interest into a single-frame image.

3. The non-contact blood pressure monitoring system as described in claim 1, characterized in that, The blood pressure estimation module also includes: Blood pressure classification unit, used to classify blood pressure according to blood pressure classification standards.

Images