A multi-band pulse wave transmission time estimation method, system, terminal and medium

Abstract

The present invention provides a multi-band pulse wave transmission time estimation method, system, terminal and medium, which specifically relates to the field of biomedical engineering technology. The scheme includes: using a rolling exposure camera to scan the pixels of the target skin area line by line to obtain a number of frame images; extracting the pixel data of two target color channels in all frame images, and grouping and sampling the pixel data of each target color channel corresponding to all frame images to construct a sampling point matrix for each target color channel; using each sampling point matrix, obtaining the rPPG signal corresponding to each target color channel; based on the time relationship between the peaks of the two rPPG signals in the same time window, obtaining the pulse wave transmission time. The scheme realizes more continuous and accurate multi-band PTT estimation by using a rolling exposure camera set to a defocused state to enhance the frame rate of the rPPG signal, effectively reducing the equipment cost and equipment power consumption.

Description

Technical Field

[0001] The present invention relates to the field of biomedical engineering technology, and in particular to a multi-band pulse wave transmission time estimation method, system, terminal and medium. Background Art

[0002] Common methods for cuffless blood pressure estimation include multi-point pulse transit time (PTT) estimation and multi-band pulse transit time (PTT) estimation. Among them, the multi-point PTT estimation method requires measuring subcutaneous tissues in multiple parts of the body to detect the time difference of the pulse wave from one part of the skin to another part of the skin; the multi-band PTT estimation method only requires measuring a single subcutaneous tissue to detect the time difference of the pulse wave from a certain depth of the skin in the same part to another depth, without measuring additional information such as the physical distance between different parts of the body for blood pressure calibration, which is easier to implement and use.

[0003] Currently, the measurement of multi-band PTT relies on high-frame-rate optical cameras to collect skin information of corresponding body parts, which has the problems of high hardware cost and high equipment power consumption. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a multi-band pulse wave transmission time estimation method, system, terminal and medium, aiming to solve the problems of high hardware cost and high power consumption of equipment for collecting skin information of body parts in the prior art.

[0005] In order to achieve the above object, the present invention provides a multi-band pulse wave transmission time estimation method in a first aspect, comprising:

[0006] The pixels of the target skin area are scanned line by line using a rolling exposure camera to obtain several frames of images;

[0007] Extracting pixel data of two target color channels from all the frame images, and grouping and sampling the pixel data of each target color channel corresponding to all the frame images to construct a sampling point matrix of each target color channel;

[0008] Using each of the sampling point matrices, obtaining an rPPG signal corresponding to each of the target color channels;

[0009] The pulse wave transmission time is obtained based on the time relationship between the peaks of the two rPPG signals in the same time window.

[0010] Optionally, extracting pixel data of two target color channels in all the frame images includes:

[0011] Obtaining a color space corresponding to the frame image;

[0012] Selecting color channels corresponding to two colors of light with the largest wavelength difference in the color space to obtain two target color channels;

[0013] Pixel data of the two target color channels in all frame images are extracted.

[0014] Optionally, grouping and sampling the pixel data of each target color channel corresponding to all the frame images to construct a sampling point matrix of each target color channel includes:

[0015] The pixel data of each target color channel in each frame image are divided into several groups and the average value of each group of pixel data is calculated to obtain the sampling points of each target color channel in each frame image;

[0016] The sampling points in all the frame images corresponding to each of the target color channels are arranged according to sampling positions and exposure times, so as to construct a sampling point matrix for each of the target color channels.

[0017] Optionally, obtaining the rPPG signal corresponding to each of the target color channels by using each of the sampling point matrices includes:

[0018] Arrange all the sampling points in each sampling point matrix in series in time order to obtain a sampling point sequence corresponding to each target color channel;

[0019] Based on the sampling point sequence corresponding to each of the target color channels, an rPPG signal corresponding to each of the target color channels is obtained.

[0020] Optionally, after obtaining the rPPG signal corresponding to each of the target color channels, the method further includes:

[0021] Calculate the mean of the values ​​of all sampling points in the sampling point sequence corresponding to each of the target color channels to obtain the sampling point mean of each of the target color channels;

[0022] Calculate the deviation between the value of each sampling point of the target color channel and the mean of the sampling points, and update the rPPG signal corresponding to each target color channel based on the ratio of the deviation of each sampling point to the mean of the sampling points of the corresponding target color channel.

[0023] Optionally, obtaining the pulse wave transmission time based on the time relationship between the peaks of the two rPPG signals in the same time window includes:

[0024] Calculating timestamps of all said peaks;

[0025] Based on the timestamps of all the peaks, calculating the time delay between the nearest neighbor peaks of the rPPG signals of the two target color channels;

[0026] Based on the average of all said time delays, the pulse wave transit time is obtained.

[0027] Optionally, obtaining the pulse wave transmission time based on an average value of all the time delays comprises:

[0028] The two rPPG signals are defined as a first rPPG signal and a second rPPG signal, and the wavelength of light in a target color channel corresponding to the first rPPG signal is greater than the wavelength of light in a target color channel corresponding to the second rPPG signal;

[0029] Respectively calculating average time intervals between respective peaks of the first rPPG signal and the second rPPG signal to obtain a first average time interval and a second average time interval;

[0030] If the pulse wave transit time is greater than zero, the pulse wave transit time is equal to an average of time delays of all nearest neighbor peaks between the second rPPG signal and the first rPPG signal, and the pulse wave transit time is less than the first average time interval;

[0031] If the pulse wave transit time is less than zero, the pulse wave transit time is equal to the average of the time delays of all nearest neighbor peaks between the first rPPG signal and the second rPPG signal, and the absolute value of the pulse wave transit time is less than the second average time interval.

[0032] A second aspect of the present invention provides a multi-band pulse wave transit time estimation system, the system comprising:

[0033] An image acquisition module, used to scan the pixels of the target skin area line by line using a rolling exposure camera to obtain a plurality of frames of images;

[0034] A sampling point matrix construction module is used to extract pixel data of two target color channels in all the frame images, and group and sample the pixel data of each target color channel corresponding to all the frame images to construct a sampling point matrix for each target color channel;

[0035] An rPPG signal generating module, used to obtain the rPPG signal corresponding to each of the target color channels using each of the sampling point matrices;

[0036] The pulse wave transmission time solving module is used to obtain the pulse wave transmission time based on the time relationship between the peaks of the two rPPG signals in the same time window.

[0037] The third aspect of the present invention provides an intelligent terminal, which includes a memory for storing executable instructions; and a processor for calling and running the executable instructions in the memory to execute any step of the above-mentioned multi-band pulse wave conduction time estimation method.

[0038] A fourth aspect of the present invention provides a computer-readable storage medium having program instructions stored therein. When the program instructions are executed by a processor, any step of the multi-band pulse wave conduction time estimation method described above is implemented.

[0039] Compared with the prior art, the beneficial effects of this solution are as follows:

[0040] The present invention is based on the camera rolling exposure mechanism, takes the average of the values ​​of multiple rows of pixel points in a series of frame images collected as a single sampling point, and uses the sampling points on the two target color channels of all frame images to respectively construct a sampling point matrix, and then constructs a frame rate enhanced rPPG signal based on the sampling point matrix; then, by detecting all the peaks on the rPPG signals of the two target color channels in the same time window, the time delay (i.e., PTT) between the nearest neighbor peaks on the two rPPG signals is calculated, and the average of all PTTs is taken as the final PTT at that moment. It can be seen that the present invention enhances the frame rate of the rPPG signal by using a rolling exposure camera set to a defocused state, constructs a frame rate enhanced rPPG signal, and at the same time, based on the high frame rate rPPG signal, provides a more continuous and accurate multi-band PTT estimation, effectively reducing the device cost and device power consumption. Furthermore, a rolling exposure camera set to a defocused state is used to reduce the influence of phase shift caused by skin chromatic aberration and different spatial positions. At the same time, setting constraints on the time delay PTT between the nearest neighbor peaks on the two rPPG signals in the same time window can effectively avoid deviations caused by noise, which is conducive to further improving the accuracy and effectiveness of the pulse wave transmission time finally obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 It is a flow chart of the multi-band pulse wave transmission time estimation method of the present invention;

[0043] Figure 2 This is a schematic diagram of the rolling exposure principle;

[0044] Figure 3 This is a schematic diagram of the defocus and focus effects of a rolling exposure camera;

[0045] Figure 4 It is a schematic diagram of multi-band PTT measurement of the present invention;

[0046] Figure 5 It is a schematic diagram of the modules of the multi-band pulse wave transmission time estimation system of the present invention;

[0047] Figure 6 It is a schematic diagram of the structure of the intelligent terminal of the present invention. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0049] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0050] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0051] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0052] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0053] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0055] The existing multi-band PTT measurement often uses a high-frame rate optical camera to collect skin area image information and calculate the difference in the same pulse wave transmission time on the remote photoplethysmography (rPPG) signal extracted from different color channels of the image. However, the hardware cost of the high-frame rate optical camera is high and the device power consumption is large. Therefore, the present invention proposes a multi-band pulse wave transmission time estimation method assisted by a rolling exposure camera, which mainly uses a rolling exposure camera set to a defocused state to reduce the influence of the phase shift caused by the skin's own chromatic aberration and different spatial positions, and based on the camera rolling exposure mechanism, the average value of the values ​​of multiple rows of pixel points in a series of frame images collected is used as a single sampling point, and the sampling points on the two target color channels of all frame images are used to construct a sampling point matrix respectively, and then a frame rate enhanced rPPG signal is constructed based on the sampling point matrix. Then, by detecting all the peaks on the rPPG signals of the two target color channels in the same time window, the time delay (i.e., PTT) between the nearest neighbor peaks on the two rPPG signals is calculated, and the average value of all PTTs is taken as the final PTT at that moment. It can be seen that the present invention aims to use a rolling exposure camera set to a defocused state to enhance the frame rate of the rPPG signal, construct a frame rate enhanced rPPG signal, and at the same time provide a more continuous and accurate multi-band PTT estimation based on the high frame rate rPPG signal, thereby effectively reducing the device cost and device power consumption.

[0056] The embodiment of the present invention provides a multi-band pulse wave transmission time estimation method, which is deployed on electronic devices such as computers and servers, and is applied to the scenario of measuring the image of the skin area using a rolling exposure camera, and using the rPPG signal extracted from different color channels in the color space to which the image belongs to perform PTT estimation. Specifically, Figure 1 As shown, the steps of the method in this embodiment include:

[0057] Step S100: Scan the pixels of the target skin area line by line using a rolling exposure camera to obtain a plurality of frames of images.

[0058] Specifically, rolling exposure (or rolling shutter exposure) is based on the rolling shutter mechanism to expose each row of pixels in the image in chronological order, continuously capture a series of frame images in a very short period of time, and achieve a higher image acquisition frame rate than global exposure by compressing the imaging time difference between each frame image, but it still cannot meet the requirements of accurate multi-band PTT acquisition. The schematic diagram of the rolling exposure principle is shown in the figure below. Figure 2 As shown. A frame image is collected within a frame period (i.e. 1 / frame rate). Pixel reset is required before each frame image is collected to ensure that the light of each color channel is not disturbed when each frame image is collected. During the exposure stage, the shutter of the camera opens to allow light to enter the photosensitive element (such as CMOS) and accumulate charge. In rolling exposure, charge transfer may be continuous or staged, allowing the device to capture image data of multiple exposure stages at different time intervals. Charge transfer is to transfer the charge accumulated on the photosensitive element to a device such as a readout register, so that the primary color of the color space emits light of the corresponding color. Then, the pixel data is read from the photosensitive element to generate a frame image within the current frame period.

[0059] Based on the above-mentioned rolling exposure principle, when actually using a rolling exposure camera with a rolling exposure function to collect frame images, the desired image quality or effect can be achieved by adjusting exposure parameters, charge transfer, pixel reading and other processes. Figure 2 The different color blocks in the figure indicate that each row of pixels performs the corresponding color operation at the same time, wherein the exposure time is much longer than the sum of all other operation times, which is beneficial to improving the resolution of the acquired frame image.

[0060] Since the multi-band PTT estimation method mainly uses light of different wavelengths to capture the time difference between the pulse wave reaching different skin depths to estimate blood pressure, and green light can penetrate the skin epidermis, and red light or near-infrared light can penetrate the subcutaneous tissue, this embodiment uses the pixel data of the two color channels in the frame image captured by the camera in the RGB color space as the processing object to realize the multi-band PTT estimation. It should be stated that the present invention does not limit the color space of the frame image captured by the camera, as long as the color space corresponding to the frame image includes at least two primary colors with different bands.

[0061] In order to more accurately extract the pixel data of the color channels corresponding to the light of different bands in the frame image, the rolling exposure camera selected in this embodiment not only has a rolling exposure function, but also has an adjustable lens focal length, and the camera is adjusted to a defocused state to capture the frame image, because the pixel data of different color channels of the frame image in the defocused state is easier to separate.

[0062] Considering that in the actual image acquisition process, if the skin area captured by the camera is too large, such as capturing the entire face, the following situations may occur: 1) There are multiple PTT points from the skin at the chin to the skin at the forehead; 2) The skin at different positions will inevitably have color differences when the camera is imaging. In order to minimize the impact of these two situations on the accuracy of multi-band PTT estimation, Figure 3 As shown, in this embodiment, a small skin area is selected during measurement and the camera lens is adjusted so that the actual image distance di is less than the focal length f, that is, the camera is in a defocused state. Then, the light from different positions will mix with each other, causing the camera image to be blurred, thereby reducing the impact of color differences during imaging. As can be seen from the figure, the skin image captured when the camera is out of focus will be blurrier than the skin image presented on the focal plane. At the same time, since the object distance do is too large, the defocused image captured by the image sensor in the camera may contain pixels of non-skin areas (such as hair and clothing), thereby interfering with the PTT measurement. Therefore, it should be ensured that the object distance do is not too large and the actual skin area photographed is not too large. It is worth noting that the skin area to be collected can be flexibly selected according to the physiological index to be measured, and the physiological indexes that can be measured by the present invention include but are not limited to heart rate, respiration, and heart rate variability.

[0063] Then, the pixels of the target skin area are scanned line by line using a camera with adjusted exposure parameters and other parameters to obtain a video stream including several frame images in a defocused state. Subsequently, each frame image in the video stream is used as a processing object, and the same processing operation is performed on each frame image.

[0064] Step S200: extracting pixel data of two target color channels in all the frame images, and performing group sampling on the pixel data of each target color channel corresponding to all the frame images to construct a sampling point matrix of each target color channel.

[0065] Specifically, in order to enhance the time difference of light of different wavelengths penetrating the skin, as a preferred embodiment, the color channels corresponding to the two light with the largest wavelength difference in the color space are selected as the two target color channels, which specifically includes: obtaining the color space corresponding to the frame image; screening the color channels corresponding to the two color lights with the largest wavelength difference in the color space to obtain the two target color channels; extracting the pixel data of the two target color channels in all frame images. For example, the color space corresponding to the camera is RGB, which has a good response to green (Green, G), red (Red, R) or near infrared (NIR) light. Since the wavelengths of near infrared light, red light and green light decrease in turn, it is obvious that the wavelength difference between near infrared light and green light is the largest. Therefore, the G color channel and the NIR color channel are selected as the two target color channels, and the pixel data in the G color channel and the NIR color channel in all frame images are extracted as the basis for the subsequent extraction of rPPG signals.

[0066] like Figure 4 As shown, in a preferred embodiment, the pixel data of each target color channel corresponding to all the frame images are grouped and sampled to construct a sampling point matrix of each target color channel, including:

[0067] Step S210: Divide the pixel data of each target color channel in each frame image into several groups and calculate the average value of each group of pixel data to obtain the sampling points of each target color channel in each frame image.

[0068] Specifically, for each frame image, the pixel data of each target color channel is divided into several groups according to the pixel rows, and each group of pixel data is sampled according to the preset sampling rule to obtain the sampling points corresponding to each group of pixel data. The sampling rule can be set according to the application requirements such as frame rate and calculation amount. For example, if you want to convert a picture with a resolution of The pixel data in the frame image is divided into 9 groups according to the row, so every 200 rows are a group, and the pixels in each group are Then the average value of each group of pixel data is calculated to obtain the pixel average value, and the pixel average value is used as a sampling point, and finally 9 sampling points are collected from each target color channel in the frame image.

[0069] Step S220: Arrange the sampling points in all the frame images corresponding to each of the target color channels according to sampling positions and exposure times, and construct a sampling point matrix for each of the target color channels.

[0070] Specifically, according to step S220, by grouping and sampling each frame image according to pixel rows, the same number of sampling points in each target color channel can be obtained. Video stream of frame images, collected from each target color channel of each frame image by pixel row Sampling points are arranged in each frame image according to the sampling position and exposure time, and the sampling point matrix of the two target color channels can be obtained. and ,Right now:

[0071] ,

[0072] ,

[0073] in, and Respectively represent the sampling point matrices corresponding to the two target color channels and The sampling points in , . Sampling point matrix The first column element in represents the division of the pixel rows in a target color channel of a frame image in the video stream into groups, and extract a sampling point from each group, and get sampling points.

[0074] It can be seen that this embodiment groups and samples the pixels in each target color channel in the frame image, and constructs a sampling point matrix using the sampling points of the same target color channel in all frame images, so that the key information in the entire video stream is integrated together, which not only helps to reduce the amount of calculation, but also can retain the key image information of each frame image.

[0075] Step S300: using each of the sampling point matrices to obtain the rPPG signal corresponding to each of the target color channels.

[0076] Specifically, in order to obtain an rPPG signal with enhanced frame rate, this embodiment arranges all the sampling points in each sampling point matrix in series in time order to obtain a sampling point sequence corresponding to each target color channel; and performs pixel value contrast enhancement processing on each sampling point in the sampling point sequence corresponding to each target color channel, thereby obtaining an rPPG signal corresponding to each target color channel. Since the exposure action of each pixel point on the rolling exposure CMOS is executed very quickly and cannot be distinguished by the naked eye, the timestamps of each row and column element in the target color channel can be determined through the CMOS manual, thereby obtaining the time sequence of each sampling point in the sampling point matrix.

[0077] For example, the two target color channels are C channel and D channel respectively. The timestamp of each column element in the sampling point matrix is ​​determined by using the CMOS manual. , the time difference between adjacent column elements is The time difference between adjacent row elements is Therefore, the timestamp of each element is , so that all elements in the two sampling point matrices are arranged in series according to the time sequence of each sampling point in each sampling point matrix, and the sampling point matrices are obtained respectively. and The corresponding sampling point sequence and ,Right now:

[0078] ,

[0079] .

[0080] Then, the sampling point sequence and The corresponding signal is de-DC-componented, and post-processed by signal filtering, detrending, etc., to obtain the rPPG signals corresponding to the two target color channels with enhanced frame rates, namely: and ,like Figure 4 shown.

[0081] In this embodiment, by converting the sampling point matrix into a data sequence according to the sampling time sequence of the sampling points, it is helpful to make the fitted frame rate enhanced rPPG signal more consistent with the conduction time trajectory of the pulse wave, thereby improving the authenticity and accuracy of the rPPG signal.

[0082] In a preferred embodiment, after obtaining the rPPG signal corresponding to each of the target color channels, the method further includes:

[0083] Step S310: Calculate the mean of the values ​​of all sampling points in the sampling point sequence corresponding to each of the target color channels to obtain the sampling point mean of each of the target color channels.

[0084] Step S320: Calculate the deviation between the value of each sampling point of the target color channel and the mean of the sampling points, and update the rPPG signal corresponding to each target color channel based on the ratio of the deviation of each sampling point to the mean of the sampling points of the corresponding target color channel.

[0085] Specifically, in order to remove the sampling point sequence and The DC component in the corresponding signal, this embodiment calculates the mean of the values ​​of all sampling points in the sampling point sequence to obtain the sampling point mean of each target color channel, and for each sampling point sequence, respectively uses the ratio relationship shown in the following formula to eliminate the influence of the DC component caused by the mean, and the formula is specifically:

[0086] ,

[0087] in, rPPG signal with enhanced frame rate or , Represents a sequence of sampling points or For each element in (i.e. each sampling point), It means taking the mean of all elements in the sampling point sequence, that is, the sampling point mean.

[0088] It is worth stating that the present invention can be used to measure physiological indicators including but not limited to heart rate, respiration and heart rate variability. Similarly, the enhanced rPPG signal constructed using the method of the present invention can be used to measure physiological indicators including but not limited to heart rate, respiration and heart rate variability.

[0089] Step S400: Obtaining the pulse wave transmission time based on the time relationship between the peaks of the two rPPG signals in the same time window.

[0090] Specifically, the timestamps of all the peaks of the two rPPG signals in the same time window are calculated; based on the timestamps of all the peaks, the time delay between the nearest neighbor peaks of the rPPG signals of the two target color channels is calculated, that is, the two rPPG signals are aligned according to the time window, and the time when the peaks in the two rPPG signals appear is compared, and the peak with the smallest time difference is regarded as the nearest neighbor peak, such as Figure 4 As shown, and so on, the time delay between each group of nearest neighbor peaks is calculated, and the average of the time delays between all groups of nearest neighbor peaks is calculated, and the average is used as the pulse wave transmission time.

[0091] For example, the collected video stream is divided into multiple segments according to a sliding window of preset duration (such as 5 seconds, 10 seconds, 30 seconds, etc.). For the segments in the same time window, all the peaks of the two rPPG signals are detected, and all the peaks in the PPG segment of one rPPG signal are used as reference points to determine the nearest lagging peak in the PPG segment of the other rPPG signal, that is, the nearest neighbor peak. For example, if the target color channels corresponding to the two rPPG signals are NIR channel and G channel respectively, and all the peaks in the PPG segment of one rPPG signal are used as reference points, the nearest neighbor peak is determined. All the peaks of the fragment are taken as the reference. Find the distance between each The time delay between the two segment peaks is calculated and the average value is taken as the final pulse wave transmission time (PTT).

[0092] In this embodiment, by collecting all the peaks of the two rPPG signals in the same time window, the time delay between each group of nearest neighbor peaks is calculated, and the average value of the time delay between the nearest neighbor peaks is used as the final pulse wave conduction time. This method can reduce the influence of signal noise caused by movement of the subject, etc., and improve the accuracy of the time delay between the two rPPG signals, which is beneficial to improve the accuracy of the calculated pulse wave conduction time.

[0093] In a preferred embodiment, the pulse wave transmission time is obtained based on the average value of all the time delays, comprising:

[0094] The two rPPG signals are defined as a first rPPG signal and a second rPPG signal, and the wavelength of light in a target color channel corresponding to the first rPPG signal is greater than the wavelength of light in a target color channel corresponding to the second rPPG signal; the average time intervals between the peaks of the first rPPG signal and the second rPPG signal are calculated to obtain a first average time interval and a second average time interval; if the pulse wave transmission time is greater than zero, the pulse wave transmission time is equal to the average time delay of all nearest neighboring peaks between the second rPPG signal and the first rPPG signal, and the pulse wave transmission time is less than the first average time interval; if the pulse wave transmission time is less than zero, the pulse wave transmission time is equal to the average time delay of all nearest neighboring peaks between the first rPPG signal and the second rPPG signal, and the absolute value of the pulse wave transmission time is less than the second average time interval.

[0095] For example, in actual measurement, the two-channel PPG signal waveform may cause noise due to the subject's movement, so the subject should try to stay still during the measurement. At the same time, there are multiple groups of corresponding peak feature points in the two-channel PPG signal segments in the same time window, and the timestamps of all peak feature points are represented as , , where the first rPPG signal The wavelength of light in the corresponding target color channel is longer, and the second rPPG signal The wavelength of the light in the corresponding target color channel is shorter. Since some peaks may deviate from the actual pulse due to noise, the PTT corresponding to the time deviation between each group of nearest neighbor peaks used to calculate the pulse wave transmission time needs to meet the following constraints, namely:

[0096] when When:

[0097] ,

[0098] in, is an adjustable parameter, ; represents the average time interval between the PPG signal peaks corresponding to the target color channel corresponding to all the light with longer wavelengths. At this time, if the light in the target color channel corresponding to the first rPPG signal and the second rPPG signal is near infrared light NIR and green light G respectively, then , represents all near-infrared channel PPG signals ( ) is the average interval between wave peaks.

[0099] when When:

[0100] ,

[0101] in, is an adjustable parameter, ; represents the average time interval between the PPG signal peaks corresponding to the target color channel corresponding to all the light with shorter wavelengths. At this time, if the light in the target color channel corresponding to the first rPPG signal and the second rPPG signal is near infrared light NIR and green light G respectively, then , indicating all green channel PPG signals ( ) is the average interval between wave peaks.

[0102] Therefore, the PTT corresponding to the time deviation between each group of nearest neighbor peaks in the two PPG signal waveforms within the same time window that meets the above constraints is selected, and the average value of all these PTTs is calculated as the final PTT.

[0103] In this embodiment, by setting constraints on the PTT corresponding to the time deviation between each group of nearest neighbor peaks in the two PPG signal waveforms within the same time window, it is equivalent to indirectly screening the valid peaks, which can effectively avoid the deviation caused by noise, and is conducive to further improving the accuracy and effectiveness of the pulse wave transmission time finally obtained.

[0104] Furthermore, based on the measured multi-band PTT, pulse wave velocity estimation, cuff-free blood pressure estimation, and assessment of physiological parameters such as arteriosclerosis can be performed.

[0105] like Figure 5As shown, corresponding to the above-mentioned multi-band pulse wave transmission time estimation method, an embodiment of the present invention further provides a multi-band pulse wave transmission time estimation system, and the above-mentioned multi-band pulse wave transmission time estimation system includes:

[0106] An image acquisition module 510 is used to scan pixels of a target skin area line by line using a rolling exposure camera to obtain a plurality of frames of images;

[0107] The sampling point matrix construction module 520 is used to extract the pixel data of the two target color channels in all the frame images, and group and sample the pixel data of each target color channel corresponding to all the frame images to construct a sampling point matrix of each target color channel;

[0108] An rPPG signal generating module 530 is used to obtain an rPPG signal corresponding to each of the target color channels using each of the sampling point matrices;

[0109] The pulse wave transit time solving module 540 is used to obtain the pulse wave transit time based on the time relationship between the peaks of the two rPPG signals in the same time window.

[0110] Specifically, in this embodiment, the specific functions of the multi-band pulse wave transmission time estimation system can also refer to the corresponding description in the multi-band pulse wave transmission time estimation method, which will not be repeated here.

[0111] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 6 As shown. The intelligent terminal can be used to execute the multi-band pulse wave transmission time estimation method provided in the above embodiment, which will not be described here for brevity. The intelligent terminal includes: a processor, the processor is coupled to a memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions stored in the memory, so that the method in the above method embodiment is executed.

[0112] The present invention also provides a computer-readable storage medium on which computer instructions for implementing the method in the above method embodiment are stored.

[0113] For example, when the computer program is executed by a computer, the computer can implement the method in the above method embodiment.

[0114] An embodiment of the present invention further provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to implement the method in the above method embodiment.

[0115] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0117] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0120] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A multi-band pulse wave transmission time estimation method, characterized in that: The following steps are involved: The pixels of the target skin area are scanned line by line using a rolling exposure camera to obtain several frames of images; Extracting pixel data of two target color channels from all the frame images, and grouping and sampling the pixel data of each target color channel corresponding to all the frame images to construct a sampling point matrix of each target color channel; Using each of the sampling point matrices, obtaining an rPPG signal corresponding to each of the target color channels; Obtaining the pulse wave transmission time based on the time relationship between the peaks of the two rPPG signals in the same time window; The method of obtaining the rPPG signal corresponding to each target color channel by using each sampling point matrix includes: Arrange all the sampling points in each of the sampling point matrices in series in time order to obtain a sampling point sequence corresponding to each of the target color channels; obtain an rPPG signal corresponding to each of the target color channels based on the sampling point sequence corresponding to each of the target color channels; After obtaining the rPPG signal corresponding to each of the target color channels, the method further includes: Calculate the mean of the values ​​of all sampling points in the sampling point sequence corresponding to each of the target color channels to obtain the sampling point mean of each of the target color channels; calculate the deviation between the value of the sampling point of each of the target color channels and the sampling point mean, and update the rPPG signal corresponding to each target color channel based on the ratio of the deviation of each of the sampling points to the sampling point mean of the corresponding target color channel.

2. The multi-band pulse wave transmission time estimation method according to claim 1, characterized in that: The step of extracting pixel data of two target color channels in all the frame images includes: Obtaining a color space corresponding to the frame image; Selecting color channels corresponding to two colors of light with the largest wavelength difference in the color space to obtain two target color channels; Pixel data of the two target color channels in all frame images are extracted.

3. The multi-band pulse wave transmission time estimation method according to claim 1, characterized in that: The pixel data of each target color channel corresponding to all the frame images are grouped and sampled to construct a sampling point matrix of each target color channel, including: The pixel data of each target color channel in each frame image are divided into several groups and the average value of each group of pixel data is calculated to obtain the sampling points of each target color channel in each frame image; The sampling points in all the frame images corresponding to each of the target color channels are arranged according to sampling positions and exposure times, so as to construct a sampling point matrix for each of the target color channels.

4. The multi-band pulse wave transmission time estimation method according to any one of claims 1 to 3, characterized in that: The step of obtaining the pulse wave transmission time based on the time relationship between the peaks of the two rPPG signals in the same time window comprises: Calculating timestamps of all said peaks; Based on the timestamps of all the peaks, calculating the time delay between the nearest neighbor peaks of the rPPG signals of the two target color channels; Based on the average of all said time delays, the pulse wave transit time is obtained.

5. The multi-band pulse wave transmission time estimation method according to claim 4, characterized in that: The pulse wave transmission time is obtained based on the average value of all the time delays, comprising: The two rPPG signals are defined as a first rPPG signal and a second rPPG signal, and the wavelength of light in a target color channel corresponding to the first rPPG signal is greater than the wavelength of light in a target color channel corresponding to the second rPPG signal; Respectively calculating average time intervals between respective peaks of the first rPPG signal and the second rPPG signal to obtain a first average time interval and a second average time interval; If the pulse wave transit time is greater than zero, the pulse wave transit time is equal to an average of time delays of all nearest neighbor peaks between the second rPPG signal and the first rPPG signal, and the pulse wave transit time is less than the first average time interval; If the pulse wave transit time is less than zero, the pulse wave transit time is equal to the average of the time delays of all nearest neighbor peaks between the first rPPG signal and the second rPPG signal, and the absolute value of the pulse wave transit time is less than the second average time interval.

6. A multi-band pulse wave transmission time estimation system, characterized in that: The system is used to implement the steps of the multi-band pulse wave transmission time estimation method according to any one of claims 1 to 5, comprising: An image acquisition module, used to scan the pixels of the target skin area line by line using a rolling exposure camera to obtain a plurality of frames of images; A sampling point matrix construction module is used to extract pixel data of two target color channels in all the frame images, and group and sample the pixel data of each target color channel corresponding to all the frame images to construct a sampling point matrix for each target color channel; An rPPG signal generating module, used to obtain the rPPG signal corresponding to each of the target color channels using each of the sampling point matrices; The pulse wave transmission time solving module is used to obtain the pulse wave transmission time based on the time relationship between the peaks of the two rPPG signals in the same time window.

7. An intelligent terminal, characterized in that: include: A memory for storing executable instructions; A processor is used to call and run the executable instructions in the memory to execute the steps of the multi-band pulse wave transmission time estimation method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed by a processor, the multi-band pulse wave transmission time estimation method as described in any one of claims 1-5 is implemented.

Images