A method and device for extracting pulse waves based on video images

Abstract

The present invention discloses a method and device for extracting pulse waves based on video images. The method includes: acquiring video data of a preset body part of a person to be detected for a preset duration in a static state; processing the video data to obtain all frame images in the video data, converting each obtained frame image into a grayscale format respectively, then segmenting each frame image to extract a target region in each frame image; calculating the grayscale centroid coordinates of the target region in each frame image using the grayscale centroid method; obtaining an original pulse wave signal based on the grayscale centroid coordinates of the target region in each frame image; performing empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal. By adopting the technical solution of the present invention, there is no need for a complex detection device, and the double-wave signal component of the pulse wave can be extracted more accurately, reducing the workload and harm to the human body.

Description

Technical Field

[0001] The present invention relates to the technical field of non-contact signal extraction and detection, and particularly relates to a method and device for extracting pulse waves based on video images. Background Art

[0002] Vascular aging is the pathophysiological basis for the occurrence or development of cardiovascular and cerebrovascular diseases, and the risk of cardiovascular diseases in the early stage of vascular aging population is significantly increased. The pulse wave refers to that during the beating of the heart, blood is pulsatively ejected into the aorta, and a pulse pressure wave will be generated on the aortic wall, simply referred to as the pulse wave (PPG). The pulse wave signal contains rich human physiological information. By analyzing the pulse wave, important physiological indicators such as the heart rate, respiratory rate, and blood oxygen of the measured person's cardiovascular system can be easily estimated, which has important reference value for human health assessment and the prevention and monitoring of cardiovascular diseases, and also provides a suitable and effective technical means for the pre-disease examination of clinical patients or the scientific guidance during the treatment process after the disease occurs. The dicrotic wave is one of the important components of the pulse wave. However, there are differences in the dicrotic waves of people of different age groups. Among normal people, the pulse waves of people before the age of 40 are mostly obvious or present, while the pulse waves of people after the age of 40 are mostly faint or disappear. The shape and its changes of the dicrotic wave are also important bases for clinically judging problems such as vascular sclerosis. Generally, it is considered that large peripheral resistance and poor vascular wall elasticity are the main reasons for the small and unclear amplitude of the dicrotic wave.

[0003] The principle of pulse wave signal measurement is: The light transmitted through biological tissues can be absorbed by different substances, including pigments in the skin, bones, arteries, and venous blood. Traditional methods for extracting pulse wave signals are divided into two major categories: contact type and non-contact type. Contact measurement generally uses an optical sensor to obtain pulse wave signals from positions such as fingers, wrists, and necks. Some scholars have also proposed a non-cuff continuous blood pressure measurement method based on pulse wave signals. Blood flow changes mainly occur in arteries and arterioles (but not in veins). For example, arteries contain more blood volume during the systolic phase of the cardiac cycle than during the diastolic phase. The PPG sensor detects changes in the blood volume in the tissue microvascular bed (i.e., changes in the detected light intensity) through reflection from the tissue or transmission through the tissue. The DC component of the pulse wave waveform corresponds to the transmitted or reflected light signal detected from the tissue and depends on the structure of the tissue and the average blood volume of arterial and venous blood. It should be noted that the DC component changes slowly with respiration. The AC component shows the blood volume changes between the systolic and diastolic phases.

[0004] The non-contact measurement of pulse wave signals is mainly obtained by processing the video information collected by a camera. Currently, the pulse wave signals are mainly processed from videos of the face, fingers, ears, wrists, etc. The main reason for choosing these parts as the acquisition sites is that the cortex of these parts is relatively thin and there are many blood vessels under the skin, which can provide rich physiological information. The method of obtaining the pulse wave signal is generally to use the method of calculating the average pixel value of the region of interest to obtain the blood perfusion information, and then use relevant algorithms (mainly filtering) to obtain the pulse wave signal. Generally speaking, due to the small optical interference of the signals in the videos of fingers and ears, the pulse wave signals obtained from these videos are better than those obtained from other parts. One method of extracting the video pulse wave is the gray weight method, which calculates the third quartile of all pixel points in the corresponding target region of each frame of the video, and obtains the average value of the third quartiles of the corresponding pixel points of all frames of the image; for each frame of the image, count the total number of pixel points with gray values greater than the average value in its corresponding target region, and use the counted total number of pixel points as the intensity of the pulse wave signal corresponding to the current frame of the image. Some scholars also use the gray center of gravity of the image to extract the pulse wave. The specific method is to calculate the gray center of gravity coordinates of the video image using the gray center of gravity method, and then calculate the square root of the sum of the squares of the horizontal and vertical coordinates as the pulse wave intensity, so as to extract the pulse wave signal. This method has two deficiencies. One is that the pulse wave is an embodiment of the change in blood volume in the arterial blood vessels. Simply calculating the square sum of the horizontal and vertical coordinates of the center of gravity to represent the pulse wave intensity cannot reflect the changing process. The other is that this method requires a relatively precise image detection instrument to obtain the video signal. The deficiencies of the above methods are that, on the one hand, they are easily affected by changes in light, which affects the quality of the pulse wave signal, and on the other hand, they cannot extract the dicrotic wave signal component in the pulse wave signal, resulting in the loss of some information. Summary of the Invention

[0005] The present invention provides a method and device for extracting pulse waves based on video images to solve the problems of complex detection devices, large workload, and loss of dicrotic waves in the extracted pulse waves existing in the prior art.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] On the one hand, the present invention provides a method for extracting pulse waves based on video images, including:

[0008] Obtaining video data of a preset body part of the person to be detected taken for a preset duration in a static state;

[0009] Process the video data to obtain all frame images in the video data, convert each obtained frame image into grayscale format, and then segment each frame image to extract the target region in each frame image; wherein, in each frame image, the position and size of the target region in the image are the same;

[0010] Use the gray centroid method to calculate the gray centroid coordinates of the target region in each frame image;

[0011] Based on the gray centroid coordinates of the target region in each frame image, obtain the original pulse wave signal;

[0012] Perform empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal.

[0013] Further, the formula for using the gray centroid method to calculate the gray centroid coordinates of the target region in each frame image is:

[0014]

[0015]

[0016] wherein, a0 and b0 respectively represent the horizontal and vertical coordinates of the gray centroid of the corresponding target region; Q represents the corresponding target region; W(x, y) represents the gray value of the point with coordinates (x, y) in the corresponding target region.

[0017] Further, the obtaining of the original pulse wave signal based on the gray centroid coordinates of the target region in each frame image includes:

[0018] Based on the gray centroid coordinates of the target region, draw the gray centroid trajectory curve, including: taking the horizontal coordinate of the gray centroid of the target region in each frame image as the vertical coordinate and the video frame number where the image frame is located as the horizontal coordinate, draw the gray centroid horizontal coordinate curve; taking the vertical coordinate of the gray centroid of the target region in each frame image as the vertical coordinate and the video frame number where the image frame is located as the horizontal coordinate, draw the gray centroid vertical coordinate curve;

[0019] According to the drawn gray centroid trajectory curve, judge whether the calculated and extracted gray centroid is accurate and appropriate;

[0020] Calculate the displacement differences of the horizontal and vertical coordinates of the gray centroid of the target region in two adjacent frame images respectively; and draw the displacement curve of the gray centroid horizontal coordinate and the displacement curve of the gray centroid vertical coordinate respectively;

[0021] Use the principal component analysis method to extract the pulse wave signal information components in the displacement curve of the gray centroid horizontal coordinate and the displacement curve of the gray centroid vertical coordinate to obtain the original pulse wave signal curve.

[0022] Further, obtaining the original pulse wave signal based on the gray center-of-gravity coordinates of the target region in each frame of image includes:

[0023] Calculating the distance difference between the gray centers of gravity of the target regions in two adjacent frames of images, using the calculated distance difference as an index for measuring the intensity of the pulse wave signal, plotting the pulse wave curve, and extracting the original pulse wave signal.

[0024] Further, performing empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal includes:

[0025] Performing empirical mode decomposition on the original pulse wave signal, decomposing the original pulse wave signal into signals of different frequency components, removing baseline drift and high-frequency noise, and obtaining a pure pulse wave signal.

[0026] On the other hand, the present invention also provides a pulse wave extraction device based on video images, including:

[0027] A video data acquisition module, configured to acquire video data of a preset duration of a preset body part of a person to be detected taken in a static state;

[0028] A data processing module, configured to perform the following steps:

[0029] Processing the video data, acquiring all frames of images in the video data, respectively converting the acquired frames of images into grayscale format, then segmenting each frame of image, and extracting the target region in each frame of image; wherein, in each frame of image, the position and size of the target region in the image are the same;

[0030] Calculating the gray center-of-gravity coordinates of the target region in each frame of image using the gray center-of-gravity method;

[0031] Obtaining the original pulse wave signal based on the gray center-of-gravity coordinates of the target region in each frame of image;

[0032] Performing empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal.

[0033] Further, the formula for calculating the gray center-of-gravity coordinates of the target region in each frame of image using the gray center-of-gravity method is:

[0034]

[0035]

[0036] wherein, a0 and b0 respectively represent the horizontal and vertical coordinates of the gray center of gravity of the corresponding target region; Q represents the corresponding target region; and W(x, y) represents the gray value of the point with coordinates (x, y) in the corresponding target region.

[0037] Further, obtaining the original pulse wave signal based on the gray center-of-gravity coordinates of the target region in each frame of image includes:

[0038] Based on the gray center-of-gravity coordinates of the target region, draw the gray center-of-gravity trajectory curve, including: taking the abscissa of the gray center-of-gravity of the target region in each frame of image as the ordinate and the video frame number where the image frame is located as the abscissa to draw the gray center-of-gravity abscissa curve graph; taking the ordinate of the gray center-of-gravity of the target region in each frame of image as the ordinate and the video frame number where the image frame is located as the abscissa to draw the gray center-of-gravity ordinate curve graph.

[0039] According to the drawn gray center-of-gravity trajectory curve, judge whether the calculated and extracted gray center-of-gravity is accurate and appropriate.

[0040] Calculate the displacement differences of the abscissa and ordinate of the gray center-of-gravity of the target region in two adjacent frames of images respectively; and draw the displacement curve graph of the gray center-of-gravity abscissa and the displacement curve graph of the gray center-of-gravity ordinate respectively.

[0041] Use the principal component analysis method to extract the pulse wave signal information components in the displacement curve graph of the gray center-of-gravity abscissa and the displacement curve graph of the gray center-of-gravity ordinate to obtain the original pulse wave signal curve.

[0042] Further, obtaining the original pulse wave signal based on the gray center-of-gravity coordinates of the target region in each frame of image includes:

[0043] Calculate the distance difference between the gray centers-of-gravity of the target region in two adjacent frames of images, use the calculated distance difference as an index to measure the pulse wave signal intensity, draw the pulse wave curve, and extract the original pulse wave signal.

[0044] Further, performing empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal includes:

[0045] Perform empirical mode decomposition on the original pulse wave signal, decompose the original pulse wave signal into signals with different frequency components, remove the baseline shift and high-frequency noise, and obtain a pure pulse wave signal.

[0046] On the other hand, the present invention also provides an electronic device, which includes a processor and a memory; wherein, at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the above method.

[0047] On another hand, the present invention also provides a computer-readable storage medium, and at least one instruction is stored in the storage medium, and the instruction is loaded and executed by the processor to implement the above method.

[0048] The beneficial effects brought by the technical solution provided by the present invention at least include:

[0049] The method for extracting pulse wave based on video image provided by the present invention uses a smart phone or other video recording devices to capture video image data of the finger part of the detector for 10s to 60s in the normal video recording state, and then uses a computer and corresponding software to process the collected data to obtain the video image frame data of each frame of the video, segment the image frame to obtain the target area, and perform empirical mode decomposition on the signal to filter out the baseline shift and high-frequency noise to obtain a relatively pure pulse wave signal. The present invention does not require a complex detection device, reduces the workload and harm to the human body; it can extract the dicrotic wave signal component in the pulse wave signal, which helps to judge the disease diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0051] Figure 1 is a schematic flowchart of the method for extracting pulse wave based on video image provided by the embodiment of the present invention;

[0052] Figure 2a is one frame of the 10s finger video taken by the mobile phone;

[0053] Figure 2b is for Figure 2a the image obtained by segmenting the shown image to obtain the target area;

[0054] Figure 3 is the pulse wave signal curve graph extracted by using the gray weight method;

[0055] Figure 4a is the gray center of gravity abscissa curve graph;

[0056] Figure 4b is the gray center of gravity ordinate curve graph;

[0057] Figure 5a is the displacement difference curve graph of the gray center of gravity abscissa;

[0058] Figure 5b is the displacement difference curve graph of the gray center of gravity ordinate;

[0059] Figure 5c is the square root curve graph of the sum of squares of the gray center of gravity abscissa and ordinate;

[0060] Figure 6 is for Figure 5a and Figure 5bThe pulse wave curve diagram obtained by performing principal component analysis on the shown curve;

[0061] Figure 7 It is the curve diagram of the distance difference of the gray - scale centroid;

[0062] Figure 8 It is the method flow chart for performing empirical mode decomposition on the original pulse wave signal;

[0063] Figure 9 It is for Figure 7 The schematic diagram of different - frequency signals obtained by performing empirical mode decomposition on the shown curve;

[0064] Figure 10a It is for Figure 7 The pulse wave curve diagram obtained by performing empirical mode decomposition on the shown curve;

[0065] Figure 10b It is for Figure 6 The pulse wave curve diagram obtained by performing empirical mode decomposition on the shown curve;

[0066] Figure 10c It is for Figure 5c The pulse wave curve diagram obtained by performing empirical mode decomposition on the shown curve. Specific implementation manner

[0067] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the accompanying drawings.

[0068] The first embodiment

[0069] This embodiment provides a method for extracting pulse waves based on video images. This method can be implemented by an electronic device, and the execution process of this method is as Figure 1 shown, including the following steps:

[0070] S1, Obtain video data of a preset duration of a preset body part of the person to be detected taken in a stationary state;

[0071] Among them, the video data is obtained by using a smartphone, a camera, or other video - recording devices; the video - recording device should have a relatively high image resolution (not less than 1920*1080) to ensure obtaining sufficient video image data of pulse - wave characteristic information. Specifically, in this embodiment, a smartphone is used to record a video of a finger being stationary for 10 s to 60 s in the normal video - recording state, and the video data is transmitted to a computer through a connecting device for storage.

[0072] Among them, it should be noted that the device for shooting videos is not limited to smartphones. Other devices with shooting functions such as cameras and monitors can also be used as shooting devices to obtain the original videos. In order to ensure sufficient pulse wave information is obtained from the videos, the resolution of the video images should not be lower than 1920*1080. Moreover, the shooting part is not limited to the finger. Even the limbs and face of the body can also be used as the video source for obtaining pulse wave signals. Compared with other parts of the body, the finger has the advantages of rich blood vessels, strong pulse wave signals, and less optical interference in finger videos. In addition, when shooting the video, try to keep the shooting part stationary and reduce jitter. The length of the shooting video is not limited to 10s to 60s. 10s to 60s is just a more appropriate time interval.

[0073] S2. Process the video data to obtain all the frame images in the video data, convert each of the obtained frame images into grayscale format respectively, and then segment each frame image to extract the target area in each frame image; among them, in each frame image, the position and size of the target area in the image are the same.

[0074] Specifically, in this embodiment, the video duration is 10.22s, the video frame rate is 30.0583, the total number of video frames is 307, the video resolution is 1920*1080, the video format is RGB24, and one of the frame images of the video is as Figure 2a shown. Extract all the frame images of the video, segment the image frames to obtain the target area, and for the Figure 2a image frame shown, the segmentation result is as Figure 2b shown, where the area circled by the white square is the target area. In this embodiment, the size of the target area is 301*301 and it is located at the center position of the image frame. Among them, it should be noted that the selection of the target area is not fixed and is not limited to 301*301. The principle of selection is to minimize the influence of ambient light at the edge, so it should not be too large.

[0075] S3. Use the gray centroid method to calculate the gray centroid coordinates of the target area in each frame image.

[0076] The formula for calculating the gray centroid coordinates of the target area in each frame image using the gray centroid method is:

[0077]

[0078]

[0079] Among them, a0 and b0 respectively represent the horizontal and vertical coordinates of the gray centroid of the corresponding target area; Q represents the corresponding target area; W(x, y) represents the gray value of the point with coordinates (x, y) in the corresponding target area.

[0080] S4. Based on the gray center-of-gravity coordinates of the target region in each frame of the image, obtain the original pulse wave signal;

[0081] It should be noted that the traditional method is to use the gray weight method to extract the pulse wave signal. The pulse wave signal curve obtained by using such a method is as Figure 3 shown. In this embodiment, the following method is used to obtain it.

[0082] Method 1:

[0083] Step 1. Based on the gray center-of-gravity coordinates of the target region, draw the gray center-of-gravity trajectory curve;

[0084] Specifically, drawing the gray center-of-gravity trajectory curve includes: taking the abscissa of the gray center of gravity of the target region in each frame of the image as the ordinate and the video frame number where the image frame is located as the abscissa, and drawing the gray center-of-gravity abscissa curve graph as Figure 4a shown; taking the ordinate of the gray center of gravity of the target region in each frame of the image as the ordinate and the video frame number where the image frame is located as the abscissa, and drawing the gray center-of-gravity ordinate curve graph as Figure 4b shown;

[0085] Step 2. According to the gray center-of-gravity trajectory curve, judge whether the calculated gray center of gravity is accurate and appropriate;

[0086] Step 3. Calculate the displacement differences of the abscissa and ordinate of the gray center of gravity of the target region in two adjacent frames of the image respectively; and draw the displacement curve graph of the gray center-of-gravity abscissa as Figure 5a shown and the displacement curve graph of the gray center-of-gravity ordinate as Figure 5b shown; these two curves jointly contain the information of the pulse wave. The principal component analysis method can be used to extract the common pulse wave signal therein, so as to obtain the corresponding pulse wave signal;

[0087] Step 4. Use the principal component analysis method to extract the pulse wave signal from the displacement curve graph of the gray center-of-gravity abscissa and the displacement curve graph of the gray center-of-gravity ordinate, and obtain the original pulse wave signal curve as Figure 6 shown.

[0088] Specifically, in this embodiment, the above Step 3 is specifically: recording the gray center-of-gravity coordinates of each frame of the image calculated above as (x1, y1), (x2, y2),..., (x n , y n ), where n is the total number of frames of the video. In the video of this embodiment, this value is 307. Then subtract the abscissa and ordinate of the current frame from the abscissa and ordinate of the next frame respectively to obtain the displacement of the abscissa and ordinate of the gray center of gravity of the target region in these two frames. Specifically, the displacement calculation formula of the abscissa of the gray center of gravity of the target region in the m-th frame of the image is Δx m = xm+i -x m ; Similarly, the displacement calculation formula for the vertical coordinate of the gray center of gravity of the target area in the m-th frame image can be obtained as Δy m = y m+i - y m . Thus, the displacement curve graph of the horizontal coordinate of the gray center of gravity as shown in Figure 5a and the displacement curve graph of the vertical coordinate of the gray center of gravity as shown in Figure 5b are obtained. Further, the above step 4 is specifically: using the principal component analysis method to extract the principal components of the pulse wave signals contained in the two curves. Specifically, first, decentralize the displacement data of the two horizontal and vertical coordinates, that is, subtract the respective average values from each feature, then calculate the covariance matrix, and use the method of eigenvalue decomposition to obtain the eigenvalues and eigenvectors of the covariance matrix. Further, sort the eigenvalues from large to small and select the largest eigenvalue and its corresponding eigenvector. Finally, transform the data into the space corresponding to the eigenvector to obtain the extracted pulse wave signal, and thus obtain the pulse wave curve graph as shown in Figure 6 .

[0089] In addition, it should also be noted that for the convenience of subsequent comparison between the pulse wave signal curve obtained by the method of this embodiment and the pulse wave signal curve obtained by the traditional method, this embodiment also draws the square root curve graph of the sum of the squares of the horizontal and vertical coordinates of the gray center of gravity as shown in Figure 5c .

[0090] Method 2

[0091] Calculate the distance difference between the gray centers of gravity of the target areas in two adjacent frames of images, use the calculated distance difference as an index to measure the intensity of the pulse wave signal, draw the pulse wave curve, and extract the original pulse wave signal.

[0092] Specifically, in this embodiment, the above steps are: record the gray center of gravity coordinates of each frame of image calculated above as (x1, y1), (x2, y2),..., (x n , y n ), where n is the total number of frames of the video, and in the video of this embodiment, this value is 307. Then, find the Euclidean distance between the gray center of gravity coordinates of the next frame and the current frame to obtain the displacement of the gray center of gravity of the target area in these two frames of images. Specifically, the displacement calculation formula for the gray center of gravity of the target area in the k-th frame image is Based on this, use the calculated distance difference as an index to measure the intensity of the pulse wave signal, and draw the pulse wave curve as shown in Figure 7 .

[0093] This method takes into account the changes in both the horizontal and vertical coordinates of the gray center of gravity at the same time, and the calculation process is relatively simple. It can also be seen from the graph that the pulse wave signal extracted by this method retains the signal component of the dicrotic wave to a certain extent.

[0094] S5. Perform empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal.

[0095] It should be noted that performing empirical mode decomposition on the pulse wave signal is to obtain the intrinsic mode functions. According to the concept of intrinsic mode functions, any signal is composed of several intrinsic mode functions, and the intrinsic mode functions overlap and affect each other to form a composite function. Therefore, performing empirical mode decomposition on the pulse wave signal is to obtain the intrinsic mode functions of each order of the pulse wave signal.

[0096] The specific steps for performing empirical mode decomposition on the pulse wave signal are as follows: Obtain the maximum envelope line and the minimum envelope line of the original pulse wave signal to obtain the average envelope line; Subtract the average envelope line from the original pulse wave signal to obtain a new signal; Take the first-order or higher-order differential of the original pulse signal and repeat the above steps to obtain the intrinsic mode functions of each order of the original pulse wave; Filter out the baseline shift and high-frequency noise according to the frequency characteristics of the pulse wave signal and retain the useful signal components to obtain a pure pulse wave signal.

[0097] Based on the above, as Figure 8 shown, the above S5 specifically includes:

[0098] S51. Obtain the maximum envelope line and the minimum envelope line of the original pulse wave signal to obtain the average envelope line.

[0099] Specifically, the above S51 includes: First step, obtain all the maximum value points of the original pulse wave signal and fit them with a cubic spline function to obtain the upper envelope line; Second step, obtain all the minimum value points of the original pulse wave signal and fit them with a cubic spline function to obtain the lower envelope line; Third step, the average of the upper envelope line and the lower envelope line is recorded as the average envelope line, and thus the average envelope line is obtained.

[0100] S52. Subtract the average envelope line from the original pulse wave signal to obtain a new signal; Take the first-order or higher-order differential of the original pulse signal and repeat the above steps to obtain the intrinsic mode functions of each order of the original pulse wave.

[0101] Specifically, the above S52 is: Subtract the average envelope line from the original pulse wave signal to obtain a new signal, and this signal is an intrinsic mode function of the original pulse wave; Take the first-order or higher-order differential of the original pulse wave, and for the obtained signal, repeat step S51 and the step of subtracting the average envelope line from the above signal, and thus obtain the intrinsic mode functions of each order of the original pulse wave.

[0102] S53. Filter out the baseline shift and high-frequency noise according to the frequency characteristics of the pulse wave signal and retain the useful signal components to obtain a pure pulse wave signal.

[0103] Specifically, in this embodiment, forFigure 7 The schematic diagram of the different frequency signals obtained by performing empirical mode decomposition on the signal shown is as follows Figure 9 shown. According to the frequency characteristics of the pulse wave signal, the signal components of imf1 and imf2 belong to low-frequency noise interference, the signal components of imf5 and imf6 belong to high-frequency noise, and res is the residual interference caused by baseline shift. Therefore, the sum of the signal components of imf3 and imf4 is the extracted pulse wave signal component.

[0104] Applying this method further, for Figure 7 the pulse wave curve obtained by performing empirical mode decomposition on the curve shown is as follows Figure 10a shown; for Figure 6 the pulse wave curve obtained by performing empirical mode decomposition on the curve shown is as follows Figure 10b shown; for Figure 5c the pulse wave curve obtained by performing empirical mode decomposition on the curve shown is as follows Figure 10c shown.

[0105] In summary, the method for extracting the pulse wave signal by calculating the distance of the gray center-of-gravity coordinates proposed in this embodiment has the main advantage that it simultaneously considers the changes in the horizontal and vertical coordinates of the gray center of gravity, and the calculation process is relatively simple. More importantly, it can also be seen from the figure that the pulse wave signal extracted by the method of this embodiment retains the signal components of the dicrotic wave to a certain extent. The method for extracting the pulse wave signal in the horizontal and vertical displacements of the gray center of gravity by using the principal component analysis method proposed in this embodiment is very similar to the pulse wave signal result extracted by the method of calculating the square root of the sum of the squares of the gray center-of-gravity coordinates proposed by previous scholars, and both are more stable than the pulse wave signal extracted by the gray weight method. However, neither of them can extract the signal components of the dicrotic wave contained in the pulse wave signal.

[0106] Second Embodiment

[0107] This embodiment provides a pulse wave extraction device based on video images, including the following modules:

[0108] A video data acquisition module, configured to acquire video data of a preset duration of a preset body part of a person to be detected taken in a stationary state;

[0109] A data processing module, configured to perform the following steps:

[0110] Process the video data, obtain all frame images in the video data, convert each obtained frame image into a gray scale format, and then segment each frame image to extract the target region in each frame image; wherein, in each frame image, the position and size of the target region in the image are the same;

[0111] Use the gray center-of-gravity method to calculate the gray center-of-gravity coordinates of the target region in each frame image;

[0112] Based on the gray center-of-gravity coordinates of the target region in each frame of the image, an original pulse wave signal is obtained;

[0113] The original pulse wave signal is subjected to empirical mode decomposition to obtain a pure pulse wave signal.

[0114] The pulse wave extraction device based on video images in this embodiment corresponds to the pulse wave extraction method based on video images in the above first embodiment; among them, the functions implemented by each functional module in the pulse wave extraction device based on video images in this embodiment correspond one by one to the respective process steps in the pulse wave extraction method based on video images in the above first embodiment; therefore, it will not be elaborated here.

[0115] Third Embodiment

[0116] This embodiment provides an electronic device, which includes a processor and a memory; among them, at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the method of the first embodiment.

[0117] This electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPUs) and one or more memories. Among them, at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to perform the above method.

[0118] Fourth Embodiment

[0119] This embodiment provides a computer-readable storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by the processor to implement the method of the above first embodiment. Among them, the computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein can be loaded and executed by the processor in the terminal to perform the above method.

[0120] In addition, it should be noted that the present invention can be provided as a method, a device or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes.

[0121] Embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate a device for implementing the functions specified in one process Figure One one process or multiple processes and / or blocks Figure One or a device for implementing the functions specified in multiple blocks.

[0122] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in one process Figure One one process or multiple processes and / or blocks Figure One or a device for implementing the functions specified in multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one process Figure One one process or multiple processes and / or blocks Figure One or a device for implementing the functions specified in multiple blocks.

[0123] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or terminal device including the said element.

[0124] Finally, it should be noted that the above description is the preferred embodiment of the present invention. It should be pointed out that although the preferred embodiments of the present invention have been described, for those skilled in the art of this technology, once they know the basic creative concept of the present invention, without departing from the principle described in the present invention, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A method for extracting pulse waves based on video images, characterized in that Including: Obtaining video data of a preset duration captured for a preset body part of a person to be detected in a static state; Processing the video data to obtain all frame images in the video data, converting each obtained frame image into a grayscale format, and then segmenting each frame image to extract the target region in each frame image; wherein, in each frame image, the position and size of the target region in the image are the same; Calculating the grayscale centroid coordinates of the target region in each frame image using the grayscale centroid method; Obtaining an original pulse wave signal based on the grayscale centroid coordinates of the target region in each frame image; Performing empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal; Obtaining an original pulse wave signal based on the grayscale centroid coordinates of the target region in each frame image, including: Drawing a grayscale centroid trajectory curve based on the grayscale centroid coordinates of the target region, including: taking the abscissa of the grayscale centroid of the target region in each frame image as the ordinate and the video frame number where the image frame is located as the abscissa to draw a grayscale centroid abscissa curve; taking the ordinate of the grayscale centroid of the target region in each frame image as the ordinate and the video frame number where the image frame is located as the abscissa to draw a grayscale centroid ordinate curve; Judging whether the calculated and extracted grayscale centroid is accurate and appropriate according to the drawn grayscale centroid trajectory curve; Calculating the displacement differences of the abscissa and ordinate of the grayscale centroid of the target region in two adjacent frame images respectively; and drawing a displacement curve of the grayscale centroid abscissa and a displacement curve of the grayscale centroid ordinate respectively; Extracting the common pulse wave signal information components in the displacement curve of the grayscale centroid abscissa and the displacement curve of the grayscale centroid ordinate using the principal component analysis method to obtain an original pulse wave signal curve.

2. The method for extracting pulse wave based on video image according to claim 1, characterized in that, The formula for calculating the grayscale centroid coordinates of the target region in each frame image using the grayscale centroid method is: wherein, a0 and b0 respectively represent the abscissa and ordinate of the grayscale centroid of the corresponding target region; Q represents the corresponding target region; W(x, y) represents the grayscale value of the point with coordinates (x, y) in the corresponding target region.

3. The method for extracting a pulse wave based on video images according to claim 1, characterized in that, Performing empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal, including: Performing empirical mode decomposition on the original pulse wave signal, decomposing the original pulse wave signal into signals of different frequency components, removing the baseline shift and high-frequency noise, and obtaining a pure pulse wave signal.

4. A pulse wave extraction device based on video images, characterized in that, Including: A video data acquisition module for obtaining video data of a preset duration captured for a preset body part of a person to be detected in a static state; A data processing module for performing the following steps: Processing the video data to obtain all frame images in the video data, converting each obtained frame image into a grayscale format, and then segmenting each frame image to extract the target region in each frame image; wherein, in each frame image, the position and size of the target region in the image are the same; Calculating the grayscale centroid coordinates of the target region in each frame image using the grayscale centroid method; Obtaining an original pulse wave signal based on the grayscale centroid coordinates of the target region in each frame image; Perform empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal; Based on the gray center-of-gravity coordinates of the target region in each frame of the image, obtain the original pulse wave signal, including: Based on the gray center-of-gravity coordinates of the target region, draw the gray center-of-gravity trajectory curve, including: taking the abscissa of the gray center of gravity of the target region in each frame of the image as the ordinate and the video frame number where the image frame is located as the abscissa, draw the gray center-of-gravity abscissa curve; taking the ordinate of the gray center of gravity of the target region in each frame of the image as the ordinate and the video frame number where the image frame is located as the abscissa, draw the gray center-of-gravity ordinate curve; According to the drawn gray center-of-gravity trajectory curve, judge whether the calculated and extracted gray center of gravity is accurate and appropriate; Calculate the displacement differences of the abscissa and ordinate of the gray center of gravity of the target region in two adjacent frames of the image respectively; and draw the displacement curve of the gray center-of-gravity abscissa and the displacement curve of the gray center-of-gravity ordinate respectively; Use the principal component analysis method to extract the common pulse wave signal information components in the displacement curve of the gray center-of-gravity abscissa and the displacement curve of the gray center-of-gravity ordinate to obtain the original pulse wave signal curve.

5. The pulse wave extraction device based on video images according to claim 4, characterized in that, The formula for calculating the gray center-of-gravity coordinates of the target region in each frame of the image using the gray center-of-gravity method is: where a0 and b0 represent the abscissa and ordinate of the gray center of gravity of the corresponding target region respectively; Q represents the corresponding target region; W(x, y) represents the gray value of the point with coordinates (x, y) in the corresponding target region.

6. The pulse wave extraction device based on video images according to claim 4, wherein The performing empirical mode decomposition on the original pulse wave signal to obtain a pure pulse wave signal includes: Perform empirical mode decomposition on the original pulse wave signal, decompose the original pulse wave signal into signals with different frequency components, remove the baseline shift and high-frequency noise, and obtain a pure pulse wave signal.

Images