Pulse wave acquisition method, device, non-volatile storage medium and electronic device

Abstract

The present application discloses a pulse wave acquisition method, device, non-volatile storage medium, and electronic device. The method comprises: acquiring an image of a target skin area and acquiring a first target signal from the image, wherein the first target signal is a time series signal of pixels in the target skin area, and the time series signal includes a pulse wave signal; removing interference signals from the first target signal, and performing binary wavelet decomposition on the first target signal after removing the interference signals based on a target wavelet to determine a second target signal, wherein the waveform of the target wavelet corresponds to the waveform of the pulse wave signal; and determining the pulse wave signal based on the second target signal. The present application solves the technical problem of being unable to conveniently determine the health status of a target object due to the inability to extract a pulse wave signal containing detailed information in related technologies.

Description

Technical Field

[0001] The present application relates to the field of signal processing, and in particular to a pulse wave acquisition method, device, non-volatile storage medium and electronic device. Background Art

[0002] The pulse wave measurement accuracy of the remote pulse wave measurement instruments involved in the current related technologies is limited. Although they can accurately estimate the heart rate through the pulse, they cannot extract the pulse wave signal containing detailed information, and have high requirements for the measurement environment, making it difficult to apply in actual monitoring scenarios.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of the present application provide a pulse wave acquisition method, device, non-volatile storage medium and electronic device to at least solve the technical problem of being unable to conveniently determine the health status of a target object due to the inability to extract a pulse wave signal containing detailed information in related technologies.

[0005] According to one aspect of an embodiment of the present application, a pulse wave acquisition method is provided, comprising: acquiring an image of a target skin area and acquiring a first target signal from the image, wherein the first target signal is a timing signal of pixels in the target skin area, and the timing signal includes a pulse wave signal; removing interference signals from the first target signal, and performing binary wavelet decomposition on the first target signal after removing the interference signals based on a target wavelet to determine a second target signal, wherein the waveform of the target wavelet corresponds to the waveform of the pulse wave signal; and determining the pulse wave signal based on the second target signal.

[0006] Optionally, the step of obtaining an image of the target skin area includes: obtaining a target video, wherein the target video includes a target face image; identifying an eye image in the target face image, and locating a forehead area in the target face image based on the eye image; and determining a target skin area in the forehead area.

[0007] Optionally, the first target signal is a timing signal of pixels in the target skin area in a green channel.

[0008] Optionally, the interference signal in the first target signal is removed, and the first target signal after the interference signal is removed is subjected to binary wavelet decomposition according to the target wavelet to obtain the second target signal. The steps include: performing signal preprocessing on the first target signal to obtain a third target signal, wherein the processing method for performing signal preprocessing on the first target signal includes at least one of the following: removing the DC component in the first target signal and removing the singular point in the first target signal; filtering the third target signal by using a continuous wavelet transform to obtain a fourth target signal; and performing binary wavelet decomposition on the fourth target signal by using the target wavelet to obtain the second target signal.

[0009] Optionally, the third target signal is filtered using a continuous wavelet transform to obtain a fourth target signal, including: determining a pseudo frequency corresponding to the continuous wavelet transform; filtering the third target signal based on the pseudo frequency to filter out the first type of interference signal in the third target signal.

[0010] Optionally, the step of performing binary wavelet decomposition on the fourth target signal through the target wavelet to obtain the second target signal includes: determining a target scale parameter corresponding to the target wavelet, wherein the target scale parameter is used to reflect the amplification level of the target wavelet; and decomposing the fourth target signal by using an inverse binary wavelet transform according to the target scale parameter, filtering out the second type of interference signal in the fourth target signal, and obtaining the second target signal.

[0011] Optionally, the second target signal includes multiple signal components; the step of determining the pulse wave signal based on the second target signal includes: determining a component scale parameter of each signal component in the multiple signal components, and sorting the multiple signal components in order from small to large according to the component scale parameters to obtain a signal component sequence; determining a target number of target signal components based on the amplification level, wherein the target signal component is a signal component used to determine the pulse wave signal; selecting the first target number of signal components in the signal component sequence as the target signal components, and determining the pulse wave signal based on the target signal components.

[0012] According to another aspect of an embodiment of the present application, a pulse wave acquisition device is also provided, including: an acquisition module, used to acquire an image of a target skin area and acquire a first target signal from the image, wherein the first target signal is a timing signal of pixels in the target skin area, and the timing signal includes a pulse wave signal; a first processing module, used to remove interference signals from the first target signal, and perform binary wavelet decomposition on the first target signal after removing the interference signals based on a target wavelet to determine a second target signal, wherein the waveform of the target wavelet corresponds to the waveform of the pulse wave signal; and a second processing module, used to determine the pulse wave signal based on the second target signal.

[0013] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored. When the program is executed, the device where the non-volatile storage medium is located is controlled to execute the pulse wave acquisition method.

[0014] According to another aspect of an embodiment of the present application, an electronic device is further provided. The electronic device includes a memory and a processor. The processor is configured to run a program stored in the memory, wherein the pulse wave acquisition method is executed when the program is run.

[0015] In an embodiment of the present application, an image of a target skin area is acquired, and a first target signal is acquired from the image, wherein the first target signal is a time series signal of pixels in the target skin area, and the time series signal includes a pulse wave signal; a target wavelet having a waveform that is a preset waveform is determined, wherein the preset waveform is a waveform determined based on the waveform of the pulse wave signal; interference signals in the first target signal are removed, and binary wavelet decomposition is performed on the first target signal after the interference signals are removed based on the target wavelet to determine a second target signal, wherein the second target signal includes multiple pulse wave signals; at least one target pulse wave signal is determined from the multiple pulse wave signals in the second target signal, by determining the waveform of the corresponding target wavelet based on the waveform of the pulse wave signal, and performing binary wavelet decomposition on the first target signal through the target wavelet to obtain multiple pulse wave signals, thereby achieving the purpose of efficiently and accurately acquiring the pulse wave signal, thereby realizing the technical effect of accurately extracting detailed information from the pulse wave signal, and thereby solving the technical problem of being unable to conveniently determine the health status of the target object due to the inability to extract the pulse wave signal containing detailed information in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a structural diagram of a computer terminal provided according to an embodiment of the present application;

[0018] Figure 2 1 is a flow chart of a pulse wave acquisition method provided in accordance with an embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of a scenario for obtaining an image of a target area according to an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of a skin reflection model provided according to an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of a first target signal provided according to an embodiment of the present application;

[0022] Figure 6 is a schematic diagram of spectrum information of a first target signal provided according to an embodiment of the present application;

[0023] Figure 7 is a schematic diagram of a filtered interference signal and a pulse wave signal provided according to an embodiment of the present application;

[0024] Figure 8 is a schematic diagram of a pulse waveform provided according to an embodiment of the present application;

[0025] Figure 9 is a schematic diagram of a wavelet signal provided according to an embodiment of the present application;

[0026] Figure 10 is a schematic diagram of a wavelet decomposition process provided according to an embodiment of the present application;

[0027] Figure 11 1 is a waveform comparison diagram of a pulse wave signal obtained in different ways according to an embodiment of the present application;

[0028] Figure 12 1 is a schematic diagram comparing waveforms of pulse wave signals obtained in different ways in a static scenario according to an embodiment of the present application;

[0029] Figure 13 1 is a schematic diagram comparing waveforms of pulse wave signals obtained in different ways in a dynamic scenario according to an embodiment of the present application;

[0030] Figure 14 1 is a schematic diagram comparing waveform characteristics of pulse wave models obtained in different ways according to an embodiment of the present application;

[0031] Figure 15 Schematic diagram of the structure of a pulse wave acquisition device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] The arterial pulse wave is generated when the heart pumps a certain amount of blood into the arteries during contraction. Clinical measurements have shown that pulse waves contain a wealth of information about cardiovascular physiology and pathology. Specifically, the pulse wave is a complex wave formed by blood ejected from the heart through the aorta and reflected off several arterial branches. It can be considered a composite waveform, including ascending, descending, and dicrotic waves. The different pulse morphologies are closely related to cardiovascular health. Currently, the primary device for measuring pulse waves is photoplethysmography (PPG). PPG is a contact-based measurement instrument that detects changes in arterial volume caused by pulse waves by attaching a sensor to the skin surface. However, with the advancement of technology and the increasing interest in intelligent healthcare, non-contact pulse wave measurement is urgently needed. On the one hand, contact measurement methods are not suitable for monitoring patients with infectious diseases or newborns. On the other hand, non-contact measurement is also beneficial in intelligent scenarios such as driver vital sign monitoring.

[0035] Current non-contact methods primarily include radar-based measurement and video-based remote photoplethysmography (rPPG). Radar-based measurement relies on the mechanical effects of heartbeats causing chest vibrations. While this approach is simple in principle and easy to implement, it is susceptible to motion artifacts and suffers from limitations such as limited accuracy and high cost.

[0036] Video-based remote photoplethysmography (rPPG) uses a camera to capture changes in skin color to measure pulse waves. These changes are primarily caused by changes in the volume of blood flow within capillaries due to the heart's pumping. Due to the ease of video signal acquisition, video-based rPPG has been widely studied in recent years. Ming-ZherPoh et al. hypothesized that the three signals observed from the RGB channels of a video are a linear mixture of the arterial pulse wave and two unknown interferences, and proposed using independent component analysis (ICA) to demix the three signals to obtain the pulse wave signal. This method can estimate a pulse wave signal with a higher signal-to-noise ratio than the original signal, but the improvement is limited because the signals observed in real scenes are often more complex than assumed. To achieve a higher signal-to-noise ratio, Kumar et al. proposed segmenting the face into small grids and implementing adaptive region of interest (AROI) selection based on the signal-to-noise ratio within each region. This method optimizes the selection of the measurement region and has a certain degree of noise immunity. However, spectrum-based SNR calculation is prone to making incorrect selections when the noise energy exceeds the signal energy. G. De Haan et al. proposed a chrominance-based (CHROM) method that uses a priori standard skin color vectors to project and offset low-frequency noise to produce a signal with a higher signal-to-noise ratio. However, this predefined skin color vector may no longer be applicable under different lighting conditions and may result in the appearance of mirror residues. Finzgar et al., considering that the pulse wave signal may have a frequency that varies over time, proposed a frequency band division method based on continuous wavelet transform. However, due to the certain overlap between the frequency bands of the interference and pulse wave signals and the use of narrowband, the shape of the recovered pulse wave signal tends to be a sine wave.

[0037] As can be seen from the above, existing remote photoplethysmometers mostly focus on estimating pulse rate, which can only extract a rough pulse wave shape, but cannot obtain an accurate pulse wave signal or the various information contained in the pulse wave signal. However, there is still a lack of research on measuring pulse waves with preserved details using rPPG technology. In this research, Dingliang Wang et al. have made leading progress, proposing the use of singular spectrum analysis (SSA) to extract a detailed pulse wave signal from the observed signal. This method decomposes the signal into different components and then uses the first few principal components to reconstruct the detailed pulse wave. However, the ambiguity of the principal components, whether interference or the principal components of the pulse wave signal, can lead to failure in accurate waveform recovery.

[0038] In order to solve this problem, relevant solutions are provided in the embodiments of the present application, which are described in detail below.

[0039] According to an embodiment of the present application, a method embodiment of a pulse wave acquisition method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a pulse wave acquisition method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (illustrated as 102a, 102b, ..., 102n) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0041] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0042] Memory 104 can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the pulse wave acquisition method in the embodiments of the present application. Processor 102 executes the software programs and modules stored in memory 104 to execute various functional applications and data processing, thereby implementing the pulse wave acquisition method of the aforementioned application. Memory 104 can include high-speed random access memory (RAM) and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, memory 104 can further include memory remotely located relative to processor 102, and such remote memory can be connected to computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0043] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0044] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0045] Under the above operating environment, the embodiment of the present application provides a pulse wave acquisition method, such as Figure 2 As shown, the method includes the following steps:

[0046] Step S202, acquiring an image of the target skin area and obtaining a first target signal from the image, wherein the first target signal is a time sequence signal of pixels in the target skin area, and the time sequence signal includes a pulse wave signal;

[0047] In the technical solution provided in step S202, the step of obtaining an image of the target skin area includes: obtaining a target video, wherein the target video includes a target face image; identifying an eye image in the target face image, and locating a forehead area in the target face image based on the eye image; and determining a target skin area in the forehead area.

[0048] Specifically, in some embodiments of the present application, the method of obtaining the target area image is as follows: Figure 3 As shown. Figure 3As can be seen in the figure, a single camera can continuously capture facial images of the target subject, and image positioning can be performed using the eye images within the facial images, thereby determining the forehead image within the facial image. It should be noted that multiple images of the target skin area are captured. Furthermore, the forehead image is chosen to acquire the pulse wave signal because it is less affected by facial expressions and is home to the superficial temporal artery. This means that when the head image is selected to acquire the pulse signal, the pulse wave intensity in the original signal is relatively high, and interference information, particularly that caused by facial expression changes, is reduced.

[0049] In addition, it is understandable that the facial image does not have to be directly captured by a camera, and other devices may also send the captured video containing the facial image of the facial object to the device responsible for extracting the pulse wave signal.

[0050] In some embodiments of the present application, the first target signal is a timing signal of pixels in the target skin area in the green channel. Specifically, because hemoglobin in blood is most sensitive to light in the green wavelength range (550-600nm), the green channel can be selected to generate the strongest remote photoplethysmography signal.

[0051] Figure 4 is a skin reflection model provided according to an embodiment of the present application. Figure 4 It can be seen that the light signal reflected by the skin can be divided into mirror reflection signal and diffuse reflection signal, and includes the light signal reflected by the skin's epidermis, blackness, capillaries and other physiological tissues. Figure 4 , the timing signal of skin pixel reflection can be described in the form of timing signal. The specific formula is as follows:

[0052] G k (t)=I(t)·(v s (t)+v d (t))+v n (t)

[0053] In the above formula, G k (t) represents the G channel (i.e., the green channel in RGB space) value of the kth skin pixel. I(t) represents the background light captured by the camera, whose intensity may vary with the distance between the light source, the object, and the camera. s (t) and v d (t) are specular reflection and diffuse reflection respectively. v n (t) represents the noise inside the image sensor, including pattern noise and quantization noise.

[0054] Understandably, diffuse reflection v d(t) is related to the absorption and scattering of light by skin tissue, so the following formula can be used to express diffuse reflection:

[0055] v d (t) = d0 + u v v(t)+u p ·p(t)

[0056] In the above formula, d0 represents a fixed reflection factor, v(t) represents a signal generated by physiological pulsation unrelated to the arterial pulse, such as the venous pulse, and u v Indicates the strength of these signals in the G channel, p(t) and u p are the pulse wave signal and average pulse intensity of the G channel respectively.

[0057] The mirror reflection v s (t) mainly appears on the skin surface and can be expressed as the following formula:

[0058] v s (t)=s0+u s Φ(m(t), p(t))

[0059] where s0 is the stationary part of the specular reflection and Φ(m(t), p(t)) represents the varying part that depends on two components: the motion-induced m(t) and the impulse-induced p(t). s represents the signal strength in the G channel. It is worth noting that this surface reflection is primarily caused by non-physiological movements, such as head rotation and facial expressions. However, it is also somewhat affected by the pulse signal, which can originate from chest vibrations induced by the heartbeat.

[0060] After obtaining the expressions of diffuse reflection and specular reflection, the expressions of diffuse reflection and specular reflection are substituted into the expression of the timing signal of skin pixel reflection, and the following formula can be obtained:

[0061] G k (t)=I0·(1+i(t))·(s0+u s Φ(m(t),p(t))+d0+u v v(t)+u p ·p(t))+v n (t)

[0062] In the above formula, since the time-varying component is several orders of magnitude smaller than the stationary component, the product term of the time-varying component can be simply ignored. Let c0 = s0 + d0, representing the reflection intensity. This gives the following formula:

[0063] G(t)≈c0·I0+u s·I0·Φ(m(t),p(t))+c0·I0·i(t)+u v ·I0·v(t)+u p ·I0·p(t)

[0064] The above formula is the expression of the first target signal, where the first target signal is the RAW signal obtained by performing pixel space averaging processing on all pixels in the target area, and then the RAW signal is further processed to obtain the pulse wave signal. The current related art provides a video-based pulse wave extraction method, and the basic task is to extract p(t) from G(t). However, when extracting the pulse wave signal in the related art, either it is assumed that there is a linear relationship between G(t) and p(t), or m(t) and p(t) in Φ(m(t), p(t)) are ignored. These assumptions are acceptable when only a rough waveform is restored to estimate the heart rate. However, when it is necessary to retain the details of the pulse wave, the use of the above assumptions will result in the loss of information in the pulse wave, that is, p(t) cannot be accurately determined from the above formula.

[0065] In order to accurately extract p(t) from the above formula, the signal characteristics of the skin area and the non-skin area can be compared. Figure 5 It can be seen that the first target signal contains a large number of interference signals. And according to Figure 6 After comparison, it can be seen that the low-frequency part of the interference signal is usually regarded as the baseline drift of the observation signal, so a bandpass filter can be used to remove the low-frequency part (i.e. caused by motion and lighting) and the high-frequency part (i.e. flicker of the light source) in the interference. The signal after filtering is as follows Figure 7 As shown in , the residual interference signal is chaotic and has no regular pattern. In addition, considering that the non-pulse physiological beat v(t) may have a similar frequency to p(t), but show different characteristics, the waveform of the arterial pulse wave is a composite of the main wave and the reflected wave, as shown in Figure 8 Therefore, based on the unique shape of the pulse signal, a specific wavelet with a similar shape can be designed to separate the accurate p(t) from the interference signal of the first target signal.

[0066] Step S204, removing interference signals from the first target signal, and performing binary wavelet decomposition on the first target signal after removing the interference signals according to the target wavelet to determine a second target signal, wherein the waveform of the target wavelet corresponds to the waveform of the pulse wave signal;

[0067] Specifically, from Figure 8 As can be seen from the figure, the waveform of the arterial pulse wave can be regarded as a combination of the main wave and the dicrotic wave. Therefore, the following can be designed based on the waveform of the arterial pulse wave: Figure 9The wavelet signal Bior-1.3 is shown. The sharp corners of the wavelet signal Bior-1.3 can match the hill-shaped wave of the arterial pulse wave signal. The specific formula of the wavelet signal Bior-1.3 is as follows:

[0068]

[0069] where Ψ * (t) represents the conjugate of the wavelet basis Ψ(t), a and b represent the scale and translation parameters, respectively. The frequency band of the pulse wave is mainly between 0.8 and 4 Hz, so the corresponding wavelet scale range can be selected to filter the original observation signal.

[0070] In the technical solution provided in step S204, the interference signal in the first target signal is removed, and the first target signal after the interference signal is removed is subjected to binary wavelet decomposition according to the target wavelet to obtain the second target signal. The steps include: performing signal preprocessing on the first target signal to obtain a third target signal, wherein the processing method for performing signal preprocessing on the first target signal includes at least one of the following: removing the DC component in the first target signal, removing the singular point in the first target signal, filtering the third target signal by using a continuous wavelet transform to obtain a fourth target signal; and performing binary wavelet decomposition on the fourth target signal by using the target wavelet to obtain the second target signal.

[0071] As an optional implementation method, the steps of performing initial filtering on the third target signal and processing the filtered third target signal using a continuous wavelet transform include: determining the pseudo frequency corresponding to the continuous wavelet transform; filtering the third target signal based on the pseudo frequency to filter out the first type of interference signal in the third target signal.

[0072] As an optional implementation, the step of performing binary wavelet decomposition on the fourth target signal using the target wavelet to obtain the second target signal includes: determining a target scale parameter corresponding to the target wavelet, wherein the target scale parameter is used to reflect the amplification level of the target wavelet; and decomposing the fourth target signal using an inverse binary wavelet transform based on the target scale parameter, filtering out the second type of interference signal in the fourth target signal, and obtaining the second target signal.

[0073] Specifically, the first type of interference signal in the first target signal is removed, and the first target signal after the interference signal is removed is subjected to binary wavelet decomposition according to the target wavelet to remove the second type of interference signal, thereby determining the complete process of the second target signal as follows: Figure 10 As shown, Figure 10Images (a), (b), and (c) in the figure show the preliminary results of filtering and wavelet decomposition of the first target signal. Before wavelet decomposition, the original signal is initially filtered to improve the accuracy of Bior-1.3 wavelet recognition of pulse wave signals. In order to maintain the original characteristics of the signal, the pseudo-frequency corresponding to the continuous wavelet scale is used to filter out the components that differ greatly from p(t). After filtering the first target signal, the component in G(t) with a large frequency difference from p(t), that is, the first interference signal, can be eliminated. The expression of the remaining part of the first target signal is as follows:

[0074]

[0075] In the above expression, represents the filtering part of Φ(m(t), p(t)), u s ′、u v ′ and u′ p is the factor of the corresponding filter component.

[0076] It can be seen that after filtering out the first interference signal, A rough pulse wave signal has been obtained, but there are still some residual interferences (that is, the second interference signal) that are not easy to separate in the frequency domain. At this time, in order to extract the pulse wave signal from the second interference signal, the amplifier effect of the wavelet signal in the time domain can be utilized. Specifically, for a fixed scale (resolution), the scaling acute angle of the Bior-1.3 wavelet can match the local details of the pulse wave signal. In order to facilitate the selection of a suitable resolution, scale discretization is required. The binary wavelet transform is a semi-discretization of the wavelet coefficients, which samples the scale according to a geometric sequence with a ratio of 2. Therefore, The dyadic wavelet transform (DyWT) of can be expressed as:

[0077]

[0078] The scale j determines the amplification level of the wavelet to match the details of the signal. The inverse dyadic wavelet transform at scale j can be expressed as:

[0079]

[0080] The above formula is the expression of the second target signal. At this time, the waveform corresponding to the second target signal is as follows: Figure 10 Image in (d).

[0081] As can be seen from the above description, the second type of interference signal referred to in the embodiments of this application is an interference signal that is relatively similar to the pulse wave signal and cannot be removed from the frequency domain. The first type of interference signal is an interference signal that deviates significantly from the pulse wave signal and can be removed from the frequency domain. In a specific example, the frequency range of the pulse wave signal is 0.8-5Hz. The second type of interference signal is an interference signal in the frequency band of 0.8-5Hz, while the first type of interference signal is an interference signal with a frequency outside of this frequency band.

[0082] Step S206: determining a pulse wave signal according to the second target signal.

[0083] In the technical solution provided in step S206, the second target signal includes multiple signal components. The step of determining the pulse wave signal based on the second target signal includes: determining a component scale parameter for each of the multiple signal components, and sorting the multiple signal components in ascending order of the component scale parameters to obtain a signal component sequence; determining a target number of target signal components based on the amplification level, wherein the target signal components are the signal components used to determine the pulse wave signal; selecting the first target number of signal components in the signal component sequence as the target signal components, and determining the pulse wave signal based on the target signal components. It should be noted that the amplification level can be determined by the user.

[0084] Specifically, the second target signal can also be decomposed into multiple different signals, and the specific expression is as follows:

[0085]

[0086] Where N in the above formula is the amplification level, and the above formula indicates that the second target signal can be represented as the sum of the inverse transformed signal components at multiple different scales. At this point, the original signal can be decomposed into several components, such as Figure 10 As shown in the image (e) in . Assume that the response of the wavelet pulse wave signal p(t) of the first M scales is the largest. Therefore, the pulse wave signal with the highest signal-to-noise ratio can be estimated from the first M components, that is:

[0087]

[0088] Generally speaking, considering the details and global information of the pulse wave, M can be set to N / 2.

[0089] In summary, in the embodiment of the present application, the forehead is selected as the region of interest (ROI). The tracking of the ROI is completed by identifying the eyes and locating the forehead. Averaging the ROI pixels can remove part of the camera noise and obtain the original signal. In the signal preprocessing stage, in order to remove the influence of the DC component, the signal is first standardized and the abnormal point is identified and compressed. Finally, after wavelet pre-filtering and binary wavelet decomposition with Bior-1.3 wavelet as the kernel, the pulse wave signal with retained details can be extracted. The final pulse wave signal image is as follows: Figure 11 shown. Figure 11 The darker lines are the pulse wave signals obtained by the method provided by this application, and the lighter lines are the true values ​​of the pulse waves measured by the photoplethysmometer. Figure 11 It can be seen that the pulse wave signal restored by the method provided in this application is relatively similar to the true value, including waveform features such as the ascending branch, the descending branch and the dicrotic wave, and can reach a correlation of 91% with the true value.

[0090] The invention adopts the method of acquiring an image of a target skin area and acquiring a first target signal from the image, wherein the first target signal is a time series signal of pixels in the target skin area, and the time series signal includes a pulse wave signal; determining a target wavelet having a waveform that is a preset waveform, wherein the preset waveform is a waveform determined based on the waveform of the pulse wave signal; removing interference signals from the first target signal, and performing binary wavelet decomposition on the first target signal after removing the interference signals based on the target wavelet to determine a second target signal, wherein the second target signal includes multiple pulse wave signals; determining at least one target pulse wave signal from multiple pulse wave signals in the second target signal, by determining the waveform of the corresponding target wavelet based on the waveform of the pulse wave signal, and performing binary wavelet decomposition on the first target signal through the target wavelet to obtain multiple pulse wave signals, thereby achieving the purpose of efficiently and accurately acquiring pulse wave signals, thereby realizing the technical effect of accurately extracting detailed information from pulse wave signals, and thereby solving the technical problem of being unable to conveniently determine the health status of the target object due to the inability to extract pulse wave signals containing detailed information in related technologies.

[0091] Furthermore, the method provided in the embodiments of the present application does not require additional precision instruments when extracting pulse wave signals from video signals, making the system simple and easy to deploy. Furthermore, the pulse wave signals recovered in the embodiments of the present application contain more waveform features that can be used for clinical research.

[0092] In some embodiments of the present application, in order to further verify the accuracy of the method provided in the embodiments of the present application in extracting pulse wave signals, the pulse wave signals obtained by the pulse wave acquisition method provided in the present application are compared with the pulse wave signals obtained by other methods, and the mean absolute error (MAE), root mean square error (RMSE), standard deviation (SD) and Pearson correlation coefficient (PCCS) between the waveform restored by each method and the waveform measured by PPG are used to evaluate the integrity of the waveform. The performance test was also tested in two different scenarios, including static scenes and dynamic scenes. The subjects in the dynamic scene were allowed to make small head movements. The results are as follows. Figure 12 and Figure 13 As shown, Figure 12 Schematic diagram of pulse wave signals obtained by different methods in static scenes. Figure 13 The following is a schematic diagram of a pulse wave signal in a dynamic scenario. While the waveforms returned by these methods all extract cardiac cycle information, the waveform details vary. The table below lists the relative errors of each method. Compared with other methods, the pulse wave extracted by the proposed method has lower MAE, RMSE, and SD. The correlations under static and dynamic conditions are 92% and 76%, respectively, significantly better than the 78% and 65% of the existing state-of-the-art methods.

[0093]

[0094]

[0095] Figure 14 is a characteristic diagram of the pulse wave signal obtained by different methods, where Figure 14 The "*," "▲," and "●" in the diagram mark the locations of peaks, troughs, and dicrotic waves, respectively. Most physiological studies of the arterial pulse wave are based on its characteristics, as this waveform arises from the cardiac-synchronized changes in blood volume with each heartbeat. The table below shows the mean absolute error of pulse wave amplitude (PWA), dicrotic wave amplitude (DWA), pulse duration (PWD), systolic duration (SPD), and diastolic duration (DPD).

[0096]

[0097] From the above table and Figure 14It can be seen that the pulse wave recovered by the pulse wave acquisition method proposed in this application contains accurate waveform features. Kumar's method merges signals from different areas of the face according to the signal-to-noise ratio, so it causes the smallest signal offset and the smallest PWD error, which is 15.66ms. However, it performs poorly in details such as dicrotic waves, systole and diastole. Both the ICA and CHROM methods regard the pulse wave and interference as a linear mixture. Therefore, the pulse wave acquisition method in the related art can only recover an approximate waveform. Although the pulse wave extracted by the SSA method also retains details, the waveform features extracted by the pulse wave acquisition method provided in this application are more obvious and the difference is smaller than that of the SSA method.

[0098] The embodiment of the present application provides a pulse wave acquisition device, Figure 15 is a structural diagram of the device, such as Figure 15 As shown, the device includes: an acquisition module 150, used to acquire an image of the target skin area and obtain a first target signal from the image, wherein the first target signal is a time series signal of pixels in the target skin area, and the time series signal includes a pulse wave signal; a first processing module 152, used to remove interference signals from the first target signal, and perform binary wavelet decomposition on the first target signal after removing the interference signals according to the target wavelet to determine a second target signal, wherein the waveform of the target wavelet corresponds to the waveform of the pulse wave signal; and a second processing module 154, used to determine the pulse wave signal according to the second target signal.

[0099] In some embodiments of the present application, the step of acquiring the image of the target skin area by the acquisition module 150 includes: acquiring a target video, wherein the target video includes a target face image; identifying an eye image in the target face image, and locating the forehead area in the target face image based on the eye image; and determining the target skin area in the forehead area.

[0100] In some embodiments of the present application, the first target signal is a timing signal of pixels in the target skin area in a green channel.

[0101] In some embodiments of the present application, the first processing module 152 removes the interference signal in the first target signal, and performs binary wavelet decomposition on the first target signal after the interference signal is removed according to the target wavelet, and the step of obtaining the second target signal includes: performing signal preprocessing on the first target signal to obtain a third target signal, wherein the processing method for performing signal preprocessing on the first target signal includes at least one of the following: removing the DC component in the first target signal, removing the singular point in the first target signal; filtering the third target signal using a continuous wavelet transform to obtain a fourth target signal; performing binary wavelet decomposition on the fourth target signal through the target wavelet to obtain the second target signal.

[0102] In some embodiments of the present application, the first processing module 152 uses continuous wavelet transform to filter the third target signal, and the steps of obtaining the fourth target signal include: determining the pseudo frequency corresponding to the continuous wavelet transform; filtering the third target signal according to the pseudo frequency to filter out the first type of interference signal in the third target signal.

[0103] In some embodiments of the present application, the first processing module 152 performs binary wavelet decomposition on the fourth target signal through the target wavelet to obtain the second target signal, including: determining a target scale parameter corresponding to the target wavelet, wherein the target scale parameter is used to reflect the amplification level of the target wavelet; based on the target scale parameter, decomposing the fourth target signal using a binary wavelet transform, filtering out the second type of interference signal in the fourth target signal, and obtaining the second target signal.

[0104] In some embodiments of the present application, the second target signal includes multiple signal components; the step of the second processing module 154 determining the pulse wave signal based on the second target signal includes: determining the component scale parameter of each signal component in the multiple signal components, and sorting the multiple signal components in order from small to large according to the component scale parameters to obtain a signal component sequence; determining the target number of target signal components based on the amplification level, wherein the target signal component is the signal component used to determine the pulse wave signal; selecting the first target number of signal components in the signal component sequence as the target signal component, and determining the pulse wave signal based on the target signal component.

[0105] It should be noted that the various modules in the above-mentioned pulse wave acquisition device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0106] An embodiment of the present application provides a non-volatile storage medium, in which a program is stored. When the program is running, the device where the non-volatile storage medium is located is controlled to execute the following pulse wave acquisition method: acquiring an image of a target skin area and acquiring a first target signal from the image, wherein the first target signal is a timing signal of pixels in the target skin area, and the timing signal includes a pulse wave signal; removing interference signals from the first target signal, and performing binary wavelet decomposition on the first target signal after removing the interference signals based on a target wavelet to determine a second target signal, wherein the waveform of the target wavelet corresponds to the waveform of the pulse wave signal; and determining the pulse wave signal based on the second target signal.

[0107] An embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the program, when running, executes the following pulse wave acquisition method: acquiring an image of a target skin area and acquiring a first target signal from the image, wherein the first target signal is a time series signal of pixels in the target skin area, and the time series signal includes a pulse wave signal; removing interference signals from the first target signal, and performing binary wavelet decomposition on the first target signal after removing the interference signals based on a target wavelet to determine a second target signal, wherein a waveform of the target wavelet corresponds to a waveform of the pulse wave signal; and determining the pulse wave signal based on the second target signal.

[0108] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0110] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0111] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0113] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A pulse wave acquisition method, characterized in that: include: Acquire an image of a target skin area, and acquire a first target signal from the image, wherein the first target signal is a time sequence signal of pixels in the target skin area, and the time sequence signal includes a pulse wave signal; removing interference signals from the first target signal, and performing binary wavelet decomposition on the first target signal after removing the interference signals according to a target wavelet to determine a second target signal, wherein a waveform of the target wavelet corresponds to a waveform of the pulse wave signal, and a wavelet scale range of the target wavelet corresponds to a frequency band range of the pulse wave signal; Determining the pulse wave signal based on the second target signal includes: determining a component scale parameter of each signal component in a plurality of signal components, and sorting the plurality of signal components in ascending order of the component scale parameters to obtain a signal component sequence, wherein the second target signal includes a plurality of the signal components; determining a target number of target signal components based on the amplification level of the target wavelet, wherein the target signal components are signal components used to determine the pulse wave signal; selecting the first target number of signal components in the signal component sequence as the target signal components, and determining the pulse wave signal based on the target signal components.

2. The pulse wave acquisition method according to claim 1, characterized in that: The step of acquiring an image of the target skin area comprises: Acquire a target video, wherein the target video includes a target face image; Identifying an eye image in the target face image, and locating a forehead region in the target face image based on the eye image; The target skin area is determined in the forehead area.

3. The pulse wave acquisition method according to claim 1, characterized in that: The first target signal is a timing signal of pixels in the target skin area in a green channel.

4. The pulse wave acquisition method according to claim 1, wherein: The step of removing the interference signal from the first target signal and performing dyadic wavelet decomposition on the first target signal after removing the interference signal according to the target wavelet to obtain the second target signal includes: performing signal preprocessing on the first target signal to obtain a third target signal, wherein the signal preprocessing on the first target signal comprises at least one of the following: removing a DC component in the first target signal, and removing a singular point in the first target signal; Performing filtering processing on the third target signal by using a continuous wavelet transform to obtain a fourth target signal; The fourth target signal is subjected to dyadic wavelet decomposition by using the target wavelet to obtain the second target signal.

5. The pulse wave acquisition method according to claim 4, characterized in that: The step of filtering the third target signal by adopting a continuous wavelet transform to obtain a fourth target signal comprises: Determining a pseudo frequency corresponding to the continuous wavelet transform; The third target signal is filtered according to the pseudo frequency to remove the first type of interference signal in the third target signal.

6. The pulse wave acquisition method according to claim 4, characterized in that: The step of performing dyadic wavelet decomposition on the fourth target signal by using the target wavelet to obtain the second target signal comprises: Determining a target scale parameter corresponding to the target wavelet, wherein the target scale parameter is used to reflect the amplification level of the target wavelet; According to the target scale parameter, the fourth target signal is decomposed and processed by adopting a dyadic wavelet inverse transform method, and the second type of interference signal in the fourth target signal is filtered out to obtain the second target signal.

7. A pulse wave acquisition device, characterized in that: include: an acquisition module, configured to acquire an image of a target skin area and obtain a first target signal from the image, wherein the first target signal is a time sequence signal of pixels in the target skin area, and the time sequence signal includes a pulse wave signal; a first processing module, configured to remove interference signals from the first target signal and perform dyadic wavelet decomposition on the first target signal after removing the interference signals according to a target wavelet to determine a second target signal, wherein a waveform of the target wavelet corresponds to a waveform of the pulse wave signal, and a wavelet scale range of the target wavelet corresponds to a frequency band range of the pulse wave signal; A second processing module is configured to determine the pulse wave signal based on the second target signal, comprising: determining a component scale parameter of each signal component among a plurality of signal components, and sorting the plurality of signal components in ascending order of the component scale parameters to obtain a signal component sequence, wherein the second target signal includes a plurality of the signal components; determining a target number of target signal components based on an amplification level of the target wavelet, wherein the target signal components are signal components used to determine the pulse wave signal; selecting the target number of signal components from the signal component sequence as the target signal components, and determining the pulse wave signal based on the target signal components.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the pulse wave acquisition method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the pulse wave acquisition method according to any one of claims 1 to 6 is executed when the program is run.

Images