Pulse wave signal extraction method, extraction device, storage medium and electronic equipment

Abstract

This invention discloses a method, device, storage medium, and electronic device for extracting pulse wave signals. The method includes: acquiring video data of the target object, wherein the video data includes multiple video frames and timestamps corresponding to each video frame; determining the information value of each video frame and obtaining a time-series-based vibration signal based on the information value and the corresponding timestamp; inputting the time-series-based vibration signal into a preset spatial vector decomposition model to output a pulse wave signal spatial matrix; and normalizing the pulse wave signal spatial matrix to obtain the pulse wave signal of the target object. Therefore, this method can obtain a pure pulse wave signal of the target object comparable to that of a contact pulse wave sensor.

Description

Technical Field

[0001] This invention relates to the fields of bioengineering and artificial intelligence, and in particular to a method for extracting pulse wave signals, a device for extracting pulse wave signals, a computer-readable storage medium, and an electronic device. Background Technology

[0002] In related technologies, when obtaining pulse wave signals from video (image sequences), frequency domain filtering is primarily used to remove noise outside the effective frequency band (1 Hz to 3.33 Hz) to obtain the pulse wave signal. A few methods obtain a heart rate-related signal by detecting hemoglobin volume. However, the signals obtained using these techniques are not pure because noise within the frequency band is not removed, and the signal obtained from hemoglobin has a weak correlation with the pulse wave signal. Noise constitutes a large proportion of the signal and may even completely obscure the pulse wave signal. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the first objective of this invention is to provide a method for extracting pulse wave signals that yields a pure pulse wave signal from the target object comparable to that of a contact pulse wave sensor.

[0004] The second objective of this invention is to provide a device for extracting pulse wave signals.

[0005] A third objective of this invention is to provide a computer-readable storage medium.

[0006] The fourth objective of this invention is to provide an electronic device.

[0007] To achieve the above objectives, a first aspect of the present invention proposes a method for extracting pulse wave signals, comprising the following steps: acquiring video data of the target being tested, wherein the video data includes multiple video images and timestamps corresponding to each video image; determining the information value of each video image, and obtaining a vibration signal based on a time series according to the information value and the corresponding timestamp; inputting the vibration signal based on the time series into a preset spatial vector decomposition model, and outputting a pulse wave signal spatial matrix; and normalizing the pulse wave signal spatial matrix to obtain the pulse wave signal of the target being tested.

[0008] According to one embodiment of the present invention, determining the information value of a video image includes: preprocessing the video image to obtain a region of interest (ROI); performing RGB color vector decomposition on the ROI to obtain an RGB color matrix; and calculating the average value of each color component in the RGB color matrix to obtain the corresponding information value.

[0009] According to an embodiment of the present invention, obtaining a time-series-based vibration signal based on the information values ​​and corresponding timestamps includes: combining the information values ​​of each video image according to a time sequence to obtain a vibration signal matrix, and combining the timestamps of each video image according to a time sequence to obtain a timestamp matrix; combining the vibration signal matrix and the timestamp matrix to obtain the time-series-based vibration signal.

[0010] Furthermore, after obtaining the vibration signal matrix, the method further includes: inputting the vibration signal matrix into a preset first filter to filter the vibration signal matrix; and / or after obtaining the pulse wave signal, the method further includes: inputting the pulse wave signal into a preset second filter to filter the pulse wave signal.

[0011] According to an embodiment of the present invention, the preset spatial vector decomposition model includes a first spatial matrix and a second spatial matrix, wherein the pulse wave signal spatial matrix is ​​obtained by the following formula: the pulse wave signal spatial matrix = the first spatial matrix * (the second spatial matrix * the time-series-based vibration signal), wherein the shape of the first spatial matrix is ​​1*3, the shape of the second spatial matrix is ​​3*4, the shape of the time-series-based vibration signal is 4*N, and N is the number of video images.

[0012] According to one embodiment of the present invention, before inputting the time-series-based vibration signal into a preset spatial vector decomposition model, the method further includes: performing frame interpolation processing on the time-series-based vibration signal according to the timestamp.

[0013] Further, the step of performing frame interpolation processing on the time-series-based vibration signal according to the timestamp includes: calculating the difference between the timestamps of any two adjacent information values ​​in the time-series-based vibration signal; if the difference is greater than a first preset time, determining that frame interpolation processing of the time-series-based vibration signal is required, and determining the number of frames to be interpolated according to the difference; obtaining the information value to be inserted and its corresponding timestamp according to the number, two adjacent information values ​​and their timestamps; and inserting the information value to be inserted and its corresponding timestamp into the time-series-based vibration signal.

[0014] According to an embodiment of the present invention, the spatial matrix of the pulse wave signal is normalized using the following formula: Among them, w k Ψ(k) is the k-th element in the pulse wave signal space matrix, n is the number of elements in the pulse wave signal space matrix, α is an empirical coefficient, and Ψ(k) is the k-th element of the pulse wave signal obtained after normalization.

[0015] According to an embodiment of the present invention, a method for extracting pulse wave signals involves acquiring video data of a target, wherein the video data includes multiple frames of video images and timestamps corresponding to each video image; determining the information value of each frame of video image, and obtaining a time-series-based vibration signal based on the information value and the corresponding timestamp; inputting the time-series-based vibration signal into a preset spatial vector decomposition model to output a pulse wave signal spatial matrix; and normalizing the pulse wave signal spatial matrix to obtain the pulse wave signal of the target. Thus, this method for extracting pulse wave signals, by inputting a time-series-based vibration signal into a preset spatial vector decomposition model to output a pulse wave signal spatial matrix, and by normalizing the pulse wave signal spatial matrix, obtains a pure pulse wave signal of the target comparable to that of a contact pulse wave sensor.

[0016] To achieve the above objectives, a second aspect of the present invention provides a pulse wave signal extraction device, comprising: an acquisition module for acquiring video data of a target, wherein the video data includes multiple video frames and timestamps corresponding to each video frame; a determination module for determining the information value of each video frame and obtaining a time-series-based vibration signal based on the information value and the corresponding timestamp; a generation module for inputting the time-series-based vibration signal into a preset spatial vector decomposition model and outputting a pulse wave signal spatial matrix; and a processing module for normalizing the pulse wave signal spatial matrix to obtain the pulse wave signal of the target.

[0017] According to an embodiment of the present invention, a pulse wave signal extraction device acquires video data of a target, wherein the video data includes multiple frames of video images and timestamps corresponding to each video image; determines the information value of each frame of video image, and obtains a time-series-based vibration signal based on the information value and the corresponding timestamp; inputs the time-series-based vibration signal into a preset spatial vector decomposition model, and outputs a pulse wave signal spatial matrix; and performs normalization processing on the pulse wave signal spatial matrix to obtain the pulse wave signal of the target. Thus, this pulse wave signal extraction device, by inputting a time-series-based vibration signal into a preset spatial vector decomposition model, outputting a pulse wave signal spatial matrix, and performing normalization processing on the pulse wave signal spatial matrix, obtains a pure pulse wave signal of the target comparable to that of a contact pulse wave sensor.

[0018] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for extracting pulse wave signals.

[0019] To achieve the above objectives, a fourth aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the method for extracting pulse wave signals.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] Figure 1 This is a schematic flowchart of a pulse wave signal extraction method according to an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the camera receiving imaging light path according to an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of a pulse wave signal extraction device according to an embodiment of the present invention. Detailed Implementation

[0024] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0025] The following is a reference appendix. Figure 1-3 This invention describes a method, apparatus, storage medium, and electronic device for extracting pulse wave signals according to embodiments of the present invention.

[0026] In related technologies, extracting physiological signals to characterize heart rate basically involves three steps: signal acquisition, signal processing, and heart rate calculation. The signal acquisition process mainly involves: acquiring multiple frames of images containing a face using a camera (video); extracting the region of interest (ROI) from each frame; performing RGB decomposition on the ROI; and calculating the mean of each RGB image. Finally, the multiple means are combined into three mean matrices. The main process of signal processing is as follows: select one or more matrices from three sequences, perform signal merging or no processing, and input them into a frequency domain filter to obtain the filtered signal. The method for calculating heart rate is: using FFT (Fast Fourier Transform) in the amplitude-frequency domain, select the frequency point with the highest energy, and multiply this frequency point by 60 to obtain the heart rate. The physiological signal obtained by this technique does not remove noise signals within the frequency band and is fundamentally not a pulse wave signal. This is because such noise is often very large, and the target signal is usually submerged in the noise.

[0027] Figure 1 This is a schematic flowchart of a pulse wave signal extraction method according to an embodiment of the present invention. Figure 1 As shown, the method for extracting pulse wave signals includes the following steps:

[0028] S101. Collect video data of the target under test, wherein the video data includes multiple video images and timestamps corresponding to each video image.

[0029] Specifically, the pulse wave signal extraction method of this invention can be implemented by a specific pulse wave signal extraction device or electronic device such as a mobile phone. When acquiring video data of the target, the video data of the target can be acquired by a camera (such as a smartphone camera, an industrial camera, or a surveillance camera). The target can be a human face.

[0030] S102. Determine the information value of each frame of video image, and obtain the vibration signal based on the time series according to the information value and the corresponding timestamp.

[0031] In one example, determining the information value of a video image may include: preprocessing the video image to obtain the region of interest; performing RGB color vector decomposition on the region of interest to obtain an RGB color matrix; and calculating the average value of each color component in the RGB color matrix to obtain the corresponding information value.

[0032] Specifically, taking a face image in a video image as an example, regions of interest (ROIs) can be found using a facial landmark model, such as the forehead, chin, and nose regions. In other words, the function of the facial landmark model is to extract the portion containing the facial skin area (i.e., the ROI of the video image) from the entire face image. Performing RGB decomposition on N (N≥300) ROI images yields N color matrices of interest, with a shape of 3*N, which can be approximated as... Let represent the matrix composed of all R components in the Nth region of interest (ROI) image. The mathematical average of each row of the ROI color matrix in each frame yields N average matrices, each with a 3x1 shape, denoted as . Thus, the information value of each frame of video image is obtained.

[0033] It should be noted that when obtaining the information values ​​of the region of interest in a video image, the video image can be processed frame by frame. For example, during the recording process, the first frame of the video image can be processed first to obtain the corresponding information values, then the second frame of the video image can be processed to obtain the corresponding information values, and so on, until the number of information values ​​for outputting a pulse wave signal is reached. Alternatively, after obtaining the video data from the camera, a preset number of video images can be obtained directly from the video data, and then each video image can be processed simultaneously.

[0034] Furthermore, the vibration signal based on the information value and the corresponding timestamp is obtained, including: combining the information values ​​of each video image according to the time sequence to obtain a vibration signal matrix, and combining the timestamps of each video image according to the time sequence to obtain a timestamp matrix; combining the vibration signal matrix and the timestamp matrix to obtain the vibration signal based on the time sequence.

[0035] Specifically, all average matrices are synthesized into a 3*N matrix according to the time series. This refers to the vibration signal matrix. It should be noted that after obtaining the vibration signal matrix, it can be input into a preset first filter to perform filtering. The first filter can be... It belongs to a type of time-frequency domain filter, where x(n) is the input signal. It is an auxiliary function, and is itself a 'a' is an empirical coefficient, 'n' is the independent variable, and 'm' is an empirical parameter, where 'm' ranges from 1 to 1. Passing the vibration signal matrix obtained in the previous step through the first filter yields the following new vibration signal matrix: p is merely a placeholder, representing only a single number. The timestamps of each video image are combined according to a time sequence to obtain a timestamp matrix. The vibration signal matrix is ​​then combined with the timestamp matrix to obtain a time-series-based vibration signal.

[0036] S103. Input the time-series-based vibration signal into the preset spatial vector decomposition model and output the pulse wave signal spatial matrix.

[0037] The preset spatial vector decomposition model includes a first spatial matrix and a second spatial matrix. The first spatial matrix has a shape of 1*3, the second spatial matrix has a shape of 3*4, and the vibration signal based on the time series has a shape of 4*N, where N is the number of video images. The pulse wave signal spatial matrix is ​​obtained by the following formula: Pulse wave signal spatial matrix = First spatial matrix * (Second spatial matrix * Vibration signal based on time series).

[0038] Specifically, the first spatial matrix can be represented as [c1c2c3], and the second spatial matrix can be represented as... The specific values ​​of the spatial vectors are obtained empirically, and 'c' is merely a symbol and does not represent any actual meaning. First, the second spatial matrix is ​​multiplied by the vibration signal based on the time series. The resulting 3*N matrix is ​​a mixture. Each row represents a space, and 'p' in each row represents only the symbol, indicating the color information value in the spatial vector decomposition model. Then, the first spatial matrix is ​​multiplied by the mixed 3*N matrix. Finally, the pulse wave signal spatial matrix is ​​obtained, [w1 w2 w3 …w N-1 w N The shape is 1*N, where w only represents the pulse wave signal and does not represent any actual meaning.

[0039] As an example, the time intervals between two adjacent video images may be unequal. Therefore, before inputting the time-series-based vibration signal into the preset spatial vector decomposition model, the process may further include: performing frame interpolation processing on the time-series-based vibration signal according to timestamps. This frame interpolation processing may include: calculating the difference between the timestamps of any two adjacent information values ​​in the time-series-based vibration signal; if the difference is greater than a first preset time, determining that frame interpolation processing is needed for the time-series-based vibration signal, and determining the number of frames to be interpolated based on the difference; obtaining the information value to be interpolated and its corresponding timestamp based on the number of frames, the two adjacent information values, and their timestamps; and inserting the information value to be interpolated and its corresponding timestamp into the time-series-based vibration signal.

[0040] Specifically, the difference is obtained by subtracting the timestamps of any two adjacent information values ​​in the time-series vibration signal. If the difference is greater than a first preset time (e.g., 30ms), frame interpolation is required for the time-series vibration signal, and the number of frames to be interpolated is determined based on the difference. For example, if the difference between the timestamps of two adjacent time-series vibration signals is greater than 30ms, frame interpolation is required; if the difference is between 33ms and 66ms, one frame is required; if the difference is between 66ms and 99ms, two frames are required, and so on. Then, based on the required number of frames to be interpolated, any two adjacent information values ​​in the time-series vibration signal, and their timestamps, the information value to be inserted and its corresponding timestamp are obtained. For example, the signal values ​​of the time-series vibration signals on the left and right sides of the interpolation point are assumed to be y1 and y2, and the time points of the two signal values ​​are assumed to be x1 and x2. A fitting function is created. The missing time length t0 is calculated as x2 - x1. The predicted missing time point is usually assumed to be two points, i.e., t Δ=t0÷3 (rounded to four decimal places), the first time point is t1 = t0 + t Δ The second time point is t2 = t0 + 2t Δ Substituting the two time points (i.e., the corresponding timestamps) into a first-order fitting function yields... That is, two signal values ​​to be inserted. Therefore, the position where the frame needs to be inserted and the information value to be inserted are found based on the timestamp. Finally, the information value to be inserted and its corresponding timestamp are inserted into the time-series-based vibration signal.

[0041] S104. Normalize the spatial matrix of the pulse wave signal to obtain the pulse wave signal of the target being measured.

[0042] The pulse wave signal spatial matrix is ​​normalized using the following formula: w k Ψ(k) is the k-th element in the pulse wave signal space matrix, n is the number of elements in the pulse wave signal space matrix, α is the empirical coefficient, and Ψ(k) is the k-th element of the pulse wave signal obtained after normalization.

[0043] Specifically, firstly, the pulse wave signal spatial matrix is ​​normalized to obtain the pulse wave-like signal of the target being measured, [Ψ1 Ψ2 Ψ3 … Ψ]. N It should be noted that the magnitude of the pulse-like wave signal of the target represents the amplitude of vasoconstriction and vasodilation, and the amplitude value represents the amplitude of the pulse wave. Then, the pulse-like wave signal of the target is multiplied by an empirical coefficient ξ, ξ*[Ψ1 Ψ2 Ψ3 … Ψ]. N The pulse wave signal of the target being measured is obtained.

[0044] As an example, after obtaining the pulse wave signal, the pulse wave signal can also be input into a preset second filter to filter the pulse wave signal.

[0045] Specifically, the obtained pulse wave signal is input to a preset second filter to filter the pulse wave signal. The second filter can be, a k a m Let be the number of filter groups, determined by the signal length; x(n) be the input signal; y(n) be the output signal; and m and k be the translation distances, determined by the signal length itself. Therefore, the signal after passing through the filter is a pure pulse wave signal.

[0046] In summary, this pulse wave signal extraction method, by inputting a time-series-based vibration signal into a preset spatial vector decomposition model, outputting a pulse wave signal spatial matrix, and then normalizing the pulse wave signal spatial matrix, yields a pure pulse wave signal of the target object comparable to that of a contact pulse wave sensor. This pure pulse wave can not only be used to calculate heart rate but also to extract more physiological indicators, such as blood oxygen saturation, wave velocity, heart rate variability, and blood pressure.

[0047] The principles for obtaining the first and second space matrices mentioned above can be achieved through... Figure 2 The optical path experiment shown is illustrated.

[0048] Figure 2 This is a schematic diagram of the camera receiving imaging light path according to an embodiment of the present invention. Figure 2 As shown, based on the reflection, the imaging light received by camera 01 under normal circumstances mainly consists of the following three parts: ambient reflected light 02, light from the light source 03; light reflected from the skin epidermis 04 and dermis 05; and light reflected from the subcutaneous dermis 06 and blood vessels 07. Ignoring reflected light 02 and light from the light source 03, the total illumination intensity can be approximated by the following formula:

[0049] C K =I(t)*(u s *(s0+s(t))+u d *d0+u p *p(t))+v n(t) ,

[0050] Where: I(t) represents the illumination intensity level, which can be understood as the change in light source intensity and the change in distance from light source 03 to the skin and from the skin to camera 01. us represents the unit color vector of the spectrum, s0 represents the static reflection part, s(t) represents the dynamic reflection part (motion information), ud represents the unit color vector of skin tissue, d0 represents the static reflection intensity, up represents the relative pulse wave signal intensity in the RGB channel, p(t) represents the pulse wave signal, and vn(t) represents the noise of camera 01 itself. The noise of camera 01 itself is determined by the hardware of camera 01 and is a quantity that can be obtained from inside camera 01. Therefore, this parameter will not be mentioned in the following text.

[0051] Assuming the noise of camera 01 itself is known (the noise can be directly obtained from the hardware), the value reflected by the three original skin colors (such as white, yellow, and black) under pure white light source 03 will be defined as 1. The approximate formula is as follows:

[0052]

[0053] Physiological signals can be obtained using this formula. Where p(t) is the pulse wave signal of the t-th frame of the video image, I0 is the unit intensity of light intensity changing with time, and u... p Let s(t) represent the relative pulse wave signal intensity in the RGB channels, s(t) represent the dynamic reflection portion, i(t) represent the incident light intensity of the t-th frame of the video image, and u represent the relative pulse wave signal intensity in the RGB channels. s Let N be the unit color vector of the spectrum, N be the static light reflection intensity, and Cn(t) be the total illuminance. Let p(t) be the original light intensity. Assuming the light intensity does not change with time, then the only change is p(t), which is the true pulse wave signal. Therefore, based on this principle, the first and second spatial matrices can be obtained experimentally.

[0054] Figure 3 This is a schematic diagram of a pulse wave signal extraction device according to an embodiment of the present invention. Figure 3 As shown, the pulse wave signal extraction device 100 includes: an acquisition module 10 for acquiring video data of the target, wherein the video data includes multiple video images and timestamps corresponding to each video image; a determination module 20 for determining the information value of each video image and obtaining a time-series-based vibration signal based on the information value and the corresponding timestamp; a generation module 30 for inputting the time-series-based vibration signal into a preset spatial vector decomposition model and outputting a pulse wave signal spatial matrix; and a processing module 40 for normalizing the pulse wave signal spatial matrix to obtain the pulse wave signal of the target.

[0055] In one embodiment of the present invention, the pulse wave signal extraction device may further include a filtering module, which is used to input the vibration signal matrix into a preset first filter after obtaining the vibration signal matrix, so as to filter the vibration signal matrix; and / or to input the pulse wave signal into a preset second filter after obtaining the pulse wave signal, so as to filter the pulse wave signal.

[0056] It should be noted that for other specific embodiments of the pulse wave signal extraction device of the present invention, please refer to the specific embodiments of the pulse wave signal extraction method of the present invention.

[0057] In summary, this pulse wave signal extraction device obtains a pure pulse wave signal of the target that is comparable to that of a contact pulse wave sensor by inputting a time-series-based vibration signal into a preset spatial vector decomposition model, outputting a pulse wave signal spatial matrix, and normalizing the pulse wave signal spatial matrix.

[0058] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for extracting the pulse wave signal.

[0059] This embodiment also provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the pulse wave signal extraction method.

[0060] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0061] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0062] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0063] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0064] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0065] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0066] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0067] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for extracting pulse wave signals, characterized in that, Includes the following steps: Collect video data of the target under test, wherein the video data includes multiple video images and timestamps corresponding to each video image; The information value of each video frame is determined, and a vibration signal based on a time series is obtained based on the information value and the corresponding timestamp. Before inputting the time-series-based vibration signal into a preset spatial vector decomposition model, the method further includes: The time-series-based vibration signal is subjected to frame interpolation based on the timestamp; The step of performing frame interpolation processing on the time-series-based vibration signal according to the timestamp includes: Calculate the difference between the timestamps of any two adjacent information values ​​in the time-series-based vibration signal; if the difference is greater than a first preset time, it is determined that the time-series-based vibration signal needs to be frame-interpolated, and the number of frames to be interpolated is determined according to the difference; based on the number, the two adjacent information values ​​and their timestamps, the information value to be inserted and its corresponding timestamp are obtained; insert the information value to be inserted and its corresponding timestamp into the time-series-based vibration signal. A first-order fitting function is established based on the signal values ​​on the left and right sides of the interpolation position and their corresponding time points; the intermediate time point to be inserted is determined based on the missing duration of the time points on the left and right sides; the intermediate time point is substituted into the first-order fitting function to obtain the corresponding signal value to be inserted; the signal value to be inserted and its corresponding timestamp are inserted into the vibration signal based on the time series. The time-series-based vibration signal is input into a preset spatial vector decomposition model, and the pulse wave signal spatial matrix is ​​output. The pulse wave signal spatial matrix is ​​normalized to obtain the pulse wave signal of the target being measured.

2. The method for extracting pulse wave signals as described in claim 1, characterized in that, Determine the information values ​​of the video image, including: The video image is preprocessed to obtain the region of interest (ROI) of the video image; The region of interest is decomposed into RGB color vectors to obtain an RGB color matrix. The average value is calculated for each color component in the RGB color matrix to obtain the corresponding information value.

3. The method for extracting pulse wave signals as described in claim 1, characterized in that, The step of obtaining the time-series-based vibration signal based on the information value and the corresponding timestamp includes: The information values ​​of each video image are combined according to a time sequence to obtain a vibration signal matrix, and the timestamps of each video image are combined according to a time sequence to obtain a timestamp matrix. The vibration signal matrix is ​​combined with the timestamp matrix to obtain the time-series-based vibration signal.

4. The method for extracting pulse wave signals as described in claim 3, characterized in that, After obtaining the vibration signal matrix, the method further includes: inputting the vibration signal matrix into a preset first filter to perform filtering processing on the vibration signal matrix; and / or After obtaining the pulse wave signal, the method further includes: inputting the pulse wave signal into a preset second filter to filter the pulse wave signal.

5. The method for extracting pulse wave signals as described in claim 1, characterized in that, The preset spatial vector decomposition model includes a first spatial matrix and a second spatial matrix, wherein the pulse wave signal spatial matrix is ​​obtained by the following formula: The pulse wave signal spatial matrix = the first spatial matrix * (the second spatial matrix * the time-series-based vibration signal). The first spatial matrix has a shape of 1*3, the second spatial matrix has a shape of 3*4, and the vibration signal based on the time series has a shape of 4*N, where N is the number of video images.

6. The method for extracting pulse wave signals as described in claim 1, characterized in that, The pulse wave signal spatial matrix is ​​normalized using the following formula: ， in, Let be the k-th element in the pulse wave signal space matrix, n be the number of elements in the pulse wave signal space matrix, and α be an empirical coefficient. This represents the k-th element of the pulse wave signal obtained after normalization.

7. A device for extracting pulse wave signals, characterized in that, For implementing the method as described in any one of claims 1-6, comprising: The acquisition module is used to acquire video data of the target under test, wherein the video data includes multiple video images and timestamps corresponding to each video image; The determination module is used to determine the information value of each frame of video image and obtain a vibration signal based on the time series according to the information value and the corresponding timestamp; The generation module is used to input the time-series-based vibration signal into a preset spatial vector decomposition model and output a pulse wave signal spatial matrix. The processing module is used to normalize the spatial matrix of the pulse wave signal to obtain the pulse wave signal of the target being measured.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for extracting pulse wave signals as described in any one of claims 1-6.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for extracting pulse wave signals as described in any one of claims 1-6.

Images