A detection method, device and computer storage medium

Abstract

The embodiment of the present application discloses a detection method, device and computer storage medium. The method includes: determining the current heart rate value of the object to be detected according to the video data of the object to be detected; based on the current heart rate value, obtaining a target area with a signal-to-noise ratio greater than a threshold value from several areas, and taking the timing signal corresponding to the target area as the target timing signal; wherein the several areas are obtained by dividing the video data of the object to be detected; extracting a sub-signal with the same bandwidth as the current heart rate value from the target timing signal as the heart rate signal, and determining the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different bandwidths. In this way, an area with a high signal-to-noise ratio is determined in several areas of the video data, and the timing signal of the area is screened again according to the heart rate value, thereby improving the accuracy of blood oxygen saturation detection.

Description

Technical Field

[0001] The present application relates to the technical field of blood oxygen saturation detection, and in particular to a detection method, device and computer storage medium. Background Art

[0002] Blood oxygen saturation (SaO2) is the percentage of oxygenated hemoglobin (HbO2) bound to oxygen in the blood to the total bindable hemoglobin (Hb) capacity, that is, the concentration of blood oxygen in the blood. It is an important physiological parameter of the respiratory cycle.

[0003] In the related technology, although there are already methods for detecting blood oxygen saturation, for the blood oxygen saturation detection method based on non-contact face video, the signal-to-noise ratio is too low, and it is difficult to accurately determine the blood oxygen saturation of the object to be detected; and for the detection method using a fingertip oximeter, it is based on a narrow-band wavelength and cannot be applied to the face reflection light signal with a low signal-to-noise ratio collected by an ordinary camera, resulting in low accuracy of the existing blood oxygen saturation detection. Summary of the invention

[0004] The present application provides a detection method, a device and a computer storage medium.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a detection method, which may include:

[0007] Determining a current heart rate value of the object to be detected based on the video data of the object to be detected;

[0008] Based on the current heart rate value, a target area with a signal-to-noise ratio greater than a threshold value is obtained from a plurality of areas, and a timing signal corresponding to the target area is used as a target timing signal; wherein the plurality of areas are obtained by dividing the video data of the object to be detected;

[0009] A sub-signal having the same bandwidth as the current heart rate value is extracted from the target timing signal as a heart rate signal, and the blood oxygen saturation corresponding to the object to be detected is determined according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different corresponding bandwidths.

[0010] In some embodiments, extracting a sub-signal having the same bandwidth as the current heart rate value from the target time series signal as the heart rate signal includes:

[0011] Performing variational mode decomposition processing on the target time series signal to obtain at least one sub-signal;

[0012] A sub-signal that satisfies a close condition to the bandwidth of the current heart rate value is extracted from the at least one sub-signal, and the extracted sub-signal is determined as the heart rate signal.

[0013] In some embodiments, determining the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal includes:

[0014] determining a strength of the heart rate signal;

[0015] Determining a blood oxygen saturation index of each of the plurality of regions according to the strength of the heart rate signal to obtain a plurality of blood oxygen saturation indexes;

[0016] Calculating the average value of the plurality of blood oxygen saturation indices to obtain a mean value of the blood oxygen saturation indices;

[0017] The blood oxygen saturation corresponding to the object to be detected is determined according to the mean value of the blood oxygen saturation index.

[0018] In some embodiments, determining the strength of the heart rate signal comprises:

[0019] The heart rate signal is used as a signal basis, and the red signal and the blue signal in the target time series signal are projected onto the signal basis to determine the intensity of the heart rate signal.

[0020] In some embodiments, based on the current heart rate value, obtaining a target area having a signal-to-noise ratio greater than a threshold value from a plurality of areas, and determining a timing signal corresponding to the target area as a target timing signal includes:

[0021] Determining a signal-to-noise ratio corresponding to each of the plurality of regions according to the current heart rate value;

[0022] Determine a target area where the signal-to-noise ratio is greater than the threshold value according to a comparison result between the signal-to-noise ratio corresponding to each area and the threshold value;

[0023] The target timing signal is determined according to the timing signal corresponding to the target area.

[0024] In some embodiments, the method further comprises:

[0025] According to the comparison result of the signal-to-noise ratio corresponding to each area and the threshold value, when the signal-to-noise ratio corresponding to each area is less than or equal to the threshold value, the object to be detected is re-collected and the step of determining the current heart rate value of the object to be detected is returned to execute.

[0026] In some embodiments, determining the current heart rate value of the subject to be detected based on the video data of the subject to be detected includes:

[0027] Determine, according to the video data of the object to be detected, a timing signal corresponding to a key area in the video data;

[0028] Performing Fourier transform processing on the time series signal to obtain a Fourier signal;

[0029] The frequency corresponding to the maximum amplitude value in the Fourier signal is determined as the current heart rate value.

[0030] In some embodiments, determining the timing signal corresponding to the key area in the video data according to the video data of the object to be detected includes:

[0031] Performing frame-by-frame detection on the video data of the object to be detected, determining the face position, and obtaining the average value of the three-channel signals corresponding to the face position;

[0032] The average values ​​of the three-channel signals are combined to form a timing signal corresponding to the key area.

[0033] In a second aspect, an embodiment of the present application provides an electronic device, including:

[0034] A memory for storing computer programs that can be run on the processor;

[0035] A processor, configured to execute the method as described in any one of the first aspects when running a computer program.

[0036] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by at least one processor, the method as described in any one of the first aspects is implemented.

[0037] A detection method, device and computer storage medium provided by the embodiments of the present application. According to the video data of the object to be detected, the current heart rate value of the object to be detected is determined; based on the current heart rate value, a target area with a signal-to-noise ratio greater than a threshold value is obtained from several areas, and the timing signal corresponding to the target area is used as the target timing signal; wherein the several areas are obtained by dividing the video data of the object to be detected; a sub-signal with the same bandwidth as the current heart rate value is extracted from the target timing signal as the heart rate signal, and the blood oxygen saturation corresponding to the object to be detected is determined according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different bandwidths. In this way, an area with a high signal-to-noise ratio is determined in several areas of the video data, and the timing signal of the area is screened again according to the heart rate value, which can not only realize the blood oxygen saturation detection of the object to be detected through video data, but also improve the accuracy of non-contact blood oxygen saturation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of a detection method provided in an embodiment of the present application;

[0039] Figure 2 A detailed flow chart of a detection method provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of an original RGB signal and its frequency domain waveform provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of a heart rate signal extracted based on variational mode decomposition and its waveform in the frequency domain provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of the structure of a detection device provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of a specific hardware structure of an electronic device provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0047] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second\third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0048] It can be understood that blood oxygen saturation (SaO2) is the percentage of oxygenated hemoglobin (HbO2) in the blood that is bound by oxygen to the total hemoglobin (Hb) capacity, that is, the concentration of blood oxygen in the blood, which is an important physiological parameter of respiratory circulation. For example, a certain infectious disease breaks out and spreads globally, and the threat of this infectious disease to human health and life continues to exist. Here, blood oxygen saturation, as an important screening indicator for this infectious disease, can detect hypoxemia when clinical hypoxia symptoms are not obvious, so that patients can receive timely treatment and treatment.

[0049] At present, there are studies on the detection of blood oxygen saturation through non-contact equipment. Among them, non-contact blood oxygen saturation detection based on face video has attracted widespread attention due to its economy and convenience. This measurement method only requires the built-in RGB camera of a mobile phone, tablet or computer to shoot a face video, and then detect the change of blood oxygen saturation based on the signal of the face. The warning of reduced blood oxygen saturation based on ordinary RGB cameras is convenient for residents to monitor changes in blood oxygen saturation at no cost, remind residents to pay attention to their physical condition, and facilitate timely medical treatment. At the same time, special industries, such as taxi drivers, bus drivers, teachers, etc., can reduce public safety risks by long-term non-contact blood oxygen saturation monitoring.

[0050] In the related technologies, the current research results all use the detection method of the traditional fingertip oximeter. The signal-to-noise ratio of the blood oxygen saturation detection based on non-contact face video is much lower than that of the traditional fingertip oximeter. In addition, the blood oxygen saturation detection principle used by the fingertip oximeter is based on a narrow-band wavelength, which is no longer suitable for the face reflection light signal with a low signal-to-noise ratio collected by ordinary RGB cameras. Therefore, the blood oxygen saturation detection based on face video is still a problem to be solved at this stage.

[0051] Based on this, the embodiment of the present application provides a detection method, the basic idea of ​​which is: according to the video data of the object to be detected, determine the current heart rate value of the object to be detected; based on the current heart rate value, obtain a target area with a signal-to-noise ratio greater than a threshold value from several areas, and use the timing signal corresponding to the target area as the target timing signal; wherein the several areas are obtained by dividing the video data of the object to be detected; extract a sub-signal with the same bandwidth as the current heart rate value from the target timing signal as the heart rate signal, and determine the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different bandwidths. In this way, the embodiment of the present application proposes a new blood oxygen saturation detection idea, determines an area with a high signal-to-noise ratio in several areas of the video data, and re-screens the timing signal of the area according to the heart rate value, which can not only realize the blood oxygen saturation detection of the object to be detected through video data, but also make the feasibility and accuracy of non-contact blood oxygen saturation detection higher.

[0052] In one embodiment of the present application, see Figure 1 , which shows a schematic flow chart of a detection method provided in an embodiment of the present application. Figure 1 As shown, the method may include:

[0053] S101: Determine the current heart rate value of the object to be detected according to the video data of the object to be detected.

[0054] It should be noted that the detection method provided in the embodiment of the present application can be applied to electronic devices that have the need to detect blood oxygen saturation. Here, the electronic device can be such as a computer, a smart phone, a tablet computer, a laptop computer, a personal digital assistant (PDA), a wearable device, a health monitoring device, etc., but it is not specifically limited.

[0055] It should also be noted that, in the embodiment of the present application, the video data of the object to be detected may be a face video. During the process of collecting the face video, sufficient light needs to be maintained and the object to be detected needs to remain relatively still and cannot shake significantly during the collection process to ensure the clarity of the face video. Moreover, when determining the current heart rate value of the object to be detected through the face video, it is also necessary to determine the key area in the face video, and then determine the current heart rate value of the object to be detected based on the timing signal of the key area.

[0056] In some embodiments, for S101, determining the current heart rate value of the object to be detected according to the video data of the object to be detected may include:

[0057] Determine, according to the video data of the object to be detected, a timing signal corresponding to a key area in the video data;

[0058] Performing Fourier transform processing on the time series signal to obtain a Fourier signal;

[0059] The frequency corresponding to the maximum amplitude value in the Fourier signal is determined as the current heart rate value.

[0060] It should be noted that when determining the key area of ​​the video data to be detected, face detection can be performed on each frame of the image to mark the face position. For example, eye detection technology (dlib) can be used to detect the face and frame the face position; then, the key area is determined based on the detected face. For example, the key area may include the forehead, cheeks, etc.

[0061] It should also be noted that Fourier transform is a very important algorithm in the field of digital signal processing. Any continuously measured time series or signal can be expressed as an infinite superposition of sine wave signals of different frequencies. By performing Fourier transform on the time series signal, the originally difficult-to-process time series signal can be converted into a frequency domain signal that is easy to analyze; then the frequency, amplitude and phase of different sine wave signals in the signal are calculated in an accumulative manner, and the frequency corresponding to the maximum amplitude is determined as the current heart rate value.

[0062] In some embodiments, determining the timing signal corresponding to the key area in the video data according to the video data of the object to be detected may include:

[0063] Performing frame-by-frame detection on the video data of the object to be detected, determining the face position, and obtaining the average value of the three-channel signals corresponding to the face position;

[0064] The average values ​​of the three-channel signals are combined to form a timing signal corresponding to the key area.

[0065] It should be noted that the average value of the three-channel signal corresponding to the face position is obtained, and the average value here specifically refers to the average value of each channel signal in different frames of the three-channel signal. After obtaining the average value of the three-channel signal, the average value of the three-channel signal can be combined to obtain the corresponding timing signal. Among them, the three-channel signal can include a red channel signal, a green channel signal and a blue channel signal, and the timing signal can include a red signal, a green signal and a blue signal.

[0066] In this way, by analyzing the face video of the object to be detected frame by frame, the key area is determined, and then the timing signal corresponding to the key area is determined, and finally the current heart rate value of the object to be detected is obtained according to the timing signal.

[0067] S102: Based on the current heart rate value, a target area with a signal-to-noise ratio greater than a threshold value is obtained from a plurality of areas, and a timing signal corresponding to the target area is used as a target timing signal; wherein the plurality of areas are obtained by dividing the video data of the object to be detected.

[0068] It should be noted that the division criteria of the several regions can be divided equally according to the face area, or can be divided according to different criteria according to the needs, and no limitation is made here. In addition, the setting of the threshold value requires that the signal-to-noise ratio of the target area reaches a certain standard. For example, the threshold value can be set to 0.6.

[0069] It should also be noted that factors affecting the signal-to-noise ratio may include the shooting light of the face video and whether the object to be detected remains still during the shooting process, etc., and no limitation is made here.

[0070] In some embodiments, based on the current heart rate value, acquiring a target area having a signal-to-noise ratio greater than a threshold value from a plurality of areas, and determining a timing signal corresponding to the target area as a target timing signal may include:

[0071] Determining a signal-to-noise ratio corresponding to each of the plurality of regions according to the current heart rate value;

[0072] Determine a target area where the signal-to-noise ratio is greater than the threshold value according to a comparison result between the signal-to-noise ratio corresponding to each area and the threshold value;

[0073] The target timing signal is determined according to the timing signal corresponding to the target area.

[0074] It should be noted that the current heart rate value is the current heart rate value of the object to be detected. The same current heart rate value is used when calculating the signal-to-noise ratio (SNR) of each area. i When Calculate, where △ It can be equal to 1 frequency domain resolution, so that the signal-to-noise ratio corresponding to each area is calculated in turn, and then the signal-to-noise ratio of each area is compared with the corresponding threshold value. Only the area with a signal-to-noise ratio higher than the threshold value can be used for blood oxygen saturation detection, where the threshold value can be set to 0.6.

[0075] It should also be noted that after comparing the signal-to-noise ratio of each area with the threshold value in several areas, it is necessary to select no less than 3 areas that meet the signal-to-noise ratio requirements, so as to ensure that a more accurate result is obtained in the subsequent blood oxygen saturation calculation.

[0076] Furthermore, in some embodiments, the method may further include:

[0077] According to the comparison result of the signal-to-noise ratio corresponding to each area and the threshold value, when the signal-to-noise ratio corresponding to each area is less than or equal to the threshold value, the object to be detected is re-collected and the step of determining the current heart rate value of the object to be detected is returned to execute.

[0078] It should be noted that if there are less than 3 areas that meet the signal-to-noise ratio requirements, it can be determined that the signal-to-noise ratio corresponding to each area is less than or equal to the threshold value, and then it is necessary to choose to re-collect the video data. In other words, when the number of areas where the signal-to-noise ratio is greater than the threshold value is less than 3, in order to ensure the accuracy of the blood oxygen saturation calculation results, the embodiment of the present application can choose to re-collect. In the process of re-collection, the cause of the low signal-to-noise ratio can be determined according to the situation of the video data and targeted improvements can be made. For example, if the light of the video data is too dark, the light of the shooting environment needs to be improved to meet the needs of video data collection. If the object to be detected in the video data is blurred, shaking, and has ghosting, it is necessary to keep the object to be detected relatively still during the next collection process to meet the video data collection requirements.

[0079] In this way, the signal-to-noise ratio of each of the several regions is determined according to the current heart rate value, and after comparing the signal-to-noise ratio of each region with the threshold value, the time series signal corresponding to the high signal-to-noise ratio region is selected as the target time series signal.

[0080] S103: extracting a sub-signal having the same bandwidth as the current heart rate value from the target time series signal as a heart rate signal, and determining the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal; wherein the target time series signal includes a plurality of sub-signals, and different sub-signals have different corresponding bandwidths.

[0081] It should be noted that when calculating the blood oxygen saturation, since the current blood oxygen saturation calculation method can only process signals with fixed frequencies, and since the noise of the target timing signal is large, it is difficult to obtain accurate blood oxygen saturation by directly using the target timing signal for calculation. Therefore, the sub-signal in the target timing signal with the same bandwidth as the heart rate value can be used as the heart rate signal; and since the frequency of the heart rate signal is relatively fixed, using the heart rate signal for blood oxygen saturation calculation can obtain more accurate blood oxygen saturation.

[0082] Specifically, in some embodiments, for S103, extracting a sub-signal having the same bandwidth as the current heart rate value from the target time series signal as the heart rate signal may include:

[0083] Performing variational mode decomposition processing on the target time series signal to obtain at least one sub-signal;

[0084] A sub-signal that satisfies a close condition to the bandwidth of the current heart rate value is extracted from the at least one sub-signal, and the extracted sub-signal is determined as the heart rate signal.

[0085] It should be noted that the target time series signal can be first subjected to variational mode decomposition to obtain at least one sub-signal. In the process of dividing the sub-signals, it can be divided into 5 to 8 sub-signals according to different bandwidths, and the sub-signal closest to the current heart rate value bandwidth is selected from these sub-signals as the heart rate signal, and then the blood oxygen saturation is calculated. It should also be noted that if the bandwidth of the two sub-signals is close to the current heart rate value, then the two sub-signals can also be determined to be heart rate signals, and the two heart rate signals are used to calculate the blood oxygen saturation, and finally the average of the two results is taken as the final blood oxygen saturation.

[0086] Specifically, in some embodiments, determining the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal may include:

[0087] determining a strength of the heart rate signal;

[0088] Determining a blood oxygen saturation index of each of the plurality of regions according to the strength of the heart rate signal to obtain a plurality of blood oxygen saturation indexes;

[0089] Calculating the average value of the plurality of blood oxygen saturation indices to obtain a mean value of the blood oxygen saturation indices;

[0090] The blood oxygen saturation corresponding to the object to be detected is determined according to the mean value of the blood oxygen saturation index.

[0091] It should be noted that the signal strengths of the red signal and the blue signal in the heart rate signal corresponding to the target area are first determined, and then the ratio of the red signal to the blue signal is used as the blood oxygen saturation index of the area. After the blood oxygen saturation index is calculated, the blood oxygen saturation index of all areas is the average value of the blood oxygen saturation indices of multiple areas. After the hypoxia saturation index is determined, the blood oxygen saturation of the object to be detected is determined based on the correspondence between the blood oxygen saturation index and the blood oxygen saturation.

[0092] It should also be noted that, in some embodiments, determining the strength of the heart rate signal may include:

[0093] The heart rate signal is used as a signal basis, and the red signal and the blue signal in the target time series signal are projected onto the signal basis to determine the intensity of the heart rate signal.

[0094] It should be noted that when calculating the strength of the heart rate signal, the signal with good signal quality in the target time series signal can be selected for calculation. For example, the red signal and the blue signal, or the red signal and the green signal can be selected, without any limitation. as well as Calculate the intensity of the red signal and the blue signal separately.

[0095] In this way, the heart rate signal is determined from the sub-signals divided from the time series signal, and the blood oxygen saturation is calculated using the heart rate signal, so that an accurate and high blood oxygen saturation result can be obtained.

[0096] The embodiment of the present application provides a detection method, which determines the current heart rate value of the object to be detected according to the video data of the object to be detected; based on the current heart rate value, obtains a target area with a signal-to-noise ratio greater than a threshold value from several areas, and uses the timing signal corresponding to the target area as the target timing signal; wherein the several areas are obtained by dividing the video data of the object to be detected; extracts a sub-signal with the same bandwidth as the current heart rate value from the target timing signal as the heart rate signal, and determines the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different bandwidths. In this way, an area with a high signal-to-noise ratio is determined in several areas of the video data, and the timing signal of the area is screened again according to the heart rate value, which can not only realize the blood oxygen saturation detection of the object to be detected through video data, but also make the feasibility and accuracy of non-contact blood oxygen saturation detection higher.

[0097] In another embodiment of the present application, based on the detection method described in the above embodiment, see Figure 2 , a detailed flow chart of a detection method provided in an embodiment of the present application. Figure 2 As shown, taking the face video as an example, the detailed process may include:

[0098] S201: Detecting the current heart rate value based on the face video to determine the current heart rate value;

[0099] S202: acquiring a target area with a high signal-to-noise ratio from a plurality of areas according to the current heart rate value, and using a timing signal corresponding to the target area as a target timing signal;

[0100] S203: extracting a sub-signal having the same bandwidth as the current heart rate value from the target time series signal as the heart rate signal; wherein the target time series signal includes a plurality of sub-signals, and different sub-signals correspond to different bandwidths;

[0101] S204: Perform blood oxygen saturation detection according to the heart rate signal to determine the blood oxygen saturation corresponding to the face video.

[0102] That is to say, in an embodiment of the present application, the current heart rate is first detected based on the face video. According to the detected current heart rate value, the high signal-to-noise ratio area of ​​the face (i.e., the target area where the signal-to-noise ratio is greater than the threshold value) is adaptively selected. Then, blood oxygen saturation detection is performed based on the time series composed of the high signal-to-noise ratio target time series signal. First, the target time series signal is subjected to variational modal decomposition, and the time series signal is decomposed into multiple sub-signals with narrow bandwidth, and the heart rate signal is selected according to the current heart rate value. Then, the real-time heart rate signal constructed from multiple areas of the face is used as the signal basis, and the red, green and blue channels are projected onto the signal basis respectively, and the DC component is taken to obtain the intensity of the heart rate signal in the current red, green and blue channels, thereby calculating the current blood oxygen saturation.

[0103] In a specific embodiment, the detection method may include the following steps:

[0104] Step 1: Detect the current heart rate value based on the video data of the object to be detected.

[0105] 1) Perform face detection on each frame of the video data and mark the face position. You can use eye detection technology (dlib) to detect the face and frame the face position.

[0106] Based on the detected face, key area detection is performed to obtain the average values ​​of the red, green and blue three-channel signals of key areas such as the forehead and cheeks.

[0107] 2) Combine the average values ​​of the red, green and blue three-channel signals of multiple frames to form a timing signal and According to the timing signal and Perform heart rate detection to determine the current heart rate value. Specifically, the following steps may be included:

[0108] First, we use formulas (1)-(3) to and Combine them to obtain the noise reduction signal Z:

[0109]

[0110]

[0111]

[0112] Among them, δ(·) represents the standard deviation.

[0113] Then, the noise reduction signal Z is Fourier transformed using formula (4) to obtain the heart rate signal

[0114]

[0115] Since the heart rate range is generally between [40,200], the heart rate signal The signal is in the range of [0.67Hz, 3.33Hz], determine the frequency hr corresponding to the highest peak of the frequency domain amplitude average is the current heart rate value.

[0116] Step 2: Based on the current heart rate value, adaptively obtain the high signal-to-noise ratio area of ​​the face (i.e., the target area in the aforementioned embodiment), and obtain the red, green, and blue three-channel timing signals of the high signal-to-noise ratio area as the target timing signals.

[0117] After obtaining the current heart rate value, first, the face can be divided into i small blocks, and the average value of the red, green and blue signals (RGB) of each block is combined into a time series signal and Obtain the noise reduction signal (i.e., the timing signal in the aforementioned embodiment) Z of each region i .

[0118] Then, the noise reduction signal Z of each region is obtained according to formulas (5)-(6): i Signal-to-noise ratio SNR i

[0119]

[0120]

[0121] in, △ Can be equal to 1 frequency domain resolution.

[0122] When the signal-to-noise ratio SNR i When it is greater than a threshold value (exemplarily, the threshold value may be 0.6), the region may be used to calculate the blood oxygen saturation, and j regions capable of calculating the blood oxygen saturation are determined as target regions.

[0123] Step 3: Perform blood oxygen saturation detection based on the red, green and blue three-channel timing signals of the target area.

[0124] Obtained j time series signals with high signal-to-noise ratio areas of the face and Finally, it can also include:

[0125] 1) Extracting time series signals based on the heart rate signal extraction technology of variational model decomposition and The sub-signal with a bandwidth close to the current heart rate value is taken as the heart rate signal.

[0126] Specifically, since blood oxygen saturation detection needs to detect the heart rate signal strength contained in the red, green and blue three-channel signals to characterize the blood oxygen saturation, and the noise content in the blood oxygen saturation signal based on face video is much higher than the heart rate signal content, the original signal is not used directly to calculate the blood oxygen saturation.

[0127] First, the variational mode decomposition (VMD) is used to The signal is decomposed into multiple sub-signals with different bandwidths, and then based on the current heart rate value hr average Select heart rate signal and

[0128] See also Figure 3 , which shows an original RGB signal and its frequency domain schematic diagram proposed in an embodiment of the present application, the schematic diagram includes the original RGB signal and the corresponding heart rate signal, such as Figure 3 As shown, the original timing signal and The time domain signal is messy and has no obvious periodic pulsation. In the frequency domain (the absolute amplitude is not marked, but it does not affect the effect), due to the influence of noise such as movement and light changes, the heart rate value hr average The amplitude at is also very low (indicating that the heart rate signal content in the current time series signal is very low). At the same time, even if there are peaks around the heart rate value, its amplitude cannot be trusted (it is most likely the amplitude caused by noise, not the heart rate signal itself).

[0129] See also Figure 4 , which shows a schematic diagram of a heart rate signal extracted based on variational mode decomposition and its frequency domain proposed in an embodiment of the present application. Figure 4 As shown in the figure, the heart rate signal extracted based on variational mode decomposition and After that. In the time domain, the periodic pulsation is obvious, and in the frequency domain, there is an obvious peak near the heart rate value (indicating that the heart rate signal is obvious). At the same time, it can be seen that there is a deviation between the horizontal coordinate corresponding to the highest peak in the frequency domain and the heart rate value. Therefore, it is inaccurate to directly use Fast Fourier Transform (FFT) to solve the amplitude of the heart rate signal. The method of "calculating blood oxygen saturation based on real-time heart rate signal" in this embodiment can perfectly avoid this situation.

[0130] 2) Blood oxygen saturation calculation based on real-time heart rate signal

[0131] After obtaining the current heart rate signal, it is necessary to calculate the amplitude of the current heart rate signal. The traditional algorithm directly uses Fourier transform, which can only decompose fixed frequencies. Therefore, the Fourier transform method is not suitable for extracting the amplitude of the heart rate signal.

[0132] The embodiment of the present application uses the heart rate signal corresponding to the current heart rate value constructed by multiple areas of the face as the signal basis, projects the three channels of red, green and blue into the coordinate system respectively, and takes the DC component to obtain the current heart rate peak and valley values.

[0133] Specifically, the signal basis can be constructed according to formula (7).

[0134]

[0135] in:

[0136] After the signal base is obtained, the heart rate signal strength in the current red signal and blue signal can be determined according to formulas (8)-(9).

[0137]

[0138]

[0139] After the heart rate signal strength is determined, the blood oxygen saturation index corresponding to region j can be calculated according to formula (10).

[0140]

[0141] After the blood oxygen saturation index is calculated, the blood oxygen saturation index R of all regions is the average value of multiple regions. Specifically, it can be calculated according to formula (11).

[0142]

[0143] Furthermore, generally, the blood oxygen saturation is directly proportional to the blood oxygen saturation index R, and the blood oxygen saturation can be calculated by formula (12).

[0144] SpO2=A·R+B (12)

[0145] Among them, A and B are fixed parameters. If you want to obtain accurate blood oxygen saturation SpO2, you need to collect multiple sets of blood oxygen data and calibrate them to obtain A and B.

[0146] This embodiment provides a detection method, and the specific implementation of the aforementioned embodiment is described in detail based on the aforementioned embodiment. It can be seen that according to the technical solution of the aforementioned embodiment, a heart rate signal extraction technology based on variational mode decomposition extracts a heart rate signal with a high signal-to-noise ratio, and a blood oxygen saturation calculation method based on the real-time heart rate signal as a signal base. In this way, an area with a high signal-to-noise ratio is determined in several areas of the video data, and the time series signal of the area is screened again according to the heart rate value, which can not only realize the blood oxygen saturation detection of the object to be detected through the video data, but also make the feasibility and accuracy of non-contact blood oxygen saturation detection higher.

[0147] In another embodiment of the present application, see Figure 5 , which shows a schematic diagram of the composition structure of a detection device provided in an embodiment of the present application. Figure 5 As shown, the detection device 50 may include:

[0148] The determination unit 501 is configured to determine the current heart rate value of the object to be detected according to the video data of the object to be detected;

[0149] The acquisition unit 502 is configured to acquire a target area whose signal-to-noise ratio is greater than a threshold value from a plurality of areas based on the current heart rate value, and use a timing signal corresponding to the target area as a target timing signal; wherein the plurality of areas are obtained by dividing the video data of the object to be detected;

[0150] The detection unit 503 is configured to extract a sub-signal with the same bandwidth as the current heart rate value from the target timing signal as a heart rate signal, and determine the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different corresponding bandwidths.

[0151] In some embodiments, the detection unit 503 is specifically configured to perform variational mode decomposition processing on the target time series signal to obtain at least one sub-signal; and extract a sub-signal that satisfies a proximity condition with the bandwidth of the current heart rate value from the at least one sub-signal, and determine the extracted sub-signal as the heart rate signal.

[0152] In some embodiments, the detection unit 503 is specifically configured to determine the strength of the heart rate signal; and determine the blood oxygen saturation index of each of the several areas according to the strength of the heart rate signal to obtain several blood oxygen saturation indices; and calculate the average value of the several blood oxygen saturation indices to obtain a mean value of the blood oxygen saturation index; and determine the blood oxygen saturation corresponding to the object to be detected according to the mean value of the blood oxygen saturation index.

[0153] In some embodiments, the detection unit 503 is specifically configured to use the heart rate signal as a signal basis, project the red signal and the blue signal in the target timing signal onto the signal basis, and determine the intensity of the heart rate signal.

[0154] In some embodiments, the acquisition unit 502 is specifically configured to determine the signal-to-noise ratio corresponding to each of the several areas according to the current heart rate value; and determine the target area where the signal-to-noise ratio is greater than the threshold value according to the comparison result of the signal-to-noise ratio corresponding to each area and the threshold value; and determine the target timing signal according to the timing signal corresponding to the target area.

[0155] In some embodiments, the acquisition unit 502 is further configured to re-collect the object to be detected based on the comparison result of the signal-to-noise ratio corresponding to each area with the threshold value, and return to execute the step of determining the current heart rate value of the object to be detected when the signal-to-noise ratio corresponding to each area is less than or equal to the threshold value.

[0156] In some embodiments, the determination unit 501 is specifically configured to determine, based on the video data of the object to be detected, a timing signal corresponding to a key area in the video data; and perform Fourier transform processing on the timing signal to obtain a Fourier signal; and determine the frequency corresponding to the maximum amplitude value in the Fourier signal as the current heart rate value.

[0157] In some embodiments, the determination unit 501 is specifically configured to perform frame-by-frame detection on the video data of the object to be detected, determine the face position, and obtain the average value of the three-channel signal corresponding to the face position; and combine the average values ​​of the three-channel signals to form a timing signal corresponding to the key area.

[0158] It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.

[0159] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0160] Therefore, this embodiment provides a computer storage medium, which stores a computer program. When the computer program is executed by at least one processor, the steps of the method described in any one of the above embodiments are implemented.

[0161] Based on the composition of the above detection device 50 and the computer storage medium, see Figure 6 , which shows a specific hardware structure diagram of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 60 may include: a communication interface 601, a memory 602 and a processor 603; each component is coupled together via a bus system 604. It is understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus system 604. Among them, the communication interface 601 is used for receiving and sending signals in the process of sending and receiving information between other external network elements;

[0162] A memory 602, used to store a computer program that can be run on the processor 603;

[0163] The processor 603 is configured to execute, when running the computer program:

[0164] Determining a current heart rate value of the object to be detected based on the video data of the object to be detected;

[0165] Based on the current heart rate value, a target area with a signal-to-noise ratio greater than a threshold value is obtained from a plurality of areas, and a timing signal corresponding to the target area is used as a target timing signal; wherein the plurality of areas are obtained by dividing the video data of the object to be detected;

[0166] A sub-signal having the same bandwidth as the current heart rate value is extracted from the target timing signal as a heart rate signal, and the blood oxygen saturation corresponding to the object to be detected is determined according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different corresponding bandwidths.

[0167] It can be understood that the memory 602 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The memory 602 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0168] The processor 603 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 603. The above processor 603 may be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor are combined and executed. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 603 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0169] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0170] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0171] Optionally, as another embodiment, the processor 603 is further configured to execute the steps of the method described in any one of the aforementioned embodiments when running the computer program.

[0172] In some embodiments, see Figure 7 , which shows a schematic diagram of the composition structure of an electronic device 60 provided in an embodiment of the present application. Figure 7 As shown, the electronic device 60 at least includes the detection device 50 described in any one of the aforementioned embodiments.

[0173] In an embodiment of the present application, for the electronic device 60, the current heart rate value of the object to be detected is determined according to the video data of the object to be detected; based on the current heart rate value, a target area with a signal-to-noise ratio greater than a threshold value is obtained from several areas, and the timing signal corresponding to the target area is used as a target timing signal; wherein the several areas are obtained by dividing the video data of the object to be detected; a sub-signal with the same bandwidth as the current heart rate value is extracted from the target timing signal as a heart rate signal, and the blood oxygen saturation corresponding to the object to be detected is determined according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different bandwidths. In this way, an area with a high signal-to-noise ratio is determined in several areas of the video data, and the timing signal of the area is screened again according to the heart rate value, which can not only realize the blood oxygen saturation detection of the object to be detected through video data, but also improve the accuracy of non-contact blood oxygen saturation detection.

[0174] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0175] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0176] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0177] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0178] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0179] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A detection method, comprising: Determining a current heart rate value of the object to be detected based on the video data of the object to be detected; Based on the current heart rate value, a target area with a signal-to-noise ratio greater than a threshold value is obtained from a plurality of areas, and a timing signal corresponding to the target area is used as a target timing signal; wherein the plurality of areas are obtained by dividing the video data of the object to be detected; Performing variational mode decomposition processing on the target time series signal to obtain at least one sub-signal; A sub-signal that satisfies a close condition to the bandwidth of the current heart rate value is extracted from the at least one sub-signal, the extracted sub-signal is determined as a heart rate signal, and the blood oxygen saturation corresponding to the object to be detected is determined according to the heart rate signal; wherein the target timing signal includes several sub-signals, and different sub-signals have different corresponding bandwidths.

2. The method according to claim 1, wherein determining the blood oxygen saturation corresponding to the object to be detected according to the heart rate signal comprises: determining the strength of the heart rate signal; Determining a blood oxygen saturation index of each of the plurality of regions according to the strength of the heart rate signal to obtain a plurality of blood oxygen saturation indexes; Calculating the average value of the plurality of blood oxygen saturation indices to obtain a mean value of the blood oxygen saturation indices; The blood oxygen saturation corresponding to the object to be detected is determined according to the mean value of the blood oxygen saturation index.

3. The method according to claim 2, wherein determining the strength of the heart rate signal comprises: The heart rate signal is used as a signal basis, and the red signal and the blue signal in the target time series signal are projected onto the signal basis to determine the intensity of the heart rate signal.

4. The method according to claim 1, wherein based on the current heart rate value, a target area having a signal-to-noise ratio greater than a threshold value is obtained from a plurality of areas, and a timing signal corresponding to the target area is determined as a target timing signal, comprising: Determining a signal-to-noise ratio corresponding to each of the plurality of regions according to the current heart rate value; Determine a target area where the signal-to-noise ratio is greater than the threshold value according to a comparison result between the signal-to-noise ratio corresponding to each area and the threshold value; The target timing signal is determined according to the timing signal corresponding to the target area.

5. The method according to claim 4, further comprising: According to the comparison result of the signal-to-noise ratio corresponding to each area and the threshold value, when the signal-to-noise ratio corresponding to each area is less than or equal to the threshold value, the object to be detected is re-collected and the step of determining the current heart rate value of the object to be detected is returned to execute.

6. The method according to claim 1, wherein determining the current heart rate value of the subject to be detected based on the video data of the subject to be detected comprises: Determine, according to the video data of the object to be detected, a timing signal corresponding to a key area in the video data; Performing Fourier transform processing on the time series signal to obtain a Fourier signal; The frequency corresponding to the maximum amplitude value in the Fourier signal is determined as the current heart rate value.

7. The method according to claim 6, wherein determining the timing signal corresponding to the key area in the video data according to the video data of the object to be detected comprises: Performing frame-by-frame detection on the video data of the object to be detected, determining the face position, and obtaining the average value of the three-channel signals corresponding to the face position; The average values ​​of the three-channel signals are combined to form a timing signal corresponding to the key area.

8. An electronic device comprising: A memory for storing computer programs that can be run on the processor; A processor, configured to execute the method according to any one of claims 1 to 7 when running the computer program.

9. A computer storage medium storing a computer program, wherein the computer program, when executed by at least one processor, implements the method according to any one of claims 1 to 7.

Images