Color space based video ppg measurement method

Abstract

The application relates to a color space-based video PPG measurement method, wherein the method comprises the following steps: acquiring a face video captured by an RGB camera; processing the face video, extracting a face region from the face video, and determining a region of interest; based on the region of interest, adopting a proxy variable of illumination intensity obtained from a B channel pixel value in the RGB to perform weighted processing on pixels in the region of interest, calculating a PPG value, and acquiring a PPG waveform. According to the embodiment of the application, the proxy variable of illumination intensity obtained from the B channel pixel value in the RGB can be adopted to perform weighted processing on the pixels in the region of interest, calculate the PPG value, and acquire the PPG waveform according to the region of interest, so that the interference of motion on the video PPG measurement accuracy is greatly reduced, the practicability of the video PPG is improved, the measurement accuracy of the video PPG under a complex motion scene is improved, and support is provided for reliable measurement of non-contact physiological signals.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a video PPG measurement method based on a color space. BACKGROUND

[0002] In recent years, non-contact physiological signal measurement technology has attracted widespread attention. Compared with the contact measurement method, the non-contact measurement method can avoid direct contact with the user's skin / body, reduce or even eliminate the interference and discomfort caused to the user during the measurement process. With the development and application of camera, laser, radar and other technologies, non-contact physiological signal measurement technology is expected to play a major role in daily life or medical fields, such as at the Tokyo Olympics, the International Archery Federation developed a biological signal recognition system and placed some high-speed cameras on the site to realize the public broadcast of the heart rate changes of the players during the archery competition.

[0003] In related technologies, the video PPG measurement method includes a traditional algorithm and a neural network algorithm, and a new color space can be designed to obtain more accurate PPG changes and / or a neural network can be trained based on a specific ROI.

[0004] However, in related technologies, the light reflected by each part of the face and the PPG information are basically consistent, and this assumption is difficult to hold in light or more complex motion, which may cause unnecessary errors in actual application. When the neural network algorithm is used to extract the heart rate on the ROI, considering the influence of motion deformation, occlusion and other problems, the stability of the results of the neural network algorithm is greatly affected by the composition of the scene and the training set, lacks interpretability, and it is difficult to output effective pulse waveform and calculate physiological indicators such as heart rate variability, it is difficult to reduce the interference of motion on the accuracy of video PPG measurement, and it is difficult to effectively improve the practicability of video PPG, which needs to be improved. SUMMARY

[0005] The present application provides a video PPG measurement method based on a color space to solve the problems in related technologies that when the neural network algorithm is used to extract the heart rate on the ROI, considering the influence of motion deformation, occlusion and other problems, the stability of the results of the neural network algorithm is greatly affected by the composition of the scene and the training set, lacks interpretability, and it is difficult to output effective pulse waveform and calculate physiological indicators such as heart rate variability, it is difficult to reduce the interference of motion on the accuracy of video PPG measurement, and it is difficult to effectively improve the practicability of video PPG.

[0006] The first aspect of this application provides a video PPG measurement method based on color space, comprising the following steps: acquiring a face video captured by an RGB camera; processing the face video to extract a face region from the face video and determine a region of interest; and based on the region of interest, using the B channel pixel value in RGB to obtain a proxy variable of illumination intensity to weight the pixels within the ROI, calculate the PPG value, and obtain the PPG waveform.

[0007] Optionally, in one embodiment of this application, the step of extracting the face region from the face video includes: detecting changes in illumination in the current scene; when the changes in illumination meet preset conditions, extracting the face region based on a pre-built skin color calculation model; and when the changes in illumination do not meet the preset conditions, calculating and extracting the face region based on each frame of the face video.

[0008] Optionally, in one embodiment of this application, the preset condition is that the illumination change value obtained from the change information is less than a preset threshold.

[0009] Optionally, in one embodiment of this application, the method of obtaining the proxy variable of light intensity using the B channel pixel value in RGB includes: obtaining the proxy variable of light intensity using the B channel pixel value in RGB based on a preset RGB relationship.

[0010] Optionally, in one embodiment of this application, the preset RGB relationship is:

[0011] B(t) = b × I(t),

[0012] R(t)≈R(t)+k r R×P(t)=r / b×B(t)×(1+k r ×P(t)),

[0013] G(t)≈G(t)+k g G×P(t)=g / b×B(t)×(1+k g ×P(t)),

[0014] Where R(t), G(t), and B(t) represent the luminance values ​​of the red, green, and blue channels of a pixel or region at time t, respectively; r, g, and b represent the illumination intensity (or total luminance); I(t) is the projection coefficient of the corresponding channel; P(t) represents the pulse signal under the skin; and k... r k g , respectively, are the reflection coefficients of the red and green channels to the PPG signal, and b represents the magnitude of the blue component in the light source. Here, it is assumed that the magnitude of the color components in the light source remains constant and the reflection coefficient of the blue channel is equal to 0.

[0015] A second aspect of this application provides a video PPG measurement device based on color space, comprising: an acquisition module for acquiring a face video captured by an RGB camera; an extraction module for processing the face video to extract a face region from the face video and determine a region of interest; and a processing module for using a proxy variable of illumination intensity obtained from the B channel pixel values ​​in RGB based on the region of interest, to perform weighted processing on the pixels within the ROI, calculate the PPG value, and acquire the PPG waveform.

[0016] Optionally, in one embodiment of this application, the extraction module includes: a detection unit for detecting changes in illumination in the current scene; an extraction unit for extracting the face region based on a pre-built skin color calculation model when the changes in illumination meet preset conditions; and a calculation unit for calculating and extracting the face region based on each frame of the face video when the changes in illumination do not meet the preset conditions.

[0017] Optionally, in one embodiment of this application, the preset condition is that the illumination change value obtained from the change information is less than a preset threshold.

[0018] Optionally, in one embodiment of this application, the processing module includes: an acquisition unit, used to obtain a proxy variable of light intensity based on a preset RGB relationship and using the pixel value of the B channel in RGB.

[0019] Optionally, in one embodiment of this application, the preset RGB relationship is:

[0020] B(t) = b × I(t),

[0021] R(t)≈R(t)+k r R×P(t)=r / b×B(t)×(1+k r ×P(t)),

[0022] G(t)≈G(t)+k g G×P(t)=g / b×B(t)×(1+k g ×P(t)),

[0023] Where R(t), G(t), and B(t) represent the luminance values ​​of the red, green, and blue channels of a pixel or region at time t, respectively; r, g, and b represent the illumination intensity (or total luminance); I(t) is the projection coefficient of the corresponding channel; P(t) represents the pulse signal under the skin; and k... r k g , respectively, are the reflection coefficients of the red and green channels to the PPG signal, and b represents the magnitude of the blue component in the light source. Here, it is assumed that the magnitude of the color components in the light source remains constant and the reflection coefficient of the blue channel is equal to 0.

[0024] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the color space-based video PPG measurement method as described in the above embodiments.

[0025] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the color space-based video PPG measurement method described above.

[0026] This application embodiment can use the B channel pixel value in the RGB region of interest to obtain a proxy variable for illumination intensity, which is then used to weight the pixels within the ROI to calculate the PPG value and obtain the PPG waveform. This significantly reduces the interference of motion on the accuracy of video PPG measurement, improves the practicality of video PPG, and provides support for reliable measurement of non-contact physiological signals. Therefore, it solves the problems in related technologies where, when using neural network algorithms to extract heart rate from the ROI, considering the effects of motion deformation and occlusion, the stability of the results of such algorithms is greatly affected by the scene and the composition of the training set, lacks interpretability, and is difficult to output effective pulse waveforms for calculating physiological indicators such as heart rate variability. Furthermore, it is difficult to reduce the interference of motion on the accuracy of video PPG measurement, thus failing to effectively improve the practicality of video PPG.

[0027] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0028] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0029] Figure 1 This is a flowchart of a video PPG measurement method based on color space according to an embodiment of this application;

[0030] Figure 2 This is a flowchart of a color space-based video PPG measurement method according to an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of a video PPG measurement device based on a color space according to an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0034] The following describes a color space-based video PPG measurement method according to embodiments of this application with reference to the accompanying drawings. Addressing the issues mentioned in the background section regarding the extraction of heart rate from a Region of Interest (ROI) using neural network algorithms, the stability of these algorithms is greatly affected by the scene and training set composition, lacking interpretability and struggling to output effective pulse waveforms for calculating physiological indicators such as heart rate variability. Furthermore, they fail to reduce the interference of motion on the accuracy of video PPG measurements, thus hindering the practicality of video PPG measurement. This application provides a color space-based video PPG measurement method. In this method, the illumination intensity is obtained as a proxy variable based on the B channel pixel values ​​in the RGB spectrum, according to the region of interest. This proxy variable is then used to weight the pixels within the ROI, calculate the PPG value, and obtain the PPG waveform. This significantly reduces the interference of motion on the accuracy of video PPG measurements, improves the practicality of video PPG, enhances the measurement accuracy of video PPG in complex motion scenes, and provides support for reliable non-contact physiological signal measurement. This solves the problems in related technologies, such as the instability of neural network algorithms when extracting heart rate on ROI, the lack of interpretability, the difficulty in outputting effective pulse waveforms for calculating physiological indicators such as heart rate variability, the difficulty in reducing the interference of motion on the accuracy of video PPG measurement, and the inability to effectively improve the practicality of video PPG.

[0035] Specifically, Figure 1 This is a schematic flowchart of a video PPG measurement method based on color space provided in an embodiment of this application.

[0036] like Figure 1 As shown, this color space-based video PPG measurement method includes the following steps:

[0037] In step S101, a face video captured by an RGB camera is obtained.

[0038] In a specific embodiment, an RGB (RGB color mode) camera can be used to capture facial videos. The camera can be used to capture video images of the subject's face, providing a basis for extracting facial video information and accurately determining the region of interest. This further improves the performance of the video PPG (Photoplethysmogram) algorithm and enhances its practicality.

[0039] In step S102, the face video is processed to extract the face region and determine the region of interest.

[0040] In practical implementation, the embodiments of this application can process face videos and use face detection algorithms to extract face regions as ROIs (Regions of Interest). The face detection algorithm can perform face detection on face videos, extract face regions from face videos, obtain face feature points, perform face recognition on face videos, and generate recognition results. When the position of the face in the face video changes, the changing position of the face feature points can be tracked in real time to achieve more accurate face feature point localization, thereby accurately determining the region of interest, reducing the interference of motion on the accuracy of video PPG measurement, and improving the practicality of video PPG.

[0041] Optionally, in one embodiment of this application, extracting a face region from a face video includes: detecting changes in illumination in the current scene; extracting a face region based on a pre-built skin color calculation model when the changes meet preset conditions; and calculating and extracting a face region based on each frame of the face video when the changes do not meet preset conditions.

[0042] In a specific embodiment, the illumination change information of the current scene can be detected. When the illumination change is less than a certain value and the scene is relatively stable, it indicates that the illumination change information meets certain conditions. At this time, the skin color model can be pre-calculated for subsequent skin extraction (high efficiency). The extraction of the facial skin region can be adjusted according to the scene illumination change. When the illumination change is greater than a certain value and the scene is relatively unstable, it indicates that the illumination change information does not meet certain conditions. At this time, the skin region can be calculated and extracted separately for each frame (low efficiency). This reduces the interference of motion on the accuracy of video PPG measurement, improves the practicality of video PPG, provides support for the reliable measurement of non-contact physiological signals, and further improves the performance of the video PPG algorithm.

[0043] It should be noted that the preset conditions can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0044] In one embodiment of this application, the preset condition is that the illumination change value obtained from the change information is less than a preset threshold.

[0045] In some embodiments, comparing the illumination change value obtained from the illumination change information with a certain threshold is beneficial for extracting the face region from the face video in different ways. When the illumination change value obtained from the illumination change information is less than a certain threshold, the illumination is in a relatively stable scene. At this time, the skin color model can be pre-calculated for subsequent skin extraction. When the illumination change value obtained from the illumination change information is greater than a certain threshold, the illumination is in a less stable scene. At this time, the skin region can be calculated and extracted separately for each frame, further reducing the interference of motion on the accuracy of video PPG measurement and improving the practicality of video PPG.

[0046] In step S103, based on the region of interest, a proxy variable for illumination intensity is obtained using the B channel pixel value in RGB, which is used to weight the pixels within the ROI, calculate the PPG value, and obtain the PPG waveform.

[0047] It is understood that in the embodiments of this application, ROI is the part of an image or data that can be used for a specific purpose, PPG is the curve of the pulsating pulse recorded by photoplethysmography, and RGB represents the colors of the three channels: red, green, and blue.

[0048] In a specific embodiment, due to wavelength issues, even with high-quality (good lighting, still person, uncompressed video) face videos, the B channel in the RGB spectrum basically does not contain heart rate signals. Therefore, based on the region of interest, the B channel pixel values ​​can be used as proxy variables for illumination intensity to weight the pixels within the ROI, calculate the PPG value, and preprocess and calculate the corresponding PPG feature parameters based on the acquired PPG signal to obtain the PPG waveform. This further improves the measurement accuracy of video PPG in complex motion scenarios and reduces misjudgments and missed judgments in daily monitoring (such as disease prevention).

[0049] Optionally, in one embodiment of this application, the proxy variable for light intensity is obtained by using the B channel pixel value in RGB, including: obtaining the proxy variable for light intensity by using the B channel pixel value in RGB based on a preset RGB relationship.

[0050] In some cases, embodiments of this application can use the pixel value of the B channel in RGB to obtain the proxy variable of light intensity according to the preset RGB relationship. The pixel brightness value of the B channel can be obtained by calculating the formula B(t)=b×I(t), establishing a B channel brightness weighted model, and thereby constructing a complete video PPG measurement scheme to further improve the performance of the video PPG algorithm.

[0051] It should be noted that the preset RGB relationship can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0052] Optionally, in one embodiment of this application, the preset RGB relationship is:

[0053] B(t) = b × I(t),

[0054] R(t)≈R(t)+k r R×p(t)=r / b×B(t)×(1+k r ×P(t)),

[0055] G(t)≈G(t)+k g G×P(t)=g / b×B(t)×(1+k g ×P(t)),

[0056] Where R(t), G(t), and B(t) represent the luminance values ​​of the red, green, and blue channels of a pixel or region at time t, respectively; r, g, and b represent the illumination intensity (or total luminance); I(t) is the projection coefficient of the corresponding channel; P(t) represents the pulse signal under the skin; and k... r k g , respectively, are the reflection coefficients of the red and green channels to the PPG signal, and b represents the magnitude of the blue component in the light source. Here, it is assumed that the magnitude of the color components in the light source remains constant and the reflection coefficient of the blue channel is equal to 0.

[0057] In some cases, embodiments of this application can calculate certain RGB relationships using formulas. By obtaining the brightness values ​​of each RGB channel within the ROI region of a face video, the differences and connections between RGB channels in the video PPG are explored, a B-channel brightness weighted model is established, and a complete video PPG measurement scheme is constructed accordingly. In this scheme, the extraction of the facial skin region can be adjusted according to changes in scene lighting. The core of the algorithm is to establish weight relationships between different pixels, which can be combined with existing algorithms to improve the performance of the video PPG algorithm.

[0058] Specifically, in combination Figure 2 As shown, the working principle of the color space-based video PPG measurement method of this application is explained in detail with an embodiment.

[0059] In step S201, an RGB camera is used to capture facial video. This embodiment of the application can use an RGB camera to capture facial video, utilizing the camera to capture the subject's face and obtain video images of the face, providing a basis for extracting facial video information and accurately determining the region of interest.

[0060] In step S202, a face region is extracted as a Region of Interest (ROI) using a face detection algorithm. This embodiment of the application can process face videos, extracting face regions as ROIs using a face detection algorithm. The face detection algorithm can perform face detection on face videos, extract face regions from the face videos, obtain face feature points, perform face recognition on the face videos, and generate recognition results, thereby determining the region of interest.

[0061] In step S203, the B channel pixel value in RGB is used as a proxy variable for illumination intensity, and the pixels within the ROI are weighted accordingly. In this embodiment, due to wavelength issues, even with high-quality (good lighting, still person, uncompressed video) face videos, the B channel in RGB contains virtually no heart rate signal. Therefore, based on the region of interest, the B channel pixel value can be used as a proxy variable for illumination intensity to weight the pixels within the ROI.

[0062] In step S204, the PPG value is calculated and the PPG waveform is obtained. This embodiment of the application can calculate the PPG value, preprocess and calculate the corresponding PPG feature parameters based on the acquired PPG signal, and obtain the PPG waveform, thereby further improving the measurement accuracy of video PPG in complex motion scenes.

[0063] The video PPG measurement method based on color space proposed in this application can use the pixel value of the B channel in RGB to obtain a proxy variable of light intensity based on the region of interest (ROI). This proxy variable is then used to weight the pixels within the ROI to calculate the PPG value and obtain the PPG waveform. This significantly reduces the interference of motion on the accuracy of video PPG measurement, improves the practicality of video PPG, and enhances the measurement accuracy of video PPG in complex motion scenes, providing support for reliable measurement of non-contact physiological signals. This solves the problem in related technologies where, when using neural network algorithms to extract heart rate from the ROI, the stability of the results is greatly affected by the scene and the composition of the training set, lacking interpretability and making it difficult to output effective pulse waveforms for calculating physiological indicators such as heart rate variability. Furthermore, it is difficult to reduce the interference of motion on the accuracy of video PPG measurement and effectively improve the practicality of video PPG.

[0064] Next, referring to the accompanying drawings, a video PPG measurement device based on color space proposed according to an embodiment of this application is described.

[0065] Figure 3 This is a block diagram of a color space-based video PPG measurement device according to an embodiment of this application.

[0066] like Figure 3As shown, the color space-based video PPG measurement device 10 includes: an acquisition module 100, an extraction module, and a processing module 300.

[0067] Specifically, the acquisition module 100 is used to acquire face videos captured by an RGB camera;

[0068] Extraction module 200 is used to process face videos, extract face regions from the face videos, and determine regions of interest; and

[0069] The processing module 300 is used to obtain a proxy variable of illumination intensity based on the pixel value of the B channel in RGB based on the region of interest, to perform weighted processing on the pixels within the ROI, calculate the PPG value, and obtain the PPG waveform.

[0070] Optionally, in one embodiment of this application, the extraction module 200 includes: a detection unit, an extraction unit, and a calculation unit.

[0071] The detection unit is used to detect changes in the lighting in the current scene.

[0072] The extraction unit is used to extract the face region based on a pre-built calculation skin color model when the change information meets the preset conditions.

[0073] The calculation unit is used to calculate and extract the face region based on each frame of the face video when the changing information does not meet the preset conditions.

[0074] Optionally, in one embodiment of this application, the preset condition is that the illumination change value obtained from the change information is less than a preset threshold.

[0075] Optionally, in one embodiment of this application, the processing module 300 includes: an acquisition unit.

[0076] The acquisition unit is used to obtain the proxy variable of light intensity based on the B channel pixel value in RGB based on the preset RGB relationship.

[0077] Optionally, in one embodiment of this application, the preset RGB relationship is:

[0078] B(t) = b × I(t),

[0079] R(t)≈R(t)+k r R×P(t)=r / b×B(t)×(1+k r ×P(t)),

[0080] G(t)≈G(t)+k g G×P(t)=g / b×B(t)×(1+k g ×P(t)),

[0081] Where R(t), G(t), and B(t) represent the luminance values ​​of the red, green, and blue channels of a pixel or region at time t, respectively; r, g, and b represent the illumination intensity (or total luminance); I(t) is the projection coefficient of the corresponding channel; P(t) represents the pulse signal under the skin; and k... r k g , respectively, are the reflection coefficients of the red and green channels to the PPG signal, and b represents the magnitude of the blue component in the light source. Here, it is assumed that the magnitude of the color components in the light source remains constant and the reflection coefficient of the blue channel is equal to 0.

[0082] It should be noted that the foregoing explanation of the embodiment of the video PPG measurement method based on color space also applies to the video PPG measurement device based on color space in this embodiment, and will not be repeated here.

[0083] The video PPG measurement device based on color space proposed in this application can use the pixel value of the B channel in RGB to obtain a proxy variable of light intensity according to the region of interest (ROI), and then perform weighted processing on the pixels within the ROI to calculate the PPG value and obtain the PPG waveform. This significantly reduces the interference of motion on the accuracy of video PPG measurement, improves the practicality of video PPG, enhances the measurement accuracy of video PPG in complex motion scenes, and provides support for reliable measurement of non-contact physiological signals. Therefore, it solves the problem in related technologies where, when using neural network algorithms to extract heart rate from the ROI, considering the influence of motion deformation, occlusion, and other issues, the stability of the results of such algorithms is greatly affected by the scene and the composition of the training set, lacks interpretability, and is difficult to output effective pulse waveforms for calculating physiological indicators such as heart rate variability. Furthermore, it is difficult to reduce the interference of motion on the accuracy of video PPG measurement and cannot effectively improve the practicality of video PPG.

[0084] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0085] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0086] When the processor 402 executes the program, it implements the color space-based video PPG measurement method provided in the above embodiments.

[0087] Furthermore, electronic devices also include:

[0088] Communication interface 403 is used for communication between memory 401 and processor 402.

[0089] The memory 401 is used to store computer programs that can run on the processor 402.

[0090] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0091] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0092] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0093] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0094] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the color space-based video PPG measurement method described above.

[0095] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0096] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0097] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0098] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0099] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0100] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0102] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A video PPG measurement method based on color space, characterized in that, Includes the following steps: Acquire face videos captured by an RGB camera; The face video is processed to extract the face region and determine the region of interest (ROI). as well as Based on the region of interest (ROI), the surrogate variable of illumination intensity is obtained by using the B channel pixel value in RGB, and then the pixels within the ROI are weighted to calculate the PPG value and obtain the PPG waveform. The proxy variable for obtaining light intensity using the B channel pixel value in RGB includes: Based on the preset RGB relationship, the surrogate variable for light intensity is obtained by using the B channel pixel value in RGB; The preset RGB relationship is as follows: ( ) = × ( )， ( )≈ / × ( )×(1+ × ( ))， G( )≈ / × ( )×(1+ g × ( ))， in, ( ), G( ), ( The values ​​of r, g, and b represent the brightness values ​​of the red, green, and blue channels of a pixel or region at time t, respectively, while r, g, and b represent the illumination intensity. ( The projection coefficients in the corresponding channels. ( This indicates the pulse signal under the skin. , These are the reflection coefficients of the PPG signal for the red and green channels, respectively. This represents the magnitude of the blue component in the light source. Here, it is assumed that the magnitude of the color components in the light source remains constant and the reflectance of the blue channel is equal to 0.

2. The method according to claim 1, characterized in that, Extracting the face region from the face video includes: Detect changes in lighting conditions in the current scene; When the change information meets the preset conditions, the face region is extracted based on a pre-constructed calculation skin color model; When the change information does not meet the preset conditions, the face region is calculated and extracted based on each frame of the face video.

3. The method according to claim 2, characterized in that, The preset condition is that the illumination change value obtained from the change information is less than a preset threshold.

4. A video PPG measurement device based on color space, characterized in that, include: The acquisition module is used to acquire face videos captured by an RGB camera; The extraction module is used to process the face video, extract the face region from the face video, and determine the region of interest (ROI). as well as The processing module is used to obtain a proxy variable of illumination intensity based on the region of interest (ROI) and the pixel value of the B channel in RGB, so as to perform weighted processing on the pixels in the ROI, calculate the PPG value, and obtain the PPG waveform. The proxy variable for obtaining light intensity using the B channel pixel value in RGB includes: Based on the preset RGB relationship, the surrogate variable for light intensity is obtained by using the B channel pixel value in RGB; The preset RGB relationship is as follows: ( ) = × ( )， ( )≈ / × ( )×(1+ × ( ))， G( )≈ / × ( )×(1+ g × ( ))， in, ( ), G( ), ( The values ​​of r, g, and b represent the brightness values ​​of the red, green, and blue channels of a pixel or region at time t, respectively, while r, g, and b represent the illumination intensity. ( The projection coefficients in the corresponding channels. ( This indicates the pulse signal under the skin. , These are the reflection coefficients of the PPG signal for the red and green channels, respectively. This represents the magnitude of the blue component in the light source. Here, it is assumed that the magnitude of the color components in the light source remains constant and the reflectance of the blue channel is equal to 0.

5. The apparatus according to claim 4, characterized in that, The extraction module includes: The detection unit is used to detect changes in the lighting conditions of the current scene. An extraction unit is used to extract the face region based on a pre-built calculation skin color model when the change information meets preset conditions. The calculation unit is used to calculate and extract the face region based on each frame of the face video when the change information does not meet the preset conditions.

6. The apparatus according to claim 5, characterized in that, The preset condition is that the illumination change value obtained from the change information is less than a preset threshold.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the color space-based video PPG measurement method as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the color space-based video PPG measurement method as described in any one of claims 1-3.

Images