Identification and Mitigation of Flicker Mimicking PPG Signals

Abstract

Disclosed herein is a method comprising the steps of acquiring a first series of images of a subject using an optical imaging system at a first frame rate, determining first clusters of pixels in the first series of images, determining that a flicker signal is present in one or more of the first clusters of pixels, and performing combined weighting of the pixels of the first clusters to determine a first photoplethysmographic signal, wherein the weighting is performed to reduce or eliminate the contribution of the flicker signal, and acquiring a second series of images at a second frame rate, determining second clusters of pixels in the second series of images, and combining the pixels of the second clusters to determine a second photoplethysmographic signal to prevent flicker signals, and deriving vital sign information of the subject from the first photoplethysmographic signal and / or the second photoplethysmographic signal.

Description

[Technical Field]

[0001] The present invention relates to camera-based photoplethysmography systems, and in particular to identifying and mitigating flicker that mimics photoplethysmography (PPG) signals. [Background technology]

[0002] Remote photoplethysmography (rPPG) is an optical measurement technique for detecting minute blood volume fluctuations in the cutaneous microcirculation using a digital camera. It allows the non-contact measurement of various physiological parameters such as pulse rate (PR) and its variability, blood pressure, and pulse transit time. Summary of the Invention [Problem to be solved by the invention]

[0003] Remote photoplethysmography can be used to measure pulse rate and current cardiac phase in real time based on a video stream of a human subject's face. This technology has a wide range of potential applications. However, rPPG measurements can be affected by the environment in which they are performed. European Patent Application Publication No. EP3207862A1 discloses a method for video-based monitoring of vital signs. [Means for solving the problem]

[0004] The technical features of the present invention are provided in the accompanying claims. The present invention provides a system, a computer program and a method in the independent claims. Embodiments are set out in the dependent claims.

[0005] Remote detection of cardiac signals by camera-based photoplethysmography (PPG) has a wide range of applications beyond the medical world. Modern ambient lighting, such as LEDs, typically applies high-frequency modulation, which can alias into the cardiac frequency band. The resulting flicker interferes with PPG detection. Embodiments allow for the identification of an aliased spectral component by a characteristic shift induced by a slight change in the camera frame rate. Embodiments are disclosed for shifting it out of the cardiac band and localizing the modulated light source.

[0006] In one aspect, the present invention provides a system having a memory storing machine-executable instructions and a processor, wherein execution of the machine-executable instructions causes the processor to acquire a first series of images of an object using an optical imaging system, the first series of images being acquired at a first frame rate; determining first clusters of pixels in the first series of images; and determining that a flicker signal is present in one or more of the first clusters of pixels; and a) performing a combined weighting of pixel values ​​of the first clusters to generate a first photoplethysmogram. a) performing at least one of the steps of: a) determining a photoplethysmographic signal at a second frame rate to prevent flicker signals, wherein the weighting is performed to reduce or eliminate the contribution of flicker signals; and b) acquiring a second series of images at a second frame rate to prevent flicker signals, determining a second cluster of pixels in the second series of images, and combining pixel values ​​of the second clusters to determine a second photoplethysmographic signal; and deriving vital sign information of the subject from the first photoplethysmographic signal and / or the second photoplethysmographic signal.

[0007] The system may be, for example, a medical system.

[0008] The first and second series of images can be acquired simultaneously. Simultaneous acquisition can be performed using multiple cameras. Alternatively, the first and second series of images can be acquired sequentially.

[0009] For example, performing at least one of a) and b) is performed in response to determining that a flicker signal is present in one or more of the first clusters of pixels.

[0010] The optical imaging system may be any imaging device capable of acquiring a series of images, preferably one or more digital video cameras that output digital video frames with timestamps annotating each frame. The images acquired by the camera may be video frames. The optical imaging system may be remote from the subject. This may allow, for example, to remotely measure vital sign information without the need to attach a sensor to the subject. For example, the optical imaging system may be configured to acquire a first series of images from an imaging zone of the subject. The imaging zone may be, for example, the subject's face. The first series of images may form a video. The first series of images comprises a sequence of images acquired at successive, evenly spaced time points. The first series of images may be acquired at a first frame rate f. The frame rate refers to the number of frames per second. The optical imaging system may be configured to store and transmit the first series of images.

[0011] For example, a light source or illumination source can be provided to illuminate the imaging zone of the subject. For example, the illumination can be infrared light or any visible light. The use of a light source can be beneficial because it can provide better operation of the optical imaging system and more accurate determination of vital sign information. The light source can be, for example, a light emitting diode (LED) or an array of LEDs, an incandescent lamp, a halogen lamp, etc. The light source may or may not be part of the optical imaging system.

[0012] A repetitive change, or modulation, in the magnitude of the light flux of the light source over time can occur. This modulation occurs at a frequency f that is higher than the first frame rate f. L However, this may cause a flicker signal in the first series of images. For example, an optical imaging system may capture video frames many times per second (e.g., f=n / s), and the light from the light source may flicker many times per second (e.g., f L =n L / s). Thus, optical imaging systems may capture images when the light is not fully illuminated. The image may flicker.

[0013] The present subject matter can advantageously process the first series of images using first clusters to prevent flicker. In one example, each first cluster of the first clusters can represent a separate region of interest (ROI) of the imaging zone. Using different ROIs allows for covering different features of the imaged zone, thus obtaining an accurate prediction of vital sign information. Alternatively, the first cluster can represent one ROI. In this case, the first cluster can be randomly selected within the image. Each first cluster of the first clusters can have a respective set of pixels within each image of the first series of images.

[0014] The present subject matter may determine a first photoplethysmography signal using a first cluster, and / or determine a second photoplethysmography signal using a second cluster. The first photoplethysmography signal may be obtained by performing a weighted sum of pixel values ​​of the first cluster. The weighted sum may be performed to reduce or completely eliminate flicker detected in one or more first clusters. The second photoplethysmography signal may be obtained by using a new cluster obtained using a second frame rate f''. The first photoplethysmography signal and the second photoplethysmography signal may be remote photoplethysmography signals.

[0015] If only the first photoplethysmography signal is determined by the medical system, vital sign information can be derived from the first photoplethysmography signal. If only the second photoplethysmography signal is determined by the medical system, vital sign information can be derived from the second photoplethysmography signal.

[0016] If both the first photoplethysmography signal and the second photoplethysmography signal are determined by the medical system, vital sign information can be derived from the first photoplethysmography signal and / or the second photoplethysmography signal.

[0017] For example, vital sign information can be determined using the first photoplethysmography signal and the second photoplethysmography signal by combining (e.g., averaging) the first photoplethysmography signal and the second photoplethysmography signal and deriving the vital sign information using the combined photoplethysmography signal. Alternatively, the first vital sign information can be derived from the first photoplethysmography signal and the second vital sign information can be derived from the second photoplethysmography signal, and the first and second vital sign information can be combined (e.g., averaged) to obtain the vital sign information.

[0018] Thus, the present subject matter can provide accurate vital sign information. The derived vital sign information can be advantageously used for different applications. For example, the vital sign information can be used to monitor the health of patients in a waiting room. In another example, the vital sign information can be used to monitor the health of automobile drivers. In another example, the vital sign information can be used to acquire cardiac magnetic resonance imaging (MRI) data.

[0019] According to one embodiment, acquiring the second series of images is performed to shift the flicker signal out of the cardiac frequency band. According to one embodiment, execution of the machine-executable instructions by the processor includes applying a bandpass filter that passes frequencies within the cardiac frequency band and rejects frequencies outside the cardiac frequency band to acquire a filtered second photoplethysmographic signal, and deriving vital sign information from the filtered signal.

[0020] According to one embodiment, the second frame rate f" is obtained by shifting the first frame rate by an epsilon shift ε. For example, the first frame rate can be shifted as follows: f" = f ± ε, where ε ≦ 0.02 f.

[0021] An example implementation of this embodiment can consist of repeatedly shifting the frame rate with different values ​​of epsilon shift until obtaining a flicker signal outside the cardiac frequency band. For example, if an LED driver were to drive the light source at f L = 1002 Hz and the first frame rate is f = 20 Hz, the alias frequency is A = 2 Hz = 120 beats per minute (bpm), which is within the cardiac frequency band. If the first frame rate is changed to f' = 20.01 Hz, the alias frequency will be f A If the frame rate is further changed to f''=19.9 Hz, the alias frequency becomes f Af'" = 7 Hz = 420 bpm, which is outside the cardiac frequency band. This can be particularly advantageous because any true PPG signal can remain at the same frequency after successive acquisitions, so that the true PPG signal and the flicker component can be distinguished. In some cases, the heart rate can change between two successive acquisitions. In this case, the change in frequency rate from one acquisition to another can be performed so that the expected shift in flicker is much larger than any change in heart rate that may be expected between the two acquisitions. Alternatively, two cameras can be used to simultaneously acquire a first series of images and a second series of images at two different frame rates, for example, f and f', or f' and f".

[0022] According to one embodiment, the flicker signal is caused by a modulation of the luminous flux of the light source, the modulation having a modulation frequency f defined as a function of the first frame rate f as follows: L Having: TIFF2026507438000002.tif869Here, the frequency of the flicker signal is f A =|β|f, where the second frame rate f″ is defined as: TIFF2026507438000003.tif987 where f' is the mean frame rate and f A ' is the intermediate frame rate f A ' is the frequency of the flicker signal obtained, and f A where f is the desired frequency of the flicker signal associated with the second frame rate f. This embodiment can enable an accurate and systematic method for deriving the second frame rate.

[0023] According to one embodiment, the combined weighting is performed using weights, and the combined weighting is performed to ignore one or more first clusters using null weights or to downweight one or more first clusters using weights that have lower values ​​compared to the other weights.

[0024] According to one embodiment, combining the pixel values ​​of the second clusters includes, for each image in the second series of images, determining an individual photoplethysmography signal for each second cluster using the pixel values ​​of the second clusters and combining the individual photoplethysmography signals.

[0025] For example, the determined second clusters are N2 second clusters cl1 2 ...cl N2 2 where N2≧2. The photoplethysmographic signal value for a given image in the second series of images may be obtained, for example, as follows: TIFF2026507438000004.tif812 where Av k is the cluster cl in a given image. k 2 is the pixel value or the average of the individual photoplethysmography signal values.

[0026] According to one embodiment, the combined weighting of the pixel values ​​of the first clusters includes, for each image in the first series of images, determining an individual photoplethysmography signal for each first cluster using the pixel values ​​of the first clusters, and performing combined weighting of the individual photoplethysmography signals using the respective weights.

[0027] For example, the determined first clusters are N1 first clusters cl1 1 ...cl N1 1 where N1≧2. 1 ...cl N1 1 are 0≦w j 1 Weight w1 < 1 1 ...w N1 1Each first cluster cl of the one or more first clusters having a flicker signal can be associated with j 1 is the null weight w to remove the flicker signal. j 1 = 0, where j is an index pointing to one or more first clusters. Alternatively, each first cluster cl of one or more first clusters having a flicker signal can be assigned j 1 is used to reduce the contribution of the flicker signal to the first photoplethysmography signal by filtering out other first clusters cl i 1 Weight w is smaller than the weight of j 1 <w i 1 where i is an index that refers to the cluster that does not have a flicker signal. The photoplethysmography signal value for a given image in the first series of images can be obtained, for example, as follows: TIFF2026507438000005.tif1016 where Av k is the cluster cl in a given image. k 1 is the pixel value or the average of the individual photoplethysmography signal values.

[0028] According to one embodiment, the first cluster is different from or the same as the second cluster. Having the same cluster for the first and second acquisitions can allow for consistent combination of the first and second photoplethysmographic signals to obtain vital sign information. Having different clusters for the first and second acquisitions can cover different features of the acquired images and therefore provide an accurate estimation of the vital sign information.

[0029] According to one embodiment, each of the first clusters has the same set of pixels in each image of the first series of images. This embodiment may be advantageous because clustering can save processing resources compared to adjustable / dynamic clustering. This embodiment may be particularly advantageous when the imaging zone is not moving during acquisition of the first series of images.

[0030] According to one embodiment, each cluster of the first clusters has a different set of pixels in two or more images of the first series of images. This clustering may be advantageous when the imaged zone may move during acquisition of the first series of images. The movement of the imaged zone may be tracked across the series of images to adapt the clusters for each image. For example, if the imaged zone is a subject's face, a face tracker may be used to track the movement of the subject's face and readjust the clusters from image to image. An advantage of this embodiment is that it provides improved vital sign information even when the subject is moving during acquisition of the first series of images.

[0031] According to one embodiment, the optical imaging system is configured to image the skin surface of the subject. This embodiment may be beneficial as the optical imaging system directly images the surface of the subject.

[0032] According to one embodiment, the optical imaging system includes one or more video cameras, where the optical imaging system includes multiple cameras, these cameras may acquire video data from the same imaging zone and may operate at the same frame rate or slightly different frame rates to acquire video data simultaneously.

[0033] For example, the first series of images may be divided into subsets of images, each subset acquired by a respective camera. An individual first photoplethysmography signal may be acquired for each subset of images. The individual first photoplethysmography signals may be combined (e.g., averaged) to obtain a first photoplethysmography signal. This may be particularly advantageous when the same frame rate is used by multiple cameras.

[0034] The second series of images can be divided into subsets of images. An individual second photoplethysmography signal can be acquired for each subset. The individual second photoplethysmography signals can be combined to acquire a second photoplethysmography signal. This can be particularly advantageous when the same frame rate is used by multiple cameras. Alternatively, the cameras can acquire the first series of images and the second series of images using slightly different frame rates, for example, a first frame rate and a second frame rate. This can be advantageous because the derived heart rate does not change between the two acquisitions, but the position of the flicker signal does.

[0035] According to one embodiment, the medical system further includes a magnetic resonance imaging system, and the memory further includes pulse sequence commands configured to control the magnetic resonance imaging system to acquire k-space data according to a magnetic resonance imaging protocol. Execution of the machine-executable instructions causes the computing system to acquire k-space data by controlling the magnetic resonance imaging system according to the determined vital sign information. This embodiment may be advantageous because it can enable accurate acquisition of MR images. For example, this can enable acquisition of high-quality cardiac MRI data by synchronizing image acquisition with specific times during the subject's cardiac cycle. The vital sign information can be used, for example, to detect the largest and steepest peaks in the cardiac cycle to trigger acquisition of portions of k-space.

[0036] According to one embodiment, the vital signs information includes at least one of heart rate, respiratory rate, blood pressure, pulse transit time, and real-time triggers.

[0037] In another aspect, the invention relates to a computer program having machine-executable instructions for execution by a computing system, the execution of the machine-executable instructions causing the computing system to acquire a first series of images of an object using an optical imaging system, the first series of images being acquired at a first frame rate; determining first clusters of pixels in the first series of images; determining that a flicker signal is present in one or more of the first clusters of pixels; and performing a combined weighting of the pixel values ​​of the first clusters to generate a first photoprecipitating signal. performing at least one of the steps of: determining a photoplethysmography signal, wherein the weighting is performed to reduce or eliminate the contribution of flicker signals; acquiring a second series of images at a second frame rate to prevent flicker signals, determining a second cluster of pixels in the second series of images, and combining the pixel values ​​of the second cluster to determine a second photoplethysmography signal; and deriving vital sign information of the subject from the first photoplethysmography signal and / or the second photoplethysmography signal.

[0038] In another aspect, the present invention relates to a method comprising performing at least one of the steps of acquiring a first series of images of a subject using an optical imaging system, the first series of images being acquired at a first frame rate; determining first clusters of pixels in the first series of images; determining that a flicker signal is present in one or more of the first clusters of pixels; and performing a combined weighting of pixel values ​​of the first clusters to determine a first photoplethysmographic signal, the combined weighting being performed to reduce or eliminate the contribution of the flicker signal; and acquiring a second series of images at a second frame rate, determining a second cluster of pixels in the second series of images, and combining pixel values ​​of the second clusters to determine a second photoplethysmographic signal to prevent flicker signals; and deriving vital sign information of the subject from the first photoplethysmographic signal and / or the second photoplethysmographic signal.

[0039] It should be understood that one or more of the above-described embodiments of the present invention may be combined, provided that the combined embodiments are not mutually exclusive. As will be appreciated by those skilled in the art, aspects of the present invention may be embodied as an apparatus, a method, or a computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware form, an entirely software form (including firmware, resident software, microcode, etc.), or a combination of software and hardware forms, all of which may be referred to herein as a "circuit," "module," or "system." Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-executable code embodied thereon.

[0040] Any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. As used herein, a "computer-readable storage medium" encompasses any tangible storage medium capable of storing instructions executable by a processor or computing system of a computing device. The computer-readable storage medium may also be referred to as a computer-readable non-transitory storage medium. The computer-readable storage medium may also be referred to as a tangible computer-readable medium. In some embodiments, the computer-readable storage medium may also store data accessible by the computing system of a computing device. Examples of computer-readable storage media include, but are not limited to, floppy disks, magnetic hard disk drives, solid-state hard disks, flash memory, USB thumb drives, random access memory, read-only memory (ROM), optical disks, magneto-optical disks, and computing system register files. Examples of optical disks include compact discs (CDs) and digital versatile discs (DVDs), such as CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R discs. The term computer-readable storage medium also refers to various types of storage media that can be accessed by a computer device over a network or communications link. For example, data can be retrieved via a modem, over the Internet, or over a local area network. Computer-executable code embodied on a computer-readable medium can be transmitted using any suitable medium, including, but not limited to, wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

[0041] A computer-readable signal medium may include a propagated data signal having computer-executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium, and such computer-readable medium is not a computer-readable storage medium, but is capable of communicating, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0042] "Computer memory" or "memory" is one example of a computer-readable storage medium. Computer memory is any memory directly accessible to a computing system. "Computer storage" or "storage" is another example of a computer-readable storage medium. Computer storage is any non-volatile computer-readable storage medium. In some embodiments, computer storage may be computer memory, or vice versa.

[0043] As used herein, a "computing system" includes electronic components capable of executing programs, machine-executable instructions, or computer-executable code. References to a computing system, including examples of a "computing system," should be interpreted as including two or more computing systems or processing cores, as the case may be. A computing system may be, for example, a multi-core processor. A computing system can also refer to a collection of computing systems within a single computer system or distributed among multiple computer systems. The term computing system should also be interpreted as referring to a collection or network of computing devices, possibly each having a processor or computing system. Machine-executable code or instructions can be executed by multiple computing systems or processors, which may be within the same computing device or distributed across multiple computing devices.

[0044] Machine-executable instructions or computer-executable code may comprise instructions or programs that cause a processor or other computing system to perform aspects of the present invention. Computer-executable code for performing operations for aspects of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages, compiled into machine-executable instructions. In some examples, the computer-executable code may be in the form of a high-level language or pre-compiled, or may be used in conjunction with an interpreter that generates machine-executable instructions on the fly. In other cases, the machine-executable instructions or computer-executable code may be in the form of a program for a programmable logic gate array.

[0045] The computer executable code may run entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter situation, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0046] Aspects of the present invention will be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block or portion of a block in the flowcharts, diagrams, and / or block diagrams can be implemented by computer program instructions in the form of computer-executable code, where applicable. It will also be understood that blocks in different flowcharts, diagrams, and / or block diagrams can be combined, if not mutually exclusive. These computer program instructions can be provided to a general-purpose computer, special-purpose computer, or other programmable data processing device computing system to generate machine-implemented means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0047] These machine-executable instructions or computer program instructions may be stored on a computer-readable medium that can instruct a computer, other programmable data processing apparatus, or other device to function in a particular manner, such that the instructions stored on the computer-readable medium create an article of manufacture including instructions that implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0048] The machine-executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps executed on the computer, other programmable apparatus, or other device to be executed on the computer, creating a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus provide a process for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0049] As used herein, a "user interface" is an interface that allows a user or operator to interact with a computer or computer system. A "user interface" is also referred to as a "human interface device," and a user interface can provide information or data to an operator and / or receive information or data from an operator. A user interface can allow a computer to receive input from an operator and provide output from the computer to a user. In other words, a user interface can allow an operator to control or manipulate a computer, and an interface can allow a computer to show the effects of the operator's control or manipulation. The display of data or information on a display or graphical user interface is an example of providing information to an operator. Receiving data via a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, gamepad, webcam, headset, pedals, wired gloves, remote control, and accelerometer are all examples of user interface components that allow receiving information or data from an operator.

[0050] As used herein, a "hardware interface" includes an interface that allows a computing system of a computer system to interact with and / or control external computing devices and / or equipment. A hardware interface may allow a computing system to send control signals or instructions to external computing devices and / or equipment. A hardware interface may also allow a computing system to exchange data with external computing devices and / or equipment. Examples of hardware interfaces include, but are not limited to, a universal serial bus, an IEEE 1394 port, a parallel port, an IEEE 1284 port, a serial port, an RS-232 port, an IEEE-488 port, a Bluetooth connection, a wireless local area network connection, a TCP / IP connection, an Ethernet connection, a control voltage interface, a MIDI interface, an analog input interface, and a digital input interface.

[0051] As used herein, a "display" or "display device" encompasses an output device or user interface adapted to display images or data. A display can output visual, auditory, and / or tactile data. Examples of displays include, but are not limited to, computer monitors, television screens, touch screens, tactile electronic displays, Braille screens, cathode ray tubes (CRTs), memory tubes, bi-stable displays, electronic paper, vector displays, flat panel displays, vacuum fluorescent displays (VFs), light-emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light-emitting diode displays (OLEDs), projectors, and head-mounted displays.

[0052] K-space data is defined herein as the recorded measurements of radio frequency signals emitted by atomic spins using the antenna of a magnetic resonance imaging system during a magnetic resonance computed tomography (CT) scan. It is an example of tomographic medical imaging data.

[0053] A magnetic resonance image or MR image is defined herein as a reconstructed two-dimensional or three-dimensional visualization of the anatomical data contained within the k-space data, which visualization can be performed, for example, by a computer. [Brief explanation of the drawings]

[0054] [Figure 1] FIG. 1 is a diagram illustrating an example of a medical system. [Figure 2] 2 is a flowchart of a method of using the medical system of FIG. 1 . [Figure 3] 2 is a flowchart of a method of using the medical system of FIG. 1 . [Figure 4A] 2 is a flowchart of a method of using the medical system of FIG. 1 . [Figure 4B] 1 shows a set of images I1...IM and several clusters cl1, cl2,...,clN for a method of using the medical system of FIG. 1. [Figure 5] FIG. 1 is a diagram illustrating an example of a medical system. [Figure 6] 6 is a flowchart of a method of using the medical system of FIG. 5 . [Figure 7a] FIG. 1 shows a plot representing a flicker signal. [Figure 7b] FIG. 1 shows a plot representing a flicker signal. DETAILED DESCRIPTION OF THE INVENTION

[0055] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the drawings in which:

[0056] Like numbered components in these figures are either equivalent components or perform the same function. An aforementioned component is not necessarily described in a subsequent figure if the functionality is equivalent.

[0057] 1 shows an example of a medical system 100. A subject 104 is seated on a seat. A camera 101 is focused on an imaging zone 105 of the subject 104. The camera 101 is connected to a hardware interface 116 of a computer system 102. The camera 101 may be configured to acquire video data from the imaging zone 105, store the video data, and transmit the video data to the computer system 102. The video data may be transmitted, for example, as a stream. For example, the computer system 102 may control the acquisition of the video data by the camera 101. A light source 107 is provided to illuminate the subject 104, and in particular the imaging zone 105.

[0058] The computer system 102 includes a processor 114. The computer system 102 is intended to represent one or more computers or computer systems. The processor 104 is intended to represent one or more computing systems or computing cores. The computer system 102 is further shown as including a hardware interface 116 connected to the processor 114. If other components of the medical system 100, such as a magnetic resonance imaging system, are present or included, the hardware interface 116 can be used to exchange data and commands with these other components. The computer system 102 is further shown as including an optional user interface 108, which can provide various means for relaying data and receiving data and commands from an operator.

[0059] Computer system 102 is further shown as having memory 110 coupled to processor 114. Memory 110 is intended to represent various types of memory accessible by processor 114. Memory 110 is shown as including machine-executable instructions 120. Machine-executable instructions 120 are instructions that enable processor 114 to perform various control, data processing, and image processing tasks. Memory 110 is further shown as including vital signs information 122 obtained from video 124 acquired by camera 101.

[0060] The medical system 100 can be used to derive an rPPG signal. However, use of the rPPG signal can be hindered by limited PPG signal amplitude and signal disturbances. In particular, flicker due to modulated ambient lighting provided by the light source 107 can interfere with PPG signal detection or mimic the PPG signal. In fact, the light source 107 can be modulated at a high frequency that is not directly perceptible by the human eye. If the camera 105 acquires raw PPG data at a fixed frame rate less than the modulation frequency, interference between the light modulation and the frame rate can result in aliasing into the expected range of cardiac frequencies, approximately 0.5 Hz to 3 Hz. Furthermore, applying a bandpass filter in the cardiac frequency band may not be sufficient to remove the flicker contribution. Therefore, the present subject matter can use the medical system 100 to perform a method, such as the method of FIG. 2, to prevent flicker in the rPPG signal.

[0061] 2 is a flowchart of a method for determining vital sign information of a subject, according to an example of the present subject matter. The method can be performed, for example, by the medical system of FIG.

[0062] In step 201, a first series of images of an object may be acquired from an imaging zone of the object using an optical imaging system such as camera 101. The first series of images is acquired at a first frame rate.

[0063] In step 203, a first cluster of pixels can be determined within a first series of images.

[0064] In step 205, it can be determined whether a flicker signal is present in one or more of the first clusters of pixels. Figure 7a shows an example of a flicker signal.

[0065] In response to determining that a flicker signal is present in one or more of the first clusters of pixels, at least one of step 207 and step 209 may be performed.

[0066] In step 207, a combined weighting of the pixel values ​​of the first cluster can be performed to determine a first photoplethysmographic signal, said weighting being performed to reduce or remove the contribution of the flicker signal.

[0067] Step 209 includes steps 209A to 209C.

[0068] In step 209A, a second series of images may be acquired from an imaging zone of the subject at a second frame rate to prevent flicker signals. In step 209B, a second cluster of pixels in the second series of images may be determined. In step 209C, pixel values ​​of the second cluster may be combined to determine a second photoplethysmographic signal.

[0069] In step 211, vital sign information of the subject can be derived from the first photoplethysmography signal and / or the second photoplethysmography signal.

[0070] In one example, steps 201-211 can be repeated periodically, for example at intervals determined by a typical time scale over which ambient lighting changes, for example every few minutes, or in response to changes in ambient lighting levels, or in response to receiving a request for vital sign information.

[0071] In one exemplary alternative implementation of the method of Figure 2, acquisition step 209A can be performed simultaneously with acquisition step 201, for example using multiple cameras. In this case, step 209 can be composed of sub-steps 209B-209C.

[0072] 3 is a flowchart of a method for determining a frame rate f″ according to an example of the present subject matter. The method can be performed, for example, by the medical system of FIG. 1. The method of FIG. 3 can be used, for example, to perform acquisition step 209A of FIG. 2.

[0073] For example, the light source 107 emits light at a frequency f L and a first series of images can be recorded by the camera 101 at a frame rate f. In step 301, the modulation frequency f L can be defined as a function of the camera frame rate f as follows: TIFF2026507438000006.tif765

[0074] In step 303, the alias frequency or flicker frequency f of the light source in the video stream is calculated. A is the formula f A For example, if the LED driver has a modulation frequency f L If you modulate the light source at f = 1002 Hz and the frame rate is f = 20 Hz, the alias frequency is f A= 2 Hz = 120 bpm, which is within the cardiac frequency band. This can be resolved by shifting the aliasing frequency out of the cardiac frequency band using a new frame rate f''. However, the modulation frequency of the light source, f L Since is not known, we can estimate the appropriate aliasing frequency f based on measurements using a single frame rate f. A It can be difficult to calculate a frame rate f that brings f outside the cardiac frequency band.

[0075] To do so, in step 305, the frame rate can be slightly changed from f to f', and the aliasing frequency f A and f A Each pair of ' is derived by identifying the spectral peaks that change position. Furthermore, using the above formula, α and f L can be calculated, which gives: TIFF2026507438000007.tif842This calculation assumes that the frame rate changes slightly by β, but the integer part α remains unchanged (i.e., α = α'):f L =(α+β)f=(α'+β')f'. Since α=α', this gives: TIFF2026507438000008.tif1031

[0076] With the same assumptions, in step 307, a new target frame rate f'' can be calculated, which will reduce the flicker to a new frequency f A '' can be shifted to: TIFF2026507438000009.tif990This is f, f', f A , f A ', and f A Continuing with the numerical example above, we denote the target flicker frequency f A A frame rate f''=7Hz=420 bpm can be chosen, which, according to the above formula, requires a frame rate f''=19.9 Hz.

[0077] 4A is a flowchart of a method for determining a photoplethysmography signal according to an example of the present subject matter. The method can be performed, for example, by the medical system of FIG.

[0078] In step 401, a sequence of images can be provided. FIG. 4B shows a sequence of images I1...I M For example, a sequence of images I1...I M may be the first series of images acquired in step 201 of FIG. 2, or the second series of images acquired in step 209A.

[0079] In step 403, clustering can be performed to determine clusters within the sequence of images. For example, different ROIs may be located within an imaging zone, and in step 403, a cluster can be generated for each ROI. Such ROIs may, for example, cover different parts of the face of the subject 104. In this example, the ROIs do not overlap, although the present subject matter is not limited to be used for partially or completely overlapping ROIs. Figure 4B shows several clusters cl1, cl2, ..., cl N Each cluster cl i has its own m i pixel set px1 i ...pf mi i where i varies between 1 and N.

[0080] Image I1...I M For each image in the image, step 405 can be performed. In step 405, the pixel values ​​of the clusters present in the image can be combined to determine a photoplethysmography signal value. This results in a set of M photoplethysmography signal values ​​PPG1...PPG, which form the photoplethysmography signal. M The present subject matter may provide alternative techniques for performing step 405.

[0081] In the first example, the current kth image I k Regarding photoplethysmography signal values ​​PPG k can be calculated cluster-wise. k Each cluster in can be processed to calculate the average of the pixel values ​​of the cluster. For each ith cluster, the average Av i can be defined as follows: TIFF2026507438000010.tif1022 Also, the photoplethysmography signal value can be defined as: TIFF2026507438000011.tif823

[0082] In one second example, the current kth image I k For , the photoplethysmography signal value can be obtained in pixels, which can be defined as: TIFF2026507438000012.tif923

[0083] where m T Here is the image I k The total number of pixels in a cluster of m T =Σ i=1 N m i is.

[0084] In one third example, the current kth image I k Regarding photoplethysmography signal values ​​PPG k can be calculated cluster-wise. k Each cluster in can be processed to calculate the average of the pixel values ​​of the cluster. For each ith cluster, the average Av i can be defined as follows: TIFF2026507438000013.tif1020Also, the photoplethysmography signal value can be defined as a weighted sum of the averages: TIFF2026507438000014.tif927where, w i is the weight assigned to the i-th cluster. Using weights allows us to downweight or ignore some clusters, e.g., w i can be set to zero to ignore the i-th cluster, or can be set to a value less than the remaining weights to downweight the i-th cluster. This weighted sum can be used, for example, to perform step 207 of FIG. 2.

[0085] In one fourth example, the current kth image I k For , the photoplethysmography signal value can be obtained in pixels, which can be defined as: TIFF2026507438000015.tif1127where m T Here is the image I k The total number of pixels in a cluster of m T =Σ i-1 N m i and w l is the weight assigned to the l-th pixel. For example, the same weight can be assigned to pixels in the same cluster. l can be set to zero to ignore the l-th pixel, or to a value less than the remaining weights to downweight the l-th pixel.

[0086] FIG. 5 illustrates another example of a medical device 500. In this example, the medical device 500 includes a magnetic resonance imaging system 502. The magnetic resonance imaging system 502 includes a magnet 504. The magnet 504 is a cylindrical superconducting magnet with a bore 506 extending therethrough. Different types of magnets can be used, including split-type cylindrical magnets and so-called open magnets. Split-type cylindrical magnets are similar to standard cylindrical magnets except that they are divided into two sections to allow a cryostat to access the magnet's equal surface. Such magnets can be used, for example, in conjunction with charged particle beam therapy. Open magnets have two magnet sections, one above the other, with a space between them large enough to accommodate a subject. The arrangement of the two magnet sections resembles a Helmholtz coil. Open magnets are popular because the subject is not enclosed. Inside the cryostat of the cylindrical magnet is a collection of superconducting coils. Within the bore 506 of the cylindrical magnet 504 is an imaging volume 508 where the magnetic field is strong and sufficiently uniform to perform magnetic resonance imaging. A region of interest 509 is shown within the imaging volume 508. A subject 518 is shown supported by a subject support 520 such that at least a portion of the subject 518 is within the imaging volume 508 and the region of interest 509.

[0087] Also within the magnet bore 506 are a set of magnetic field gradient coils 510 used to acquire magnetic resonance data for spatially encoding magnetic spins within the imaging volume 508 of the magnet 504. The magnetic field gradient coils 510 are connected to a magnetic field gradient coil power supply 512. The magnetic field gradient coils 510 are intended to be exemplary. Typically, the magnetic field gradient coils 510 have three separate coil sets for spatial encoding in three orthogonal spatial directions. The magnetic field gradient power supply supplies current to the magnetic field gradient coils 510. The current supplied to the magnetic field gradient coils 510 is controlled as a function of time and may be ramped or pulsed.

[0088] Adjacent to the imaging volume 508 is a radio frequency coil or surface coil 514 for manipulating the orientation of magnetic spins within the imaging volume 508 and for receiving radio frequency transmissions from the spins within the imaging volume 508. The radio frequency antenna can have multiple coil elements. The radio frequency antenna is sometimes referred to as a channel or antenna. The radio frequency coil 514 is connected to a radio frequency transceiver 516. The radio frequency coil 514 and the radio frequency transceiver 516 can be replaced with separate transmit and receive coils, as well as separate transmitters and receivers. It is understood that the radio frequency coil 514 and the radio frequency transceiver 516 are representative examples. The radio frequency coil 514 is also intended to represent a dedicated transmit antenna and a dedicated receive antenna. Similarly, the transceiver 516 can also represent separate transmitters and receivers. The radio frequency coil 514 can also have multiple receive / transmit elements, and the radio frequency transceiver 516 can have multiple receive / transmit channels. For example, when a parallel imaging technique such as SENSE is implemented, the radio frequency coil 514 can have multiple coil elements.

[0089] In this example, camera 501 is shown attached to the flange of magnet 504. The subject's face is imaged by camera 501.

[0090] The transceiver 516, gradient controller 112, and camera 106 are shown connected to a hardware interface 528 of a computer system 526. The computer system further includes a processor 530 in communication with the hardware system 528, a memory 534, and a user interface 532. The memory 534 may be any combination of memory accessible to the processor 530, including memory such as main memory, cache memory, and non-volatile memory such as flash RAM, a hard drive, or other storage device. In some examples, the memory 530 may be considered a non-transitory computer-readable medium.

[0091] The computer memory 534 is shown as including machine-executable instructions 540. The machine-executable instructions include commands or instructions that enable the processor 530 to control the operation and functionality of the magnetic resonance imaging system 502. The computer memory 534 is further shown as including pulse sequence commands 542. The pulse sequence commands 542 are either instructions or data that can be converted into instructions that enable the processor 530 to control the magnetic resonance imaging system 502 to acquire magnetic resonance data.

[0092] Computer memory 534 is shown having a series of images 544 acquired by camera 501. Various images in the series 544 can be used to derive vital sign information 546, which is also shown as being stored in memory 534. For example, the method shown in Figure 2 can be executed iteratively to generate vital sign information 546 that can be used to trigger the acquisition of magnetic resonance data 548 using pulse sequence commands 542. The magnetic resonance data 548 can then be reconstructed into a magnetic resonance image 550.

[0093] 6 is a flowchart of a method for acquiring MRI data according to one example of the present subject matter. The method can be performed, for example, by the medical system of FIG.

[0094] In step 601, the method of FIG. 2 can be performed to determine vital signal information of a subject.

[0095] In step 603, the MRI system can be controlled to acquire MRI data according to the vital signal information. For example, this can enable acquisition of high-quality cardiac MRI data by synchronizing image acquisition with specific times within the subject's cardiac cycle. The vital sign information can be used, for example, to detect the largest and steepest peaks in the cardiac cycle or other features of the vital signals to trigger acquisition of portions of k-space. Steps 601 and 603 can be repeated over many cardiac cycles to acquire MRI data for one MRI scan, e.g., a single MR image or set of MR images.

[0096] Figure 7a shows the measured heart rate (vertical axis) as a function of acquisition time (horizontal axis), representing the true PPG signal at 60 bpm and the flicker-based signal at 140 bpm. Figure 7b shows a gray-based color overlay of the amplitude of the flicker-based signal at 140 bpm. The color bar indicates the normalized value of the signal amplitude. This plot indicates that the flicker signal does not originate from human skin but from the MR system bore wall, visible in the upper left corner of the image; this is likely a reflection of flicker ambient illumination in that portion of the bore wall.

Claims

1. 1. A system comprising: an optical imaging system configured to acquire a series of optical images; a memory storing machine-executable instructions; a processor for controlling the system; and wherein execution of the machine-executable instructions causes the processor to: acquiring a first series of images of an object using the optical imaging system, the first series of images being acquired at a first frame rate; determining a first cluster of pixels in the first series of images; determining that a flicker signal is present in one or more of the first clusters of pixels; performing at least one of the steps of: performing a combined weighting of pixel values ​​of the first cluster to determine a first photoplethysmographic signal, the weighting being performed to reduce or eliminate the contribution of the flicker signal; and acquiring a second series of images at a second frame rate to prevent the flicker signal, determining a second cluster of pixels in the second series of images, and combining pixel values ​​of the second cluster to determine a second photoplethysmographic signal. deriving vital sign information of the subject from the first photoplethysmographic signal and / or the second photoplethysmographic signal; To run the system.

2. The system of claim 1 , wherein the acquisition of the second series of images is performed to shift the flicker signal out of a cardiac frequency band.

3. 3. The system of claim 1, wherein the second frame rate is obtained by shifting the first frame rate by an epsilon shift.

4. The flicker signal is caused by a modulation of the luminous flux of a light source, the modulation having a modulation frequency f defined as a function of the first frame rate f. L , where the frequency of the flicker signal is f A = |β|f, where the second frame rate f″ is defined as where f' is the intermediate frame rate and f A ' is the frequency of the flicker signal obtained for the intermediate frame rate f', and f A 4. The system of claim 1, wherein f'' is a desired frequency of the flicker signal relative to the second frame rate f''.

5. 5. The system of claim 1, wherein the combined weighting is performed using weights, and the combined weighting is performed to ignore the one or more first clusters using null weights or to downweight the one or more first clusters using weights having lower values ​​compared to the other weights.

6. the combining of pixel values ​​of the second clusters comprises, for each image in the second series of images, determining an individual photoplethysmography signal for each cluster using pixel values ​​of the clusters, and combining the individual photoplethysmography signals to obtain the second photoplethysmography signal; 6. The system of claim 1, wherein the combined weighting of pixel values ​​of the clusters comprises: for each image in the first series of images, determining an individual photoplethysmography signal for each cluster using the pixel values ​​of the clusters; and performing combined weighting of the individual photoplethysmography signals using their respective weights to obtain the first photoplethysmography signal.

7. The system of claim 1 , wherein the first cluster is different from or the same as the second cluster.

8. 8. The system of claim 1, wherein each cluster of the first clusters comprises the same set of pixels in each image of the first series of images.

9. 8. The system of claim 1, wherein each cluster of the first clusters comprises a different set of pixels in two or more images of the first series of images.

10. The system of claim 1 , wherein the optical imaging system is configured to image a skin surface of the subject.

11. The system of claim 1 , wherein the optical imaging system comprises a plurality of cameras.

12. 12. The system of claim 1, further comprising a magnetic resonance imaging system, wherein the memory further comprises pulse sequence commands configured to control the magnetic resonance imaging system to acquire k-space data in accordance with the magnetic resonance imaging protocol, and wherein execution of the machine-executable instructions causes the computing system to further perform the step of acquiring the k-space data by controlling the magnetic resonance imaging system in accordance with the determined vital sign information.

13. 13. The system of claim 1, wherein the vital signs information comprises at least one of heart rate, respiratory rate, and real-time triggers.

14. 1. A computer program having machine-executable instructions, the execution of which causes a computing system to: acquiring a first series of images of an object using an optical imaging system, the first series of images being acquired at a first frame rate; determining a first cluster of pixels in the first series of images; determining that a flicker signal is present in one or more of the first clusters of pixels; performing at least one of the steps of: performing a combined weighting of pixel values ​​of a first cluster to determine a first photoplethysmographic signal, said weighting being performed to reduce or eliminate the contribution of a flicker signal; and acquiring a second series of images at a second frame rate to prevent a flicker signal, determining a second cluster of pixels in the second series of images, and combining pixel values ​​of the second cluster to determine a second photoplethysmographic signal; deriving vital sign information of the subject from the first photoplethysmographic signal and / or the second photoplethysmographic signal; A computer program that executes

15. acquiring a first series of images of an object using an optical imaging system, the first series of images being acquired at a first frame rate; determining a first cluster of pixels in the first series of images; determining that a flicker signal is present in one or more of the first clusters of pixels; performing at least one of the steps of: performing a combined weighting of pixel values ​​of the first cluster to determine a first photoplethysmographic signal, the weighting being performed to reduce or eliminate the contribution of the flicker signal; and acquiring a second series of images at a second frame rate to prevent the flicker signal, determining a second cluster of pixels in the second series of images, and combining pixel values ​​of the second cluster to determine a second photoplethysmographic signal. deriving vital sign information of the subject from the first photoplethysmographic signal and / or the second photoplethysmographic signal; A method having the following.

Images