Image processing for determining a physiological parameter

Abstract

The invention relates to a method for determining a physiological parameter such as a heart rate of an individual based on a source image stream (IS1, IS2,... ISM), comprising generating averaged images and an operation of individually selecting pixels in each of the averaged images on the basis of a variation over time in the value of these pixels.

Description

Image processing for the determination of a physiological parameter

[0001] The invention relates to the field of image processing.

[0002] The invention is of particular interest in the field of non-contact photoplethysmography. State of the prior art

[0003] Non-contact photoplethysmography makes it possible to determine a physiological parameter of an individual such as heart rate, that is to say the average number of contractions of the heart muscle in one minute, called "beats per minute".

[0004] Generally speaking, this non-invasive optical technique consists of evaluating variations in blood volume in superficial tissues from variations in light absorption in these tissues.

[0005] Known non-contact photoplethysmography techniques provide measurements whose accuracy and reliability depend heavily on variations in ambient lighting and individual movement, leading to measurement errors.

[0006] An aim of the invention is to reduce such measurement errors.

[0007] Another aim of the invention is to provide an image processing solution compatible with acquisition of these images using different types of camera, for example monochrome or polychrome cameras, and / or in different lighting conditions, for example in the absence of visible light.

[0008] To this end, the invention relates to a method for determining a physiological parameter of an individual from a flow, called a “source flow”, of images of a region of the individual's body, this method comprising the following steps:– generation of a flow, called an “averaged flow”, of images which are each obtained by averaging several successive images of the source flow selected by moving a sliding window;– generation of a stream, called “processed stream”, of images which are each obtained from two respective successive images of the averaged stream so that, for each of the images of the processed stream, one or more pixels of this image has a value:- equal to the value of a corresponding pixel of one of said two respective images of the averaged stream when the absolute value of the difference between this value and the value of a corresponding pixel of the other of said two respective images of the averaged stream is greater than a first threshold and less than a second threshold,- equal to zero in the opposite case;– determination of said physiological parameter from the variations in pixel values ​​of the images of the processed stream.;

[0009] Conventionally, an image comprises a set of pixels forming a matrix of numbers which each represent a light intensity of a respective one of these pixels. The value of a pixel is the light intensity of this pixel. For the purposes of the present invention, an image is therefore a series of numbers which does not necessarily take the form of a visual representation.

[0010] From a terminological point of view, a "corresponding" pixel of an image, relative to a pixel of another image, is a pixel of the same coordinate. For example, in a matrix coordinate system, a pixel of coordinates (2, 3) of a first image is a pixel corresponding to the pixel of coordinates (2, 3) of a second image, and vice versa.

[0011] The step of generating the averaged flow makes it possible in particular to reduce the noise present in the source flow, by averaging images of this source flow. More precisely, for each series of images of the source flow undergoing this operation, the averaging is preferably carried out by calculating the average of the respective values ​​of corresponding pixels of the images of this series, individually for each of the pixels. In other words, the value of each of the pixels of each of the images of the averaged flow preferably corresponds to the average of the values ​​of corresponding pixels of images of the series of images of the source flow used to generate this image of the averaged flow.

[0012] The method of the invention as defined above, and in particular the step of generating the processed flow, makes it possible to determine a physiological parameter on the basis of an individual selection of pixels on images of the averaged flow, by not taking into account pixels having variations in light intensity that are too small or too large from one image to another of the averaged flow. This makes it possible in particular to discard data that would be representative not of an actual temporal evolution of the physiological parameter of interest but for example of a modification of the brightness or ambient lighting.

[0013] This results in a reduction of errors in the determination of the physiological parameter, while providing a corresponding confidence index, the latter being proportional to the number of non-zero pixels in the images of the processed flow.

[0014] In addition, such image processing reduces the constraints relating to the luminous properties of the signals represented by the images of the source stream. In particular, a simple variation in contrast is sufficient to vary the luminous intensity of one or more pixels from one image to another. In a non-limiting manner, the method is thus fully functional when the source stream is acquired in the presence of backlighting or a light defect, or when this source stream represents electromagnetic waves located in the near infrared.

[0015] More generally, the invention makes it possible to provide reliable and precise results from a stream of images which can be acquired in various light conditions and using different types of cameras, for example a monochrome or trichromatic camera.

[0016] The physiological parameter is preferably a vital biological parameter such as heart rate. Other physiological parameters may be determined using the method of the invention, for example heart rate variability, blood pressure or respiratory rate. These examples are not limiting.

[0017] In one embodiment, said sliding window is moved by a step less than the number of images of the source stream used to obtain any of the images of the averaged stream.

[0018] In other words, the displacement step of the sliding window is preferably less than the size of this window.

[0019] For example, this step can be one image or two images.

[0020] In one embodiment, the number of images of the source stream used to obtain each of the images of the averaged stream is odd, for example equal to 3 or 5.

[0021] In one embodiment, said first threshold is equal to 0.

[0022] In one embodiment, said second threshold is equal to 1 / ((N1-1)*K), N1 corresponding to the number of values ​​that the pixels of the images of the source stream can have, K corresponding to the number of images of the source stream used to obtain each of the images of the averaged stream.

[0023] In one embodiment, the method comprises generating a signal from values ​​which each correspond to the average value of the pixels of a respective one of the images of the processed stream whose value is different from zero.

[0024] In other words, for each of the images in the processed stream, the non-zero pixels of this image can be averaged in order to constitute one of the respective values ​​of this signal.

[0025] It is preferred that the determination of the physiological parameter be carried out on the basis of this signal.

[0026] In one embodiment, the method comprises a step of acquiring the source stream using a camera.

[0027] According to a first embodiment, the camera is monochrome.

[0028] According to a second embodiment, the camera is polychrome.

[0029] In this second variant, it is preferred that the pixel value corresponds to the light intensity of a single color channel.

[0030] This makes it possible to simplify processing.

[0031] For example, this color can be green.

[0032] Of course, several color channels can each be processed in parallel as described above.

[0033] In one embodiment, the method further comprises, before the acquisition of the source stream, a step of adjusting exposure and / or brightness parameters of the camera and, during the acquisition of the source stream, maintaining the values ​​of these parameters.

[0034] Such a setting allows to optimize the useful information present in the source stream images and to reduce the information due to measurement artifacts.

[0035] The invention also relates to a device comprising means suitable for carrying out the steps of the method as defined above.

[0036] In one embodiment, this device further comprises a camera configured to carry out the step of acquiring the source stream of the method as defined above.

[0037] For example, this camera may belong to a digital device such as a mobile phone, tablet or computer.

[0038] The invention also relates to a computer program comprising executable computer instructions which, when executed by computer, implement the steps of the method as defined above.

[0039] Other advantages and characteristics of the invention will appear on reading the detailed, non-limiting description which follows.

[0040] The following detailed description refers to the attached drawings in which:

[0041] is a diagram illustrating steps of a method according to the invention;

[0042] is a schematic view of a source image stream;

[0043] is a schematic view illustrating a generation of a first image of an averaged stream from a first series of images of the source stream;

[0044] is a schematic view illustrating a generation of a second image of an averaged stream from a second series of images of the source stream;

[0045] is a schematic view of a processed stream of images, generated from the averaged stream. Detailed description of embodiments

[0046] An example of a method according to the invention is described below, making it possible in particular to determine the heart rate of an individual. This method or variants of this method can be implemented in a similar manner to detect another physiological parameter such as heart rate variability, blood pressure, respiratory rate, or even, in a non-limiting manner, the oxygen saturation level of this individual.

[0047] In this non-limiting example, the method successively comprises the following steps which are illustrated in the diagram:– A1: adjustment of a digital camera;– A2: acquisition of an image stream, called “source stream”;– A3: generation of an image stream, called “averaged stream”, from the source stream;– A4: generation of an image stream, called “processed stream”, from the averaged stream;– A5: determination of the physiological parameter;– A6: display of the result.

[0048] In this example, the camera is monochrome and can acquire digital images, each of which has a number of B1*B2 pixels.

[0049] In a manner known per se, for each pixel P(C1,C2) of each image organized in matrix form, C1 is a first coordinate of the pixel corresponding for example to a column number of the matrix and C2 is a second coordinate of the pixel corresponding in this example to a row number of the matrix. B1 thus corresponds to the number of columns and B2 to the number of rows of the matrix so that each pixel has a coordinate C1 between 1 and B1 and a coordinate C2 between 1 and B2.

[0050] In this example, B1 is equal to 4000 and B2 is equal to 3000, so each image acquired using this camera has a definition of 12 million pixels.

[0051] The value I(C1,C2) of each pixel P(C1,C2) represents in this example its light intensity in gray levels and is a natural integer between 0 and 255. Each pixel P(C1,C2) can thus have 256 values.

[0052] Step A1 includes fixing the camera brightness to a given value and adjusting the camera exposure, using a computer program designed to change the exposure incrementally from zero exposure until the average value I moy (C1,C2) of each pixel P(C1,C2) of a series of successively acquired images is between a lower threshold and an upper threshold, for example between 160 and 200. This series of images acquired to carry out this adjustment includes in this example 30 images per second.

[0053] If this procedure does not allow an average value to be obtained I moy (C1,C2) of each pixel P(C1,C2) between these lower and upper thresholds, the program modifies the brightness then repeats the exposure adjustment as described above, and so on until the above conditions are met.

[0054] The exposure and brightness values ​​thus obtained are maintained during the A2 acquisition step, which avoids measurement artifacts that would result from automatic adjustment of these parameters.

[0055] The camera is then positioned to acquire images of a region of the individual's body, in this example part of their face.

[0056] It schematically represents a succession of M IS images i (i ranging from 1 to M) thus acquired and constituting said source flow. These IS images i are called “source images”.

[0057] The source flux thus acquired is representative of a temporal variation of light information backscattered by superficial tissues of the individual's face, corresponding to a temporal variation of blood volume in vessels running through these tissues, a larger blood volume resulting in a greater absorption of light by the oxygen contained in the blood.

[0058] Step A3 of generating said averaged flow is described below with reference to Figures 3 and 4.

[0059] To generate the averaged flow, several series are made up, each containing K images of the source flow, K generally being greater than 1 and less than M and in this example being equal to 3.

[0060] The first series, illustrated in the figure, includes the first three images IS1, IS2 and IS3 of the source stream.

[0061] The second series, illustrated in the, includes images IS2, IS3 and IS4 of the source stream, that is, the three consecutive images starting from the second image IS2 of the source stream.

[0062] The third series, not shown, comprises three consecutive images of the source stream starting from the third IS3 image.

[0063] More generally, each of the series comprises K consecutive images of the source stream starting from a respective image of the source stream, which in this case makes it possible to constitute a number of L=M-(K-1) series.

[0064] In other words, these L series of K successive source images are selected by moving a sliding window with in this example a step of 1 image.

[0065] For each of the series thus constituted, the source images of this series are averaged so as to form one of the respective IM images j (j ranging from 1 to L) of said averaged flux.

[0066] Thus, with reference to the, the value of each of the pixels P(C1,C2) of the averaged image IM1 resulting from the averaging of the source images IS1, IS2 and IS3 of the first series is equal to the average of the values ​​of this pixel P(C1,C2) on each of the source images IS1, IS2 and IS3.

[0067] The same applies to the production of the averaged image IM2 resulting from the averaging of the source images IS2, IS3 and IS4 of the second series () as well as for the other images of the averaged flow.

[0068] Given these average calculations, the value of each of the pixels P(C1,C2) of the averaged images IM j is a real number, compared to the value of each pixel in the source images which is a natural whole number.

[0069] Step A4 of generating said processed flow will now be described.

[0070] The processed stream includes a number P of IT images n, called “processed images”, which each include part of the information from the averaged IM images j with j ranging from 1 to L-1, respectively.

[0071] The processed stream in this example includes one fewer image than the averaged stream.

[0072] More specifically, regarding the first image IT1 of the processed stream, each of the pixels P(C1,C2) of this image IT1 has a value which is equal to the value of the corresponding pixel P(C1,C2) of the averaged image IM1 if the absolute value of the difference between this value and the value of the corresponding pixel P(C1,C2) of the following averaged image, i.e. in this case the image IM2, is greater than a first threshold S1 and less than a second threshold S2. If this absolute value is less than or equal to 1, or if it is greater than or equal to S2, the corresponding pixel P(C1,C2) of this image IT1 has a value equal to zero.

[0073] In other words, the image IT1 of the processed flow retains on the one hand the pixels of the image IM1 of the averaged flow when the variation of their value between the image IM1 and the image IM2 is between the thresholds S1 and S2 and, on the other hand, replaces the value of the other pixels with a zero value.

[0074] The second image IT2 of the processed flow is obtained in a similar manner from the averaged images IM2 and IM3.

[0075] The same applies to IT3 images to IT P which are obtained from images IM3 to IM L , that is to say by moving a sliding window selecting at each iteration two successive averaged images, this window being in this example moved by a step of 1 image.

[0076] In this particular example, the threshold S1 is equal to 0 and the threshold S2 is equal to 1 / ((N1-1)*K), N1 being the number of values ​​that the pixels of the images of the source stream can have, that is to say in this example 256. These thresholds can be determined empirically depending for example on the nature of the physiological parameter to be determined.

[0077] From a temporal point of view, each of the IM images j of the averaged stream is positioned at the same level as the intermediate image of the series of source images used to generate this averaged image. For example, the first image IM1 of the averaged stream corresponds to the temporal position of the source image IS2. As a result, the first image IT1 of the processed stream is in this example temporally shifted by one image relative to the first image IS1 of the source stream.

[0078] Taking as an example an acquisition of 30 source images per second, this results in a time shift of 1 / 30 seconds between the processed stream and the source stream.

[0079] The physiological parameter, in this example the heart rate, can be determined A5 on the basis of the processed flow thus obtained.

[0080] In this non-limiting example, a time signal is formed, each point of which has an amplitude equal to the average value of the non-zero pixels of a respective one of the images of the processed stream. Thus, the first point of this signal has an amplitude corresponding to the average value of the non-zero pixels present on the first image IT1 of the processed stream, and so on for the following points.

[0081] Additional filtering is optionally performed in the case where the amplitudes of two successive points of the signal have too great a difference from each other, for example a difference corresponding to an increase greater than 20%. For example, the amplitude of the second point can then be reduced so that its difference corresponds to a variation of 20%.

[0082] In this example, during step A6, the result thus obtained is displayed on a screen. This display may comprise a visual representation of said signal, with and / or without additional filtering, and / or one or more values ​​representative of the heart rate and / or another physiological parameter calculated on the basis of this signal according to techniques which are well known as such.

[0083] The various steps just described are implemented using a computer program comprising corresponding executable computer instructions.

[0084] The preceding description is not limiting and numerous variants can be envisaged on this basis without departing from the scope of the invention. For example, the camera can be of a different nature and be for example a polychrome camera of a mobile telephone. The preceding description can in this case be applied by analogy for each of the color channels, it being understood that the processing can be carried out on the basis of the value of the pixels of a single color channel, for example green. Alternatively, the processing described above can be carried out on the basis of values ​​combining information provided by several color channels.

[0085] The numbers indicated in this description are given as examples. In particular, the images may contain a number of pixels less than or greater than 12 million. K may have a value other than 3, for example 5 or another preferably odd number. The step of the sliding window may be greater than 1 image, whether for the generation of the averaged flow and / or for the generation of the processed flow.

[0086] For another example, said temporal signal on the basis of which the physiological parameter is determined can be constructed not by calculating the average value of the pixels of each of the processed images but by using the values ​​of one or more given pixels having for example a non-zero value on several successive processed images.

Claims

A method for determining a physiological parameter, such as an individual's heart rate, from a stream, called the "source stream," of images (IS1, IS2, ... IS M ) of a region of the individual's body, this process comprising the following steps: – generation (A3) of a stream, called the "averaged stream", of images (IM1, IM2) each obtained by averaging several successive images (IS1, IS2, IS3; IS2, IS3, IS4) of the source stream selected by moving a sliding window; – generation (A4) of a stream, called the "processed stream", of images (IT1, IT2, … IT P ) which are each obtained from two successive respective images (IM1, IM2) of the averaged flow such that, for each of the images (IT1, IT2, … IT P) of the processed flow, one or more pixels of this image has a value: - equal to the value of a corresponding pixel of one of the two respective images (IM1, IM2) of the averaged flow when the absolute value of the difference between this value and the value of a corresponding pixel of the other of the two respective images (IM1, IM2) of the averaged flow is greater than a first threshold and less than a second threshold, - equal to zero otherwise; - determination (A5) of said physiological parameter from the variations in pixel values ​​of the images (IT1, IT2, … IT1 P ) of the processed stream. A method according to claim 1, wherein said sliding window is moved in a step smaller than the number of images of the source stream used to obtain any one of the images (IM1, IM2) of the averaged stream, this step being for example one image. A method according to claim 1 or 2, wherein said first threshold is equal to 0 and said second threshold is equal to 1 / ((N1-1)*K), N1 corresponding to the number of values ​​that the pixels of the images (IS1, IS2, … IS1) can have. M ) of the source stream, K corresponding to the number of images of the source stream used to obtain each of the images (IM1, IM2) of the averaged stream. A method according to any one of claims 1 to 3, comprising generating a signal from values ​​that each correspond to the average value of the pixels of one of the respective images (IT1, IT2, … IT1). P ) of the treated flow whose value is different from zero, the determination (A5) of the physiological parameter being carried out on the basis of this signal. A method according to any one of claims 1 to 4, comprising an acquisition step (A2) of the source stream using a camera. A method according to claim 5, wherein the camera is monochrome or polychrome, the pixel value preferably corresponding to the light intensity of a single color channel when the camera is polychrome. Method according to claim 5 or 6, further comprising, before the acquisition (A2) of the source stream, a step of adjusting (A1) the exposure and / or brightness parameters of the camera and, during the acquisition (A2) of the source stream, maintaining the values ​​of these parameters. Device comprising means adapted to carry out the steps of the process according to any one of claims 1 to 7. Device according to claim 8, further comprising a camera, belonging for example to a digital device such as a mobile phone, a tablet or a computer, the camera being configured to perform the acquisition step (A2) of the source stream of the process according to any one of claims 5 to 7. Computer program comprising executable computer instructions which, when executed by computer, implement the steps of the process according to any one of claims 1 to 7.

Images