Handheld device for the contactless measurement of vital parameters and methods for measuring vital parameters

Abstract

The invention relates to a handheld device (1) for the contactless measurement of vital parameters, comprising a housing (3) with an opening and a handle (5), an optical tube (13) for receiving an image, which is attached to the opening of the housing (3) and receives incident light, a first beam splitter (15a) which is arranged behind the optical tube (13) and directs the incident light on one side to a first image sensor (17a) and on the other side to a second beam splitter (15b), wherein the second beam splitter (15b) directs the incident light on a first filter (33a) and a second filter (33b), and a second image sensor (17b) and a third image sensor (17c) are arranged behind the first filter (33a), at least one light source (21) which is located on the housing (3) and is directed towards the opening of the housing (3), and an evaluation unit (19) includes, which consist of the image sensors (17a, 17b,17c) Vital parameters were determined from the image data recorded. The first and second filters (33a, 33b) are only transparent to near-infrared light.

Description

[0001] The invention relates to a handheld device for the contactless measurement of vital parameters, which comprises a housing with an opening, an optical tube for recording an image, which is attached to the opening of the housing and receives incident light, a first beam splitter which is arranged behind the optical tube and directs the incident light onto a first image sensor and onto a second beam splitter, wherein the second beam splitter directs the incident light onto a first filter and a second filter, and a second image sensor is arranged behind the first filter and a third image sensor is arranged behind the second filter, at least one light source which is located on the housing and is directed in the direction of the opening of the housing, and an evaluation unit which determines vital parameters from image data recorded by the image sensors.

[0002] Furthermore, the invention comprises a method for measuring vital parameters for a handheld device for contactless measurement of vital parameters.

[0003] As part of medical treatment, a number of vital signs are regularly recorded. This occurs, for example, during medical history taking, in intensive care, or when monitoring newborns in a neonatal unit. Vital signs such as pulse, respiratory rate, oxygen saturation, heart rate variability, and even body temperature are regularly measured.

[0004] In recent years, optical measurement techniques have become increasingly important in diagnostics due to their high reliability and rapid results. One widely used method is photoplethysmography (PPG), in which areas of a patient's skin are irradiated with light of various wavelengths, typically in the infrared range, to record reflected or transmitted light signals and use these to calculate heart rate and oxygen saturation. The method is based on the fact that hemoglobin in the blood absorbs light more strongly than the surrounding tissue. The temporal change in the proportion of reflected light can be used to examine changes in blood volume in order to determine the patient's pulse and variations in the volume transported per pulse.Deviations in the shape of individual pulse beats can also be used to determine parameters such as respiratory rate or heart rate variability. To determine oxygen saturation, multiple wavelengths are used, and their reflected and transmitted components are examined.

[0005] Classic, contact-based vital sign measurement methods such as pulse oximetry primarily examine heart rate and oxygen saturation, although temporal measurements can also be used to examine related vital signs such as heart rate variability. For this purpose, a pulse oximeter is typically attached to a finger or earlobe, or held against a patient's forehead.

[0006] In addition, methods for contactless measurement of vital signs are known, often based on cameras in combination with external lighting. For example, areas of the human body, such as fingers, feet, or the forehead, are imaged and analyzed with the cameras. Infrared light emitted by a photodiode is captured by the cameras, and the smallest changes in color components or light intensities are evaluated.

[0007] Such a system and an associated method are disclosed in EP 3 207 862 A1. The system comprises an illumination unit for illuminating a skin region of a patient, consisting of a first and a second radiation source of different wavelengths and / or polarization channels that emit modulated electromagnetic radiation. A detection unit receives the radiation reflected from the patient, wherein a plurality of detection elements are present, each comprising at least two sensors and each generating a detection signal. A processing unit demodulates the detection signals, from which vital parameters are subsequently extracted by means of a vital parameter determination unit using a statistical measure, such as a standard deviation, a variance, or a mean absolute difference.For example, using a weighted combination, the demodulated acquisition signals are combined linearly, resulting in a pulse signal that is further evaluated to derive vital parameters.

[0008] Such systems are location-dependent, thus limiting their application. Their use outside of a clinical setting or in outpatient care and nursing is not readily possible. Measurement using contact-based methods requires a certain amount of additional effort due to the necessary cabling of patients and the necessary hygiene measures when multiple patients need to be examined. Furthermore, such methods can sometimes be uncomfortable for the patient, as they require adhesive electrodes.

[0009] The objective is to provide a device and a corresponding method that allow mobile measurement of vital signs. Furthermore, the aforementioned disadvantages should be eliminated.

[0010] The problem is solved for the aforementioned handheld device for the contactless measurement of vital parameters in that the first and second filters are only permeable to light in the near infrared range, with the first filter only permeable to light in a first wavelength range and the second filter only permeable to light in a second wavelength range different from the first wavelength range. Such a design is compact enough to provide a portable handheld device that can reliably determine a patient's vital parameters. Such a device can be battery-operated and thus also be used outside of the clinical setting and in outpatient settings.

[0011] It is advantageous if the light source consists of two LEDs, one emitting light from the first wavelength range and the other from the second. Unlike broadband light sources, this allows the illumination to be limited to the wavelength ranges under investigation, increasing the intensity of the light in the relevant wavelength ranges, making the measurement more reliable without negatively affecting battery life.

[0012] In another advantageous embodiment, the first image sensor is a Bayer sensor. Such a sensor is covered with a checkerboard color filter, separating the color channels, usually red, green, and blue. This allows for the use of existing methods for measuring vital parameters, particularly heart rate and respiratory rate, which are already suitable for determining these parameters on their own, provided favorable lighting conditions exist. In combination with the second and third image sensors, the reliability of the handheld device can be further increased.

[0013] Furthermore, it is advantageous if the first wavelength range is in the range from 750 nm to 800 nm and the second wavelength range is in the range from 850 nm to 900 nm. Both of these wavelengths are in the near infrared range, which here primarily includes wavelengths between 700 nm and 1100 nm, and have proven particularly suitable for measuring vital signs.

[0014] Furthermore, it is advantageous if a brightness sensor is attached to the housing, which is connected to the evaluation unit, and the brightness signals emitted by the brightness sensor are taken into account when determining the vital parameters. Since the handheld device is used in a variety of lighting conditions, it is beneficial to consider the lighting conditions for reliable determination of the vital parameters. A particular advantage of a brightness sensor is that it allows for the consideration of interference frequencies projected by ambient light sources such as LED light sources, fluorescent tubes, or computer monitors due to their repetition rate.These can make the measurement more difficult if they are in the range of the heart rate or the respiratory rate or if an interference signal is superimposed, for example with the sampling rate of the image sensors, resulting in a beat in the range of the heart rate or the respiratory rate, and are taken into account by the brightness sensor when determining the vital parameters.

[0015] In a further advantageous embodiment, the handheld device has a motion sensor connected to the evaluation unit, and motion signals emitted by the motion sensor are taken into account when determining the vital parameters. When using the handheld device, a further disturbance is the tremor of the user of the handheld device, i.e., the natural vibration of the hand. This can be within the range of the vital parameters to be measured and thus interfere with the measurement. By incorporating the motion signals into the determination of the vital parameters, the reliability of the measurement can be increased. Furthermore, image stabilization can also be performed using the motion sensor.

[0016] Alternatively or additionally, it is advantageous if the housing is equipped with a barcode scanner and / or if the evaluation unit is capable of performing facial recognition based on the recorded image data, allowing the derived vital parameters to be assigned to an individual data set using the barcode scanner or facial recognition. This facilitates use in any field, as the determined vital parameters can be automatically assigned to the patient currently being examined.

[0017] The object is further achieved by a method for measuring vital parameters for a handheld device for contactless measurement of vital parameters as described above, which comprises the following steps: - Recording of incident light through an optical tube of the handset, - Projecting the light onto a first, a second and a third image sensor, wherein the second image sensor receives only light of a first wavelength range and the third image sensor receives only light of a second wavelength range different from the first wavelength range, each of which lies in the near infrared, - Recording a first image data set using the first image sensor, a second image data set using the second image sensor and a third image data set using the third image sensor, - Determination of vital parameters by an evaluation unit located in the handheld device based on the first, second and third image data sets.

[0018] It is advantageous if the evaluation unit takes into account brightness signals emitted by a brightness sensor attached to the handheld device when determining vital parameters. This allows the brightness signals to be taken into account when determining vital parameters in order to increase the accuracy of the method and evaluate the reliability of the measurement.

[0019] Furthermore, or in addition, it is advantageous if the evaluation unit considers motion signals emitted by a motion sensor located on the handheld device when determining vital parameters. This allows the motion signals to be taken into account when determining vital parameters, increasing the accuracy of the procedure and assessing the reliability of the measurement.

[0020] In another advantageous variant, the evaluation unit performs an accuracy check when determining vital parameters and rejects the measurement if the accuracy of the vital parameter determination falls below a reference value. Especially when using a handheld device, a number of interfering factors occur that influence the accuracy of the measurement. Especially when diagnosing patients, deviations induced by interfering factors can have far-reaching consequences, even leading to misdiagnosis, and must be avoided at all costs. This risk can be eliminated by performing an accuracy check of the measurement.

[0021] It is also advantageous if the evaluation unit includes a memory and the vital parameters are assigned to an individual data set that is stored in the memory. This allows the measured vital parameters to be stored individually for each patient and subsequently loaded from the handheld device.

[0022] Alternatively or additionally, it is particularly advantageous if the individual data set is selected using a barcode scanner on the handheld device or facial recognition based on the image data. This allows the data to be assigned to an individual patient particularly quickly.

[0023] Furthermore, it is advantageous for the method if the image data comprises a plurality of image sequences from which a pulse wave signal is derived. The pulse wave signal is an intermediate step in determining vital parameters and a direct representation of a change in blood volume within the blood vessels caused by the patient's heartbeat. Various methods can be considered for this determination, which examine multiple measurement time windows and determine the pulse wave signal within each of these time windows. Signal changes in color channels or statistical methods are used here.

[0024] It is understood that the features mentioned above and those to be explained below can be used not only in the combinations indicated, but also in other combinations or in isolation, without departing from the scope of the present invention.

[0025] The invention is explained in more detail below using exemplary embodiments with reference to the accompanying drawings, which also disclose features essential to the invention. These exemplary embodiments are for illustrative purposes only and are not to be interpreted as restrictive. For example, a description of an exemplary embodiment with a large number of elements or components should not be interpreted as meaning that all of these elements or components are necessary for implementation. Rather, other exemplary embodiments may also contain alternative elements and components, fewer elements or components, or additional elements or components. Elements or components of different exemplary embodiments may be combined with one another unless otherwise stated. Modifications and variations described for one of the exemplary embodiments may also be applicable to other exemplary embodiments.To avoid repetition, identical or corresponding elements in different figures are designated by the same reference numerals and are not explained more than once. They show: Fig. 1a A schematic representation of a handheld device for measuring vital parameters, Fig. 1b the handset in a frontal view, Fig. 2 Representation of an optical path of the handset and Fig. 3 a schematic representation of a method for determining vital parameters.

[0026] Fig. Figure 1a shows a handheld device 1 for measuring vital signs, comprising a housing 3, a handle 5, a screen 7 for displaying measurement results and other information, which can also be configured as a touchscreen to control the handheld device 1, and a switch 9 for triggering the measurement. The handheld device 1 is powered by a battery 11 to enable the necessary mobility. Data from the handheld device 1 can be read and the battery 11 can be charged via an interface 12.

[0027] Vital parameters are measured via an optical path. For this purpose, the handheld device 1 is aimed at a patient. The light emitted by the patient is picked up by an optical tube 13, collimated, and directed to a first beam splitter 15a. This splits the light so that a first portion is incident on a second beam splitter 15b, while a second portion of the light is directed to a first image sensor 17a. The first image sensor 17a is an RGB sensor that operates according to the Bayer sensor principle. This captures an image of the patient and outputs image data, which is then transferred to an evaluation unit 19 of the handheld device 1. Using methods such as artificial intelligence, it is then examined whether a patient has been imaged and which area is visible in the image, and the evaluation process is adjusted accordingly if necessary.The patient’s heart rate and respiratory rate, among other things, are then determined from the image data using expert methods such as the so-called “independent component analysis” (ICA).

[0028] The second part of the light, which strikes the second beam splitter 15b, is in turn split into two components, one of which is directed to a second image sensor 17b and the other to a third image sensor 17c. The second and third image sensors 17b, 17c are each sensitive to a wavelength range in the near infrared between 700 nm and 1100 nm. For this purpose, the second image sensor 17b comprises a first filter that is only permeable to light in a first wavelength range, and the third image sensor 17c comprises a second filter that is only permeable to light in a second wavelength range different from the first wavelength range. For example, the first wavelength range comprises wavelengths between 750 nm and 800 nm, and the second wavelength range comprises a range from 850 nm to 900 nm. In general, however, other, larger or smaller wavelength ranges can also be present.

[0029] The second and third image sensors 17b, 17c capture essentially the same image section as the first image sensor 17a, with the first and second wavelength ranges being used to determine additional vital parameters and to supplement the measurement with the first image sensor 17a. For this purpose, the second and third image sensors 17b, 17c are also connected to the evaluation unit 19 and each transmits the recorded image data to it.

[0030] The image data output by the second and third image sensors 17b, 17c can also be used to determine whether a patient has been scanned and which area is visible in the image. Furthermore, all image data are combined and evaluated in the evaluation unit 19 to determine the vital parameters.

[0031] To ensure that the handheld device 1 can be used even under unfavorable lighting conditions, it comprises a light source 21 that illuminates the area to be examined. Furthermore, a brightness sensor 23, a motion sensor 25, and a barcode scanner 27 are provided, which are connected to the evaluation unit 19. Brightness signals emitted by the brightness sensor 23 and motion signals emitted by the motion sensor 25 are transmitted to the evaluation unit 19 so that they are taken into account when determining the vital parameters. The brightness sensor 23 not only allows the lighting situation to be monitored to determine whether a measurement of vital parameters is possible, but can also be used to record external interference frequencies, such as those imposed on the ambient light by an LED or a nearby computer monitor.Such interference frequencies are picked up by the optical tube 13 and transmitted to the image sensors 17a, 17b, 17c, so that they can falsify the measurement if they are in the range of the heart rate or respiratory rate, or if a superposition of the interference signal with, for example, the sampling rate of the image sensors results in a beat in the range of the heart rate or respiratory rate. If an interference frequency is detected in the brightness signals by the evaluation unit 19, the image data is calculated with it and corrected accordingly. The motion signals output by the motion sensor 25 can be used for image stabilization and also contain information about the user's natural tremor, which can also impose an interference frequency and is taken into account accordingly.

[0032] Since these interference factors can have a significant influence on the measurement results, the evaluation unit 19 performs an accuracy test to determine the magnitude of the interference factors compared to the measured vital parameters. If the influence of the interference factors is so great that the determined vital parameters are inaccurate, the measurement is discarded and repeated. A warning is displayed on screen 7.

[0033] The barcode scanner 27 can be used prior to the measurement to more easily assign the vital parameter measurement to a patient. For this purpose, a barcode, QR code, or similar code is read by the barcode scanner 27 before the measurement and forwarded to the evaluation unit 19. This assigns the recorded vital parameters to an individual data set, which is stored in a memory 29. In this way, the determined vital parameters can be directly assigned to a patient during subsequent data transmission, for example, via the interface 12 or wirelessly via a WiFi chip located in the evaluation unit 19.

[0034] In addition, facial recognition is performed on the recorded image data before the actual measurement of vital signs takes place. Facial recognition can also be used to assign the data to an individual data set. The user can decide whether one or both methods—reading a code using the barcode scanner 27 and facial recognition—are used together.

[0035] The Fig. Figure 1b shows the handset 1 in a frontal view, allowing the elements facing the patient to be seen. The light source 21 is formed here by a first LED 21a and a second LED 21b. The first LED 21a primarily emits light in the first wavelength range, while the second LED 21b primarily emits light in the second wavelength range. This ensures that at least the second and third image sensors 17b, 17c are supplied with sufficient light, even in poor lighting conditions, and that the vital parameters are accurately determined.

[0036] In addition to the two LEDs 21a, 21b, the optical tube 13, and the barcode scanner 27, there is also a temperature sensor 31 for measuring the patient's body temperature. The temperature sensor 31 is connected to the evaluation unit 19.

[0037] Fig. Figure 2 shows the optical path of the handset 1 again in detail. In addition to the structure consisting of the optical tube 13, the beam splitters 15a, 15b, and the image sensors 17a, 17b, 17c, two additional filters 33a, 33b are present. A first filter 33a filters the light directed by the second beam splitter 15b to the second image sensor 17b, so that only the first wavelength range, which covers a range from 750 nm to 800 nm, reaches the second image sensor 17b. A second filter 33b filters the light directed by the second beam splitter 15b to the third image sensor 17c, so that only light from the second wavelength range, which here covers a range from 850 nm to 900 nm, reaches the second image sensor 17b.

[0038] Fig.Figure 3 schematically shows how vital parameters are extracted from the recorded image data sets, which here show a newborn. In a first block 35, an image sequence 35a, 35b, 35c is recorded using the image sensors 17a, 17b, 17c, in which regions of interest (ROIs) are selected for evaluation. These are highlighted in the first images of the image sequences 35a, 35b, 35c. Since a face was recorded here, the ROIs are typically on the forehead and cheeks and can be defined automatically or manually. Changes between individual images within the ROI are recorded and further processed. Individual pixel values ​​of the color channels output by the image sensors 17a, 17b, 17c are recorded individually and recorded over time in a second block 37.

[0039] These are then converted in a third block 39 into a five-dimensional color vector 39a, which represents several color channels: red, blue, and green from the image data of the first image sensor 17a, as well as the first and second wavelength ranges from the image data of the second and third image sensors 17b, 17c in the near infrared range. The change in the color vector 39a over time is recorded and used for further analysis.

[0040] A change in the individual degrees of freedom of the color vector 39a, i.e., the color channels, over time is extracted in order to derive a pulse wave signal 41a from it in a fourth block 41. For this purpose, a statistical evaluation, e.g., ICA, is performed, or the relative changes of several color channels are examined.

[0041] To determine the frequency components of the pulse wave signal 41a, a time window 41b of the pulse wave signal 41a is evaluated in the frequency domain, for example, using Fourier transformation, and finally, a vital parameter, such as the heart rate or respiratory rate, is derived. These are visible as a frequency peak 41c in the spectrum. Typically, averaging is performed over a series of time windows 41b. Furthermore, a progression of the vital parameters over time can be determined from successive time windows 41b, thus allowing the temporal change of the vital parameters to be examined. List of reference symbols 1 handset 3 housings 5 Handle 7 screen 9 switches 11 Battery 12 Interface 13 tube 15a first beam splitter 15b second beam splitter 17a first image sensor 17b second image sensor 17c third image sensor 19 Evaluation unit 21 Light source 21a first LED 21b second LED 23 Brightness sensor 25 motion sensor 27 barcode scanners 29 storage 31 Temperature sensor 33a first filter 33b second filter 35 first block 35a, b, c image sequence 37 second block 39 third block 39a Color vector 41 fourth block 41a Pulse wave signal 41b Time window 41c frequency peak QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] EP 3 207 862 A1

[0007]

Claims

[1] Handheld device for the contactless measurement of vital parameters, comprising - a housing (3) with an opening, - an optical tube (13) for recording an image, which is attached to the opening of the housing (3) and receives incident light, - a first beam splitter (15a) arranged behind the optical tube (13) and directing the incident light onto a first image sensor (17a) and onto a second beam splitter (15b), - wherein the second beam splitter (15b) directs the incident light onto a first filter (33a) and a second filter (33b), and a second image sensor (17b) is arranged behind the first filter (33a) and a third image sensor (17c) is arranged behind the second filter (33b), - at least one light source (21) located on the housing (3) and directed towards the opening of the housing (3), and - an evaluation unit (19) which determines vital parameters from image data recorded by the image sensors (17a, 17b, 17c), characterized by , that - the first and second filters (33a, 33b) are only permeable to light in the near infrared range, the first filter (33a) being only permeable to light of a first wavelength range and - the second filter (33b) is only permeable to light of a second wavelength range different from the first wavelength range. [2] Hand-held device according to claim 1, characterized by , that - the light source (21) consists of two light-emitting diodes (21a, 21b), one of which emits light of the first wavelength range and the other of which emits light of the second wavelength range. [3] Hand-held device according to claim 1 or 2, characterized by , that - the first image sensor (17a) is a Bayer sensor. [4] Hand-held device according to one of claims 1 to 3, characterized bythat the first wavelength range is in the range from 750 nm to 800 nm and the second wavelength range is in the range from 850 nm to 900 nm. [5] Hand-held device according to one of claims 1 to 4, characterized by that a brightness sensor (23) is attached to the housing (3) and is connected to the evaluation unit (19), and brightness signals output by the brightness sensor (23) are taken into account when determining the vital parameters. [6] Hand-held device according to one of claims 1 to 5, characterized by that the hand-held device (1) has a motion sensor (25) which is connected to the evaluation unit (19), and motion signals emitted by the motion sensor (25) are taken into account when determining the vital parameters. [7] Hand-held device according to one of claims 1 to 6, characterized bythat a barcode scanner (27) is provided on the housing (3) and / or facial recognition can be carried out with the evaluation unit (19) on the basis of the recorded image data, whereby the determined vital parameters can be assigned to an individual data set by the barcode scanner (27) or the facial recognition. [8] Method for measuring vital parameters for a handheld device (1) for contactless measurement of vital parameters according to one of claims 1 to 7, comprising the following steps: - Recording of incident light through an optical tube (13) of the handset (1), - Projecting the light onto a first, a second and a third image sensor (17a, 17b, 17c), wherein the second image sensor (17b) receives only light of a first wavelength range and the third image sensor (17c) receives only light of a second wavelength range different from the first wavelength range, each of which lies in the near infrared, - recording a first image data set by means of the first image sensor (17a), a second image data set by means of the second image sensor (17b) and a third image data set by means of the third image sensor (17c), - Determining vital parameters by an evaluation unit (19) located in the handheld device (1) based on the first, second and third image data sets. [9] Method according to claim 8, wherein the evaluation unit (19) takes into account brightness signals output by a brightness sensor (23) attached to the handset (1) when determining the vital parameters. [10] Method according to claim 8 or 9, wherein the evaluation unit (19) takes into account movement signals output by a movement sensor (25) located on the hand-held device (1) when determining the vital parameters. [11] Method according to one of claims 8 to 10, wherein the evaluation unit (19) carries out an accuracy check when determining the vital parameters and rejects the measurement if the accuracy of the determination of the vital parameters falls below a reference value. [12] Method according to one of claims 8 to 11, wherein the evaluation unit (19) comprises a memory (29) and the vital parameters are assigned to an individual data set which is stored in the memory (29). [13] Method according to claim 12, wherein the individual data set is selected using a barcode scanner (27) located on the handheld device (1) or using facial recognition performed on the basis of the image data. [14] Method according to one of claims 8 to 13, wherein the image data comprise a plurality of image sequences (35a, 35b, 35c) from which a pulse wave signal (41a) is derived, from which the vital parameters are determined.

Images