Image display device with adaptive brightness based on physiological variable

Abstract

The invention relates to a device (2) for displaying images, having an adaptive brightness, comprising: an eyepiece intended to accommodate at least one eye (4) of a user; and a digital display (6) in the visible light range, positioned behind the eyepiece, the display device comprising: an illuminator (8) intended to illuminate an eye (4) of a user housed by the eyepiece; an image sensor (10) intended to collect an image of a user's eye accommodated by the eyepiece; and a system (12) for automatically controlling the brightness of the digital display (6) based on at least one physiological variable induced by an image of the user's eye (4).

Description

Technical Field

[0001] The present invention relates to an information viewing and viewing device.

[0002] In particular, the present invention relates to immersive viewing devices and display apparatuses intended for use with the eyes of a user positioned proximate to the device. More particularly, the field relates to light adaptation of display screens of immersive viewing devices.

[0003] Generally, the present invention applies to any display screen that can vary light intensity to enhance user comfort and immersion. Technical Background

[0004] Some human-machine interfaces include a digital screen and / or projector that can be viewed by the user's eyes in close proximity to the screen, such as through eyepieces and / or goggles that allow viewing in a dark environment around the eyes. These human-machine interfaces can be found, for example, in binoculars for short or long vision or viewing ranges, in digital cameras, in augmented reality masks, or in any other type of viewfinder or display that includes a digital screen and / or projector.

[0005] Adjusting the brightness of such digital displays is important because the user may be required to be in a bright outdoor environment (e.g., in broad daylight) or, conversely, in a very dark outdoor environment (e.g., in the dark of night) while still needing to view the digital display through the eyepiece from time to time. The relative brightness of the screen relative to the external environment then needs to be adjusted so that the user's vision is not disturbed when the user's line of sight switches from viewing the external environment to viewing the screen (or vice versa). In other words, the switch needs to be made in such a way that the user is not dazzled, or conversely can see the displayed information and / or the external environment when the external environment is dark.

[0006] There are systems that automatically control the light intensity (in other words the brightness) of digital screens, for example of mobile phones. Known systems contain, for example, a sensor for the brightness of the external environment, which can be used to adapt the brightness of the screen based on the external brightness.

[0007] However, brightness sensors can be hidden or covered, thus returning erroneous information about the brightness of the external environment. Furthermore, the orientation and / or location of the brightness sensor may not necessarily correspond to the user's gaze direction; this can be particularly true for mid-range and / or long-range aiming systems. Furthermore, the system can acquire brightness information from specific dark areas while the rest of the scene is brightly lit.

[0008] Brightness sensors can also suffer from saturation effects or sensitivity limitations in very bright or dim external environments. Summary of the Invention

[0009] It is therefore an object of the present invention to overcome the above-mentioned disadvantages and to provide a display device with reliable and improved brightness adaptation.

[0010] The object of the present invention is an image display device with adaptive brightness, comprising: an eyepiece intended to accommodate at least one eye of a user; and a digital display in the visible light range, positioned behind the eyepiece, the device comprising: an illuminator intended to illuminate the user's eye accommodated by the eyepiece; an image sensor intended to collect an image of the user's eye accommodated by the eyepiece; and a system for automatically controlling the brightness of the digital display based on at least one physiological variable triggered by the image of the user's eye.

[0011] For example, the illuminator operates in the visible or infrared range.

[0012] Therefore, the display device can adjust the brightness of the digital display based on the user's physiological indicators to meet the user's expectations as much as possible. The display device can also smoothly switch the brightness according to the process of physiological variables to reduce visual fatigue and stress.

[0013] In one embodiment, the automatic control system includes: a module for detecting and measuring the pupil diameter of the user's eye on an image of the user's eye; and / or a module for detecting the user's condition and stress level, the module for detecting the user's condition and stress level including a module for restoring the user's temperature and / or a module for restoring the user's heart rate, at least one physiological variable being selected from the group consisting of the pupil diameter of the eye, the user's temperature, and the user's heart rate.

[0014] In a particular embodiment, the device includes an ambient brightness sensor and / or a lighting environment sensor and / or a user control interface, so that the system for automatically controlling the brightness of the digital display also uses the ambient brightness and / or lighting environment and / or user control to automatically control the brightness of the digital display.

[0015] Advantageously, the illuminator and the image sensor operate in the near infrared spectral range.

[0016] Advantageously, the device comprises an optical system comprising a set of focusing lenses and / or collimating lenses for directing light from the digital display and from the illuminator to the user's eye and from the user's eye to the image sensor, the optical system also comprising a dichroic filter.

[0017] Advantageously, the optical system comprises a free-form optical element comprising the dichroic filter and the set of lenses.

[0018] In one embodiment, the device includes a user eye proximity sensor for operating the image display device when a user's eye is in proximity.

[0019] Advantageously, the automatic brightness control system further includes a chromaticity control module.

[0020] Advantageously, the automatic control system comprises means for controlling operating parameters of the illuminator and operating parameters of the image sensor based on images acquired from the user's eyes.

[0021] Another object of the invention is a method for adapting the brightness of a display device as defined herein before, comprising the following steps:

[0022] - obtaining an image of the user's eyes;

[0023] - measuring an initial pupil diameter of the user's eye based on the image of the user's eye and defining an initial brightness set point for the digital display;

[0024] -Adjust the brightness of the digital display using the initial brightness set point;

[0025] - obtaining a series of images of the user's eyes;

[0026] - measuring the pupil diameter of the user's eye for each image in the series of images acquired; and

[0027] - Adjusting the brightness of a digital display based on a process of measuring the pupil diameter of a user's eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Other objects, characteristics and advantages of the present invention will become apparent on reading the following description given solely as a non-limiting example and with reference to the accompanying drawings, in which:

[0029] [ Figure 1 ] is a schematic diagram of an image display device with adaptive brightness according to the present invention;

[0030] [ Figure 2 ] is a schematic diagram of an optical system of an image display device with adaptive brightness according to the present invention;

[0031] [ Figure 3 ]yes Figure 2 Schematic diagram of a first alternative optical system;

[0032] [ Figure 4 ]yes Figure 2Schematic diagram of a second alternative optical system;

[0033] [ Figure 5 ]yes Figure 2 Schematic diagram of a third alternative optical system;

[0034] [ Figure 6 ]and

[0035] [ Figure 7 ]and

[0036] [ Figure 8 ]and

[0037] [ Figure 9 ] is a graph of brightness and a graph of pupil diameter during operation of a display device according to the present invention; and

[0038] [ Figure 10 ] is a schematic diagram of a brightness adjustment method for a display device according to the present invention. DETAILED DESCRIPTION

[0039] Figure 1 An image display device 2 with adaptive brightness is schematically shown.

[0040] Display device 2 is intended to provide an immersive display for the user. In other words, the user's eyes 4 are intended to be close to display device 2; the display device is of the "near-eye" type. In particular, display device 2 is a photographic device, binoculars, a monocular, a periscope, a virtual reality / augmented reality mask, or any other type of viewfinder or display.

[0041] The display device 2 includes an eyepiece (not shown) designed to accommodate one or both eyes of the user. The eyepiece is, for example, a piece of plastic or silicone, or a simple position where the user's eyes must be positioned in order to view the display device. The purpose of the eyepiece is to prevent viewing of the display device from being affected by the brightness of the external environment.

[0042] Optionally, the display device 2 comprises a proximity sensor (not shown) for the user's eye 4. Thus, the display device 2 is only operational when the user's eye 4 is positioned close to the display device 2, preferably against the eyepiece.

[0043] The display device 2 further comprises a digital display 6 , an illuminator 8 , an image sensor 10 and an automatic control system 12 for the brightness of the digital display 6 .

[0044] The digital display 6 emits radiation 14 in the visible range and is positioned behind the eyepiece so that when the user's eye 4 is positioned against the eyepiece, the user's eye 4 can see the digital display 6 inside the display device 2. For example, the digital display 6 is a microdisplay for displaying and transmitting information to the user. The microdisplay is, for example, a screen or projector using OLED technology. The display device 2 also includes a module 16 for inputting a video signal from a video source, which is located externally and / or internally to the input module 16. The input module 16 is connected to the digital display 6 to display an image or video of the video signal to the user. A video signal also refers to any visual information, such as symbols and / or text.

[0045] When the eyepiece receives the user's eye 4, the illuminator 8 is intended to illuminate the user's eye 4. The illuminator 8 operates by emitting radiation 18 in the infrared spectral range, in particular in near infrared wavelengths, for example between 0.7 micrometer and 1.6 micrometer wavelengths.

[0046] Image sensor 10 is designed to collect an image of the user's eye 4 when the eye 4 is placed against the eyepiece and illuminated by illuminator 8. Image sensor 10 operates at least at the same wavelength as illuminator 8. Therefore, when illuminator 8 illuminates the user's eye 4 with infrared radiation 18, the user is not dazzled, and image sensor 10 recovers an infrared image of the user's eye 4 by means of the illumination performed. In addition, the eyepiece prevents image sensor 10 from being able to collect light from the external environment.

[0047] The automatic control system 12 makes it possible to automatically adjust the brightness of the digital display 6 based on at least one physiological variable, preferably at least one physiological variable induced by an image of the user's eye. For example, the physiological variable is the pupil diameter of the user's eye 4, which is a parameter that varies according to the brightness entering the eye.

[0048] The automatic control system 12 includes an image sensing module 20 and a module 22 for automatically analyzing the quality of an image output from the image sensing module 20 .

[0049] The image sensing module 20 allows, in conjunction with the image sensor 10 , to acquire and optionally record an image of the user's eye 4 .

[0050] In particular, the analysis module 22 enables checking whether the image quality of the eye 4 is sufficient for measuring the pupil diameter, for example.

[0051] To correct the quality of the image output from the image sensor 10, the automatic control system 12 includes a compensation strategy module 24 and a module 26 for controlling the operating parameters of the illuminator 8 and the operating parameters of the image sensor 10. The compensation strategy module 24 automatically selects which parameters of the illuminator 8 and the image sensor 10 to modify based on the quality of the image acquired from the user's eye 4. To modify both the illumination of the illuminator 8 and the acquisition parameters of the image sensor 10, the parameters to be modified are sent to the control module 26 for the operating parameters of the illuminator 8 and the image sensor 10. For example, the control module 26 may increase the brightness of the illuminator 8 and reduce the integration time of the image sensor 10 to reduce noise in the image of the eye 4.

[0052] exist Figure 1 In the embodiment shown, the automatic control system 12 comprises a module 28 for detecting and measuring the pupil diameter of the user's eye 4 on the image of the user's eye 4 acquired by the image sensor 10 .

[0053] Alternatively or additionally, the automatic control system 12 includes a module 30 for detecting the user's condition and stress level. The module 30 for detecting the user's condition and stress level may include, for example, a module for recovering the user's temperature from a temperature sensor (e.g., included in a module 31 external to the display device 2), and / or the module for detecting the user's condition and stress level may include a module for recovering the user's heart rate from a heart rate sensor (e.g., included in a module 31 external to the display device 2). The module 30 may also receive as input parameters of fluctuations in the pupil diameter of the eye 4 caused by the detection and measurement module 28, and these parameters may also be stress indicators. Therefore, in a stress state measured by the heart rate, it may be necessary to provide the user with information that is more strongly highlighted by the digital display 6 with greater brightness.

[0054] Thus, the automatic control system 12 uses one or more physiological variables, such as pupil diameter, temperature, and the user's heart rate, to automatically adjust the brightness of the digital display 6 .

[0055] The automatic control system 12 also comprises a module for applying theoretical heuristics based on physiological variables 32. This module for applying theoretical heuristics 32 makes it possible to define, for example using tables or computer learning, the brightness laws applicable as a function of the value of the physiological variables.

[0056] Optionally, the display device 2 includes a user control interface 34 to enable the user to manually give instructions (e.g., instructions for brightness). The automatic control system 12 also includes a module 36 for collecting user commands generated from the control interface 34. The collection module 36 also includes a memory that includes a history of user actions and conditions. Thus, the automatic control system 12 can use user commands and / or conditions to automatically control the brightness of the digital display 6. In addition, the automatic control system can also use the history of these commands and / or conditions to determine possible automatic control of the brightness of the digital display 6, for example, through computer learning.

[0057] Optionally, the display device 2 includes an ambient brightness sensor 38 (in other words, an external ambient brightness sensor) and / or a lighting environment sensor 40. The ambient brightness sensor 38 may include a light dependent resistor (LDR) or an image sensor in the visible light range, as well as calibration and corresponding data between integration time and ambient brightness, so as to establish classification according to the scene type of the external environment, for example.

[0058] In particular, the lighting environment sensor 40 enables measurement of the chromaticity of the environment outside the display device 2. The lighting environment sensor 40 comprises, for example, a plurality of detection elements that are sensitive to different wavelengths.

[0059] The automatic control system 12 further comprises a module 42 for detecting changes in ambient brightness and / or ambiance. In an alternative to the display device 2 comprising a module 36 for collecting user commands, the module 42 for detecting changes in ambient brightness and / or ambiance makes it possible, for example, to link the history of ambient brightness and / or ambiance to the history of the user's actions and conditions in order to optimize the brightness of the display.

[0060] The automatic control system 12 includes a brightness adaptation strategy module 44, a module 46 for controlling the brightness of the digital display 6, and optionally a module 48 for controlling the chrominance of the digital display 6. The strategy module 44 compiles the parameters to be taken into account to modify the brightness and / or chrominance of the digital display 6 and sends brightness instructions to the brightness control module 46, which affects the brightness of the digital display 6, and optionally to the chrominance control module 48, so that the chrominance control module can modify the chrominance of the video signal leaving the input module 16 and send this corrected video signal to the digital display 6.

[0061] The user then closes the automatic control loop and in response to the brightness changes on the digital display 6 the user modifies the physiological variables which are in turn taken into account in the automatic control system 12 .

[0062] Display device 2 also includes an optical system 50 that includes a set of focusing and / or collimating lenses 52 for directing visible light from digital display 6 to user's eye 4, for directing infrared light from illuminator 8 to user's eye 4, and for directing infrared light from user's eye 4 to image sensor 10. Optionally, optical system 50 includes a dichroic filter 54.

[0063] Figure 2 An optical system 50 between the user's eye 4 and the digital display 6, the image sensor 10 and the illuminator 8 has been schematically shown in FIG.

[0064] In this embodiment, the dichroic filter 54 reflects infrared radiation 18 and transmits visible radiation 14. Thus, the digital display 6 is positioned on one side of the dichroic filter 54, while the illuminator 8 and image sensor 10 are on the other side. This embodiment shows the user's direct vision of the digital display 6.

[0065] Figure 3 A first alternative for an optical system 50 between the user's eye 4 and the digital display 6 , image sensor 10 and illuminator 8 has been schematically shown in .

[0066] In this embodiment, the dichroic filter 54 reflects 50% of the infrared radiation and transmits 50% of the visible radiation 14 and the infrared radiation 18. The digital display 6 and the illuminator 8 are positioned on one side of the dichroic filter, while the image sensor 10 is positioned on the other side. This embodiment shows the user's direct vision of the digital display 6, and more generally of the radiation emitting function.

[0067] exist Figure 2 and Figure 3 In the embodiment shown, the arrangement of the digital display 6, image sensor 10, and illuminator 8 on one side and the dichroic filter 54 on the other side could be completely reversed based on the transmissive and reflective properties of the dichroic filter.

[0068] Figure 4 A second alternative of an optical system 50 between the user's eye 4 and the digital display 6, image sensor 10 and illuminator 8 has been schematically shown in FIG. In this alternative, these three functions are performed by the same bidirectional component 56. Bidirectional means that the bidirectional component 56 performs both an image acquisition function using the image sensor 10 and a transmission function using the digital display 6 and illuminator 8.

[0069] In this embodiment, the optical system includes a semi-reflective plate 58, a first dichroic filter 60 that transmits infrared light and reflects visible light, thereby enabling the display 62 of the digital display to be collimated and superimposed on the visual image of the external environment, and a second dichroic filter 66 that transmits visible light and reflects infrared light to focus the infrared radiation on the bidirectional component 56 after the eye 4 has been illuminated. This embodiment is suitable for manufacturing an augmented reality device. In this embodiment, the illuminator 8 is optional, as ambient near-infrared radiation (in other words, radiation from the external environment) can be used to illuminate the eye instead of or in addition to the illuminator 8.

[0070] Figure 5 A third alternative for an optical system 50 between the user's eye 4 and the digital display 6 , image sensor 10 and illuminator 8 has been schematically shown in .

[0071] In this alternative, the optical system 50 includes a free-form optical element 68 (in other words, one or more surfaces of the free-form optical element can be free-form) that has the functionality of a dichroic filter and a set of lenses. This embodiment is also suitable for manufacturing augmented reality devices. In particular, the optical element 68 includes a first inner surface 70 and a second surface 72, on which the light generated by the illuminator 8 and the digital display 6 is reflected after entering the optical element 68, and the light is also reflected on the second surface 72 before passing through the first surface 70 toward the eye 4, which has a Figure 4 The third alternative is to use a perception such as that in which the display 62 of the digital display 6 is superimposed on the visual image of the external environment and / or the visual image of another element (e.g., a panel or a second display). The infrared radiation 18 then propagates in a direction opposite to the image sensor 10. In this embodiment, the first surface 70 behaves differently depending on the inclination of the light irradiating it.

[0072] The brightness curve and pupil diameter curve of the display device 2 during operation as a function of time t are already shown in FIG. Figure 6 、 Figure 7 、 Figure 8 and Figure 9 Shown in.

[0073] Figure 10 The steps of the method of adapting the brightness of the display device 2 have been schematically shown. Figure 10 The implementation method of Figures 6 to 9 The process of the curve graph has an impact.

[0074] After the user's eye 4 has been positioned facing the display device, a step 74 of acquiring an image of the user's eye 4 is performed. This step 74 is performed using the image sensor 10.

[0075] Then, step 75 is performed to measure the initial pupil diameter of the user's eye 4 based on the image of the user's eye 4, and to define an initial brightness set point for the digital display 6. The initial diameter is represented by DI. The initial set point is predetermined or calculated based on this initial diameter. Steps 74 and 75 can also be performed by acquiring several images of the user's eye 4 while the user faces the display device 2.

[0076] Then a step 76 of adjusting the brightness of the digital display 6 using the initial brightness set point is performed. The brightness of the digital display 6 is represented by La.

[0077] Then, if the brightness of the external environment represented by Le is different, the brightness La of the digital display 6 affects the size of the pupil of the user's eye 4.

[0078] Then, the brightness adaptation method includes repeating steps 74 to 76, acquiring a series of images of the user's eye 4, measuring the pupil diameter D of the user's eye 4 for each image in the acquired series of images, and adjusting the brightness La of the digital display 6 based on the measurement process of the pupil diameter D of the user's eye 4.

[0079] exist Figure 6 In the figure, the brightness Le and La are equal, which is why the diameter D varies very little around DI (with a tolerance range of plus or minus M). The tolerance range of plus or minus M is particularly likely to filter out small changes in brightness and avoid discomfort to the user.

[0080] exist Figure 7 In the example, the user brings his / her eye 4 into contact with the eyepiece, and after the first three steps, the brightness La is greater than the brightness Le. Therefore, the pupil of the eye 4 closes until the display device stabilizes its brightness La at the external brightness Le by studying the changes in the user's pupil diameter D.

[0081] exist Figure 8 In the example, the user brings his / her eye into contact with the eyepiece, and after the first three steps, the brightness La is less than the brightness Le. Therefore, the pupil of the eye 4 opens until the display device stabilizes its brightness La at the external brightness Le by studying the changes in the user's pupil diameter D.

[0082] exist Figure 9, an embodiment has been shown in which the initial diameter DI of the initial brightness setting point is modified based on the brightness of the external environment Le. This embodiment mode is a popular mode and enables the use of the external brightness of the user's eyes and the pupil diameter together to adapt the brightness La of the digital display 6 as much as possible. This is a post hoc adaptation in which, if the brightness Le changes after the eye 4 is positioned, the display device 2 changes its brightness La to prepare the eye 4 for a higher or lower brightness Le when the eye moves away from the display device 2, without causing the eye to be dazzled or unable to see anything.

[0083] More generally, a display device and a brightness adaptation method associated therewith enable the brightness of a digital display to be kept consistent with the brightness of an external environment, so that a user can switch from one brightness to another without experiencing visual fatigue.

Claims

1. An image display device (2), the image display device having adaptive brightness, the image display device comprising: an eyepiece intended to accommodate at least one eye (4) of a user; and a digital display (6) in the visible light range, the digital display being positioned behind the eyepiece, characterized in that the image display device comprises: an illuminator (8) intended to illuminate the user's eye (4) accommodated by the eyepiece; an image sensor (10) intended to collect an image of the user's eye accommodated by the eyepiece; and a system (12) for automatically controlling the brightness of the digital display (6) based on at least one physiological variable induced by the image of the user's eye (4).

2. The device according to claim 1, wherein: The automatic control system (12) comprises: a module (28) for detecting and measuring the pupil diameter of the user's eye (4) on an image of the user's eye; and / or a module (30) for detecting the user's condition and stress level, the module for detecting the user's condition and stress level comprising a module for restoring the user's temperature and / or a module for restoring the user's heart rate, the at least one physiological variable being selected from the pupil diameter of the eye, the user's temperature and the user's heart rate.

3. A device according to one of claims 1 and 2, comprising an ambient brightness sensor (38) and / or a lighting environment sensor (40) and / or a user control interface (34), so that the system (12) for automatically controlling the brightness of the digital display (6) also uses the ambient brightness and / or lighting environment and / or a user control to automatically control the brightness of the digital display (6).

4. The device according to any one of claims 1 and 3, wherein: The illuminator (8) and the image sensor (10) operate in the near infrared spectral range.

5. The device according to any one of claims 1 to 4, further comprising an optical system (50), the optical system comprising: a set of focusing lenses and / or collimating lenses (52) for directing light from the digital display (6) and from the illuminator (8) to the user's eye (4) and directing light from the user's eye (4) to the image sensor (10), the optical system (50) also comprising a dichroic filter (54).

6. The device according to claim 5, wherein: The optical system (50) includes a free-form optical element (68) including the dichroic filter and the set of lenses.

7. The device according to any one of claims 1 to 6, comprising a user eye proximity sensor, wherein the user eye proximity sensor is used to operate the image display device (2) when the user's eye (4) is close.

8. The apparatus according to any one of claims 1 to 7, wherein: The automatic brightness control system (12) also includes a chromaticity control module (48).

9. The device according to any one of claims 1 to 8, wherein: The automatic control system (12) includes a module (26) for controlling operating parameters of the illuminator (8) and operating parameters of the image sensor (10) based on the image acquired from the user's eye (4).

10. A method for adapting the brightness of a display device (2) according to any one of claims 1 to 9, characterized in that: The method comprises the following steps: - acquiring an image of the user's eyes (step 74); - measuring an initial pupil diameter of the user's eye based on the image of the user's eye and defining an initial brightness set point for the digital display (step 75); - adjusting the brightness of the digital display (6) using the initial brightness set point (step 76); - acquiring a series of images of the user's eyes (step 74); - measuring the pupil diameter of the user's eye for each image in the acquired series of images (step 75); and - Adjusting the brightness of the digital display based on a process of measuring the pupil diameter of the user's eye (step 76).

Images