Image display device having physiological variable-based adaptive brightness

Abstract

The invention relates to a device (2) for displaying images having adaptive brightness, comprising an eyepiece intended to accommodate at least one eye (4) of a user, and a digital display (4) in the visible range positioned behind the eyepiece, the display device comprising an illuminator (8) intended to illuminate the eye (4) of the user accommodated by the eyepiece, an image sensor (10) intended to collect an image of the eye of the user accommodated by the eyepiece, and a system (12) for the automatic control of the brightness of the digital display (6) on the basis of at least one physiological variable resulting from the image of the eye (4) of the user.

Description

[0001] DESCRIPTION

[0002] TITLE: Image display device with adaptive luminance based on a physiological variable

[0003] Technical field

[0004] The present invention relates to information viewing and observation equipment.

[0005] In particular, the present invention relates to observation equipment and immersive display devices intended to be used with a positioning of the eye of a user in proximity to said equipment. The field relates more particularly to the light adaptation of a display screen of immersive observation equipment.

[0006] Generally speaking, the invention applies to any display screen whose light intensity can be modified to increase the comfort and immersion of the user.

[0007] Previous techniques

[0008] Some human-machine interfaces include a digital screen, and / or a projector, observable by a user's eye positioned in the immediate vicinity of said screen, for example through a hood-type eyepiece and / or an eyepiece allowing observation in a dark environment around the eye. These human-machine interfaces are found, for example, in short or long-range vision or observation binoculars, in a digital camera, an augmented reality mask or any other type of viewfinder or display comprising a digital screen and / or a projector.

[0009] Adjusting the luminance of such digital screens is important because the user may be in a bright outdoor environment, for example in broad daylight, or conversely in a very dark environment, for example in the middle of the night, while being intended to look at the digital screen through the eyepiece from time to time. The relative luminance of the screen compared to the outdoor environment must then be adapted so as not to disturb the user's vision during a transition of his gaze from observing the outdoor environment to observing the screen, and vice versa. In other words, the transition must be able to take place in such a way as not to dazzle the user or conversely so that he can see the information displayed and / or the outdoor environment when it is dark.

[0010] There are systems that control the light intensity, or luminance, of a digital screen, such as a mobile phone. Known systems, for example, contain a luminance sensor for the external environment that can adjust the luminance of the screen according to the external luminance.

[0011] However, a luminance sensor may be masked and covered so that it returns erroneous information about the luminance of the external environment. In addition, a luminance sensor has an orientation and / or positioning that does not necessarily correspond to the user's gaze direction: this may be the case in particular for medium and / or long-range aiming systems. In addition, systems could acquire luminance information from a particular dark area while the rest of the scene is very bright.

[0012] The luminance sensor may also be subject to saturation or sensitivity limitation effects in very bright or dimly lit outdoor environments.

[0013] Statement of the invention

[0014] The present invention therefore aims to overcome the aforementioned drawbacks and to provide a display device with reliable and improved adaptation of the luminance.

[0015] The present invention relates to an image display device with adaptive luminance comprising an eyepiece intended to accommodate at least one eye of a user, and a digital display in the visible range positioned behind the eyepiece, the device comprising an illuminator intended to illuminate the eye of the user accommodated by the eyepiece, an image sensor intended to collect an image of the eye of the user accommodated by the eyepiece, and a system for controlling the luminance of the digital display as a function of at least one physiological variable originating from the image of the eye of the user.

[0016] The illuminator works, for example, in the visible or infrared range.

[0017] Thus, the display device makes it possible to adapt the luminance of the digital display according to the user's physiological indicators and thus to be as close as possible to the user's expectations. The display device also makes it possible to achieve smooth luminance transitions according to the evolution of physiological variables in order to reduce visual fatigue and stress.

[0018] In one embodiment, the servo-control system comprises a module for detecting and measuring the diameter of the pupil of the user's eye on an image of the user's eye, and / or a module for detecting the state and stress level of the user comprising a module for recovering the temperature of the user and / or a module for recovering the heart rate of the user, the at least one physiological variable being chosen from the diameter of the pupil of the eye, the temperature, and the heart rate of the user.

[0019] In a particular embodiment, the device comprises an ambient luminance sensor and / or a light ambiance sensor and / or a user control interface so that the system for controlling the luminance of the digital display also uses the ambient luminance and / or the light ambiance and / or a user control to control said luminance of the digital display.

[0020] Advantageously, the illuminator and image sensor operate in the near infrared spectral range.

[0021] Advantageously, the device comprises an optical system comprising a set of focusing and / or collimating lenses for guiding light from the digital display and from the illuminator to the user's eye and from the user's eye to the image sensor, the optical system also comprising a dichroic filter.

[0022] Advantageously, the optical system comprises a free-form optical element comprising the dichroic filter and the lens assembly.

[0023] In one embodiment, the device includes a user eye proximity sensor for operating the image display device when the user's eye is in proximity.

[0024] Advantageously, the luminance control system further comprises a colorimetry control module.

[0025] Advantageously, the servo system comprises a module for controlling the operating parameters of the illuminator and the image sensor as a function of the image acquired from the user's eye.

[0026] The present invention also relates to a method for adapting the luminance of a display device as defined previously, the method comprising the following steps:

[0027] - Acquisition of an image of the user's eye;

[0028] - Measurement of the initial diameter of the pupil of the user's eye from the image of the user's eye and definition of an initial luminance setpoint for the digital display;

[0029] - Adjustment of the luminance of the digital display with the initial luminance setting;

[0030] - Acquisition of a series of images of the user's eye;

[0031] - Measurement of the diameter of the pupil of the user's eye for each image in the acquired image series; and

[0032] - Adjustment of the luminance of the digital display according to the evolution of the measurement of the diameter of the pupil of the user's eye.

[0033] Brief Description of the Drawings Other aims, characteristics and advantages of the invention will appear on reading the following description, given solely by way of non-limiting example, and made with reference to the appended drawings in which:

[0034] [Fig 1] is a schematic view of an adaptive luminance image display device according to the invention;

[0035] [Fig 2] is a schematic view of an optical system of an adaptive luminance image display device according to the invention;

[0036] [Fig 3] is a schematic view of a first variant of the optical system of Figure 2;

[0037] [Fig 4] is a schematic view of a second variant of the optical system of Figure 2;

[0038] [Fig 5] is a schematic view of a third variant of the optical system of Figure 2;

[0039] [Fig 6] and

[0040] [Fig 7] and

[0041] [Fig 8] and

[0042] [Fig 9] are luminance and pupil diameter profiles during operation of the display device according to the invention; and

[0043] [Fig 10] is a schematic representation of a method for adapting the luminance of a display device according to the invention.

[0044] Detailed description of at least one embodiment

[0045] Figure 1 schematically shows an image display device 2 with adaptive luminance.

[0046] The display device 2 is intended to offer a user an immersive display. In other words, the user's eye 4 is intended to be held close to the display device 2 which is of the "near-eye" type. In particular, the display device 2 is a photographic device, or binoculars, or a monocular, or a periscope, or a virtual / augmented reality mask or any other type of viewfinder or display.

[0047] The display device 2 comprises a cup-type eyecup (not shown) intended to accommodate the user's eye or eyes. The eyecup is, for example, a piece of plastic or silicone, or even a simple location on which the user's eye must position itself to observe in the display device. The role of the eyecup is to allow observation in the display device isolated from the luminance of the external environment.

[0048] The display device 2 optionally comprises a proximity sensor (not shown) for the user's eye 4. Thus, the display device 2 is only put into operation when the user's eye 4 is positioned close to the display device 2, preferably positioned against the eyepiece.

[0049] The display device 2 also comprises a digital display 6, an illuminator 8, an image sensor 10 and a system 12 for controlling the luminance of the digital display 6.

[0050] The digital display 6 emits radiation 14 in the visible range and is positioned behind the eyepiece, so that the user's eye 4 sees the digital display 6 inside the display device 2 when his eye 4 is positioned against the eyepiece. For example, the digital display 6 is a micro-display for displaying and transmitting information to the user. It is, for example, a screen, or a projector, for example of OLED technology. The display device 2 also comprises an input module 16 for a video signal from a video source external and / or internal to the input module 16, the input module 16 being connected to the digital display 6 for displaying an image or a video of the video signal to the user. Video signal is also understood to mean any visual information such as symbols and / or text.The illuminator 8 is intended to illuminate the user's eye 4 when the latter is received by the eyepiece. The illuminator 8 operates by emitting radiation 18 in the infrared spectral range, in particular in the near infrared wavelengths, for example between 0.7 and 1.6 micrometers in wavelength.

[0051] The image sensor 10 is intended to collect an image of the user's eye 4 when the eye 4 is placed against the eyecup and illuminated by the illuminator 8. The image sensor 10 operates at least for the same wavelengths as the illuminator 8. Thus, when the illuminator 8 illuminates the user's eye 4 with infrared radiation 18, the user is not dazzled and the image sensor 10 recovers an infrared image of the user's eye 4 thanks to the illumination carried out. In addition, the eyecup prevents the image sensor 10 from being able to capture light from the external environment.

[0052] The servo system 12 allows the automatic adjustment of the luminance of the digital display 6 as a function of at least one physiological variable, preferably derived from the image of the user's eye. The physiological variable is for example the diameter of the pupil of the user's eye 4, the diameter of the pupil of an eye being a parameter varying as a function of the luminance entering said eye.

[0053] The servo system 12 comprises an image capture module 20 and an automatic analysis module 22 of the quality of the image output from the image capture module 20.

[0054] The image capture module 20 allows the acquisition and optionally the recording of an image of the user's eye 4 in conjunction with the image sensor 10.

[0055] The analysis module 22 makes it possible in particular to verify that the image quality of the eye 4 is sufficient to, for example, be able to measure the diameter of the pupil.

[0056] To correct the image quality at the output of the image sensor 10, the servo system 12 comprises a compensation strategy module 24 and a control module 26 for the operating parameters of the illuminator 8 and the image sensor 10. The compensation strategy module 24 automatically chooses the parameters of the illuminator 8 and the image sensor 10 to be modified according to the quality of the image acquired from the user's eye 4. The parameters to be modified are transmitted to the control module 26 for the operating parameters of the illuminator 8 and the image sensor 10 in order to modify the illumination of the illuminator 8 as well as the acquisition parameters of the image sensor 10. For example, the control module 26 can increase the luminance of the illuminator 8 and decrease the integration time of the image sensor 10 in order to reduce the noise of the image of the eye 4.

[0057] In the embodiment illustrated in FIG. 1, the servo-control system 12 comprises a module 28 for detecting and measuring the diameter of the pupil of the user's eye 4 on an image of the user's eye 4 acquired by the image sensor 10.

[0058] Alternatively or additionally, the servo-control system 12 comprises a module 30 for detecting the state and stress level of the user. The module 30 for detecting the state and stress level of the user comprises, for example, a module for recovering the temperature of the user from a temperature sensor, for example included in an external module 31 to the display device 2, and / or comprises a module for recovering the heart rate of the user from a heart rate sensor, for example included in the external module 31 to the display device 2. The module 30 can also receive as input parameters of fluctuations in the diameter of the pupil of the eye 4, from the detection and measurement module 28, these parameters also being able to be an indicator of stress.Thus, in a stressed state measured by heart rate, the user may need to be provided with information highlighted more strongly by the digital display 6 with greater luminance.

[0059] The luminance control system 12 thus uses one or more physiological variables among the diameter of the pupil, the temperature, and the heart rate of the user to adapt the luminance of the digital display 6 automatically. The control system 12 further comprises a module for applying theoretical heuristics 32 as a function of physiological variables. This theoretical heuristics application module 32 makes it possible to define, for example using tables or computer learning, luminance laws applicable according to the values ​​of the physiological variables.

[0060] Optionally, the display device 2 comprises a user control interface 34 so as to allow the user to give instructions, for example luminance, manually. The control system 12 further comprises a module 36 for collecting user commands from the control interface 34. This collection module 36 also comprises a memory comprising the history of the user's actions and states. Thus, the control system 12 can use a command and / or a state of the user to control the luminance of the digital display 6. In addition, it can also use the history of these commands and / or states to determine, for example by computer learning, a possible control of the luminance of the digital display 6.

[0061] The display device 2 optionally comprises an ambient luminance sensor 38, in other words a luminance sensor of the external environment, and / or a light atmosphere sensor 40. The ambient luminance sensor 38 may comprise an LDR (“Light Dependent Resistor” in English terms) or comprise an image sensor in the visible domain and calibration and correspondence data between an integration time and the ambient luminance, for example to establish a classification by type of scene of the external environment.

[0062] The ambient light sensor 40 makes it possible in particular to measure the colorimetry of the environment outside the display device 2. The ambient light sensor 40 comprises, for example, a plurality of detection elements sensitive to distinct wavelengths. The servo-control system 12 further comprises a module 42 for detecting variations in the ambient luminance and / or the ambience. In a variant of the display device 2 comprising a module 36 for collecting user commands, the module 42 for detecting variations in the ambient luminance and / or the ambience makes it possible, for example, to link a history of ambient luminance and / or ambience to the history of the user's actions and states in order to optimize the luminance of the display.

[0063] The servo system 12 comprises a luminance adaptation strategy module 44, a luminance control module 46 of the digital display 6, and optionally a colorimetry control module 48 of the digital display 6. This strategy module 44 compiles the parameters to be taken into account to modify the luminance and / or the colorimetry of the digital display 6 and addresses luminance instructions to the luminance control module 46 which influences the luminance of the digital display 6, and optionally to the colorimetry control module 48, which can thus modify the colorimetry of the video signal leaving the input module 16 and transmit this corrected video signal to the digital display 6.

[0064] The control loop is then closed by the user who, in response to a variation in the luminance of the digital display 6, modifies physiological variables, in turn taken into account in the control system 12.

[0065] The display device 2 further comprises an optical system 50 comprising a set of focusing and / or collimating lenses 52 for guiding visible light from the digital display 6 to the user's eye 4, for guiding infrared light from the illuminator 8 to the user's eye 4 and from the user's eye 4 to the image sensor 10. Optionally, the optical system 50 comprises a dichroic filter 54.

[0066] 2 schematically shows an optical system 50 between a user's eye 4 and a digital display 6, an image sensor 10 and an illuminator 8. In this embodiment, the dichroic filter 54 reflects the infrared radiation 18 and transmits the visible radiation 14. The digital display 6 is thus positioned on one side of the dichroic filter 54 while the illuminator 8 and the image sensor 10 are on the other side. This embodiment illustrates a direct view of the user on the digital display 6.

[0067] Figure 3 schematically shows a first variant of an optical system 50 between a user's eye 4 and the digital display 6, the image sensor 10 and the illuminator 8.

[0068] In this embodiment, the dichroic filter 54 reflects 50% of the infrared radiation and transmits the visible radiation 14 as well as 50% of the infrared radiation 18. The digital display 6 and the illuminator 8 are positioned on one side of the dichroic filter while the image sensor 10 is on the other side. This embodiment illustrates a direct view of the user on the digital display 6, and more broadly on the radiation emitting functions.

[0069] In the embodiments illustrated in Figures 2 and 3, the arrangement of the digital display 6, the image sensor 10 and the illuminator 8 on the one hand and the dichroic filter 54 on the other can be perfectly reversed depending on the transmission and reflection properties of the dichroic filter.

[0070] 4 schematically shows a second variant of an optical system 50 between a user's eye 4 and the functions of the digital display 6, the image sensor 10 and the illuminator 8. In this variant, these three functions are performed by a single bidirectional component 56. By bidirectional is meant that the bidirectional component 56 performs both an image acquisition function with the image sensor 10, and an emission function with the digital display 6 and the illuminator 8.

[0071] In this embodiment, the optical system comprises a semi-reflecting plate 58, a first dichroic filter 60 transmitting infrared and reflecting visible light and thus making it possible to collimate and superimpose the display 62 of the digital display on a view of the external environment 64, and a second dichroic filter 66 transmitting visible light and reflecting infrared light so as to focus the infrared radiation on the bidirectional component 56 after illuminating the eye 4. This embodiment is suitable for producing an augmented reality device. In this embodiment, the illuminator 8 is optional, the ambient near infrared radiation, in other words from the external environment, being able to illuminate the eye instead of or in addition to the illuminator 8.

[0072] Figure 5 schematically shows a third variant of an optical system 50 between a user's eye 4 and the digital display 6, the image sensor 10 and the illuminator 8.

[0073] In this variant, the optical system 50 comprises a free-form optical element 68, in other words one or more surfaces of which may be free-form, having the functions of a dichroic filter and a set of lenses. This embodiment is also suitable for producing an augmented reality device.In particular, the optical element 68 comprises a first internal surface 70 on which the radiation from the illuminator 8 and the digital display 6 is reflected after having entered the optical element 68, and a second surface 72 on which it is also reflected before passing through the first surface 70 towards the eye 4, the eye 4 having the sensation, as for the third variant of FIG. 4, that the display 62 of the digital display 6 is superimposed on the vision of the external environment and / or the vision of another element, for example a panel or a second display. The infrared radiation 18 then follows the reverse path towards the image sensor 10. In this embodiment, the first surface 70 has a different behavior depending on the inclination of the light rays striking it.

[0074] Figures 6, 7, 8 and 9 show luminance and pupil diameter profiles during operation of the display device 2 as a function of time t.

[0075] Figure 10 also schematically shows the steps of the method for adapting the luminance of the display device 2, steps whose implementation influences the evolution of the profiles of Figures 6 to 9.

[0076] After positioning the user's eye 4 opposite the display device, a step 74 is performed for acquiring an image of the user's eye 4. This step 74 is performed using the image sensor 10.

[0077] Then, a step 75 is carried out for measuring the initial diameter of the pupil of the user's eye 4 from the image of the user's eye 4 and an initial luminance setpoint is defined for the digital display 6. The initial diameter is noted DI. The initial setpoint is predetermined or calculated as a function of said initial diameter. Steps 74 and 75 can also be carried out with the acquisition of several images of the user's eye 4 when the latter positions his eye opposite the display device 2.

[0078] A step 76 is then carried out to adjust the luminance of the digital display 6 with the initial luminance setting. The luminance of the digital display 6 is denoted La.

[0079] The luminance La of the digital display 6 then influences the size of the pupil of the user's eye 4 if the luminance of the external environment noted Le differs.

[0080] The luminance adaptation method then comprises a repetition of steps 74 to 76 with the acquisition of a series of images of the user's eye 4, the measurement of the diameter D of the pupil of the user's eye 4 for each image of the series of images acquired, and the adjustment of the luminance La of the digital display 6 as a function of the change in the measurement of the diameter D of the pupil of the user's eye 4.

[0081] In Figure 6, the luminances Le and La are equal, which is why the diameter D changes very little around DI and a tolerance range of plus or minus M. The tolerance range of plus or minus M makes it possible in particular to filter micro-variations in luminance and avoid discomfort for the user.

[0082] In Figure 7, the user introduces his eye 4 into contact with the eyepiece and after the first three steps, the luminance La is greater than the luminance Le. The pupil of the eye 4 therefore closes until the display device stabilizes its luminance La at the level of the external luminance Le thanks to the study of the variations in diameter D of the user's pupil.

[0083] In Figure 8, the user brings his eye into contact with the eyepiece and after the first three steps, the luminance La is lower than the luminance Le. The pupil of the eye 4 therefore opens until the display device 2 stabilizes its luminance La at the level of the external luminance Le thanks to the study of the variations in diameter D of the user's pupil.

[0084] In Figure 9, an implementation of the method is shown in which the initial diameter DI generating the initial luminance setpoint is modified as a function of the luminance of the external environment Le. This mode of implementation is a preferred mode and makes it possible to jointly use the external luminance and the diameter of the pupil of the user's eye to best adapt the luminance La of the digital display 6. This is an a posteriori adaptation for which if the luminance Le varies after the eye 4 is positioned, the display device 2 varies its luminance La to prepare the eye 4 for a higher or lower luminance Le, without it being dazzled, or seeing nothing, when looking outside the display device 2.

[0085] More generally, the display device and the associated luminance adaptation method make it possible to standardize the luminance of the digital display with the luminance of the external environment so that the user can switch from one to the other without experiencing visual fatigue.

Claims

DEMANDS 1. Adaptive luminance image display device (2) comprising a peephole for accommodating at least one eye (4) of a user, and a digital display (6) in the visible range positioned behind the peephole, characterized in that it comprises an illuminator (8) for illuminating the user's eye (4) accommodated by the peephole, an image sensor (10) for collecting an image of the user's eye accommodated by the peephole, and a control system (12) for the luminance of the digital display (6) as a function of at least one physiological variable from the image of the user's eye (4).

2. Device according to claim 1, wherein the control system (12) comprises a module (28) for detecting and measuring the diameter of the pupil of the user's eye (4) on an image of the user's eye, and / or a module (30) for detecting the user's state and stress level comprising a module for retrieving the user's temperature and / or a module for retrieving the user's heart rate, at least one physiological variable being selected from the diameter of the eye, the temperature, and the user's heart rate.

3. Device according to any one of claims 1 and 2, comprising an ambient luminance sensor (38) and / or a light environment sensor (40) and / or a user control interface (34) such that the control system (12) of the luminance of the digital display (6) also uses the ambient luminance and / or the light environment and / or a user control to control said luminance of the digital display (6).

4. Device according to any one of claims 1 and 3, wherein the illuminator (8) and the image sensor (10) operate in the near-infrared spectral range.

5. Device according to any one of claims 1 to 4, further comprising an optical system (50) including a set of focusing and / or collimating lenses (52) for guiding light from the digital display (6) and from the illuminator (8) towards the user's eye (4) and from the user's eye (4) to the image sensor (10), the optical system (50) also including a dichroic filter (54).

6. Device according to claim 5, wherein the optical system (50) comprises a freeform optical element (68) comprising the dichroic filter and the lens assembly.

7. Device according to any one of claims 1 to 6, comprising a user eye proximity sensor for activating the image display device (2) when the user's eye (4) is nearby.

8. Device according to any one of claims 1 to 7, wherein the luminance control system (12) further comprises a colorimetry control module (48).

9. Device according to any one of claims 1 to 8, wherein the servo system (12) comprises a module (26) for controlling the operating parameters of the illuminator (8) and the image sensor (10) as a function of the image acquired from the user's eye (4).

10. A method for adapting the luminance of a display device (2) according to any one of claims 1 to 9, characterized in that it comprises the following steps: Acquisition of an image of the user's eye (step 74); Measurement of the initial diameter of the user's eye pupil from the image of the user's eye and definition of an initial luminance setpoint for the digital display (step 75); Adjusting the luminance of the digital display (6) with the initial luminance setpoint (step 76); Acquisition of a series of images of the user's eye (step 74); Measurement of the user's pupil diameter for each image in the acquired image series (step 75); and Adjusting the luminance of the digital display according to the change in the measurement of the diameter of the user's eye pupil (step 76).

Images