Display device, method for controlling display device, and program

Abstract

[Problem] To provide a display device with which high visibility can be obtained even if there is a sudden change in luminance. [Solution] A display device (10) includes: a processing unit (14) that generates image information; a first light source (11) that emits image light based on the image information; an optical system (13) that guides the image light to a pupil; a detection unit (15) that detects outside-world illuminance; and an imaging unit (16) that photoelectrically converts an outside-world image to acquire image data. The processing unit switches whether or not to generate the image information including the image data according to the illuminance.

Description

Display device, display device control method, and program

[0001] The present invention relates to a display device, a control method for a display device, and a program.

[0002] Conventionally, a technology has been known for display devices such as AR (Augmented Reality) glasses that transmit light (external light) from the outside world (real space) and display an image superimposed on an image of the outside world. Furthermore, a technology has also been known for such display devices that improves the visibility of an image by determining the brightness of the displayed image according to the illuminance of the external light. Patent Literature 1 discloses a display system that can adjust the brightness of a displayed image based on the illuminance of the external light.

[0003] Japanese Patent Application Laid-Open No. 2021-057737

[0004] The display system disclosed in Patent Document 1 can accommodate a certain degree of luminance change, but it is difficult to obtain high visibility when a sudden luminance change occurs.

[0005] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a display device that can provide high visibility even when there is a sudden change in luminance.

[0006] A display device according to one aspect of the present invention includes a processing unit that generates image information, a first light source that emits image light based on the image information, an optical system that directs the image light to a pupil, a detection unit that detects the illuminance of the outside world, and an imaging unit that photoelectrically converts an image of the outside world to obtain image data, and the processing unit switches whether or not to generate the image information including the image data depending on the illuminance.

[0007] Other objects and features of the present invention are illustrated in the following examples.

[0008] According to the present invention, it is possible to provide a display device that can obtain high visibility even when there is a sudden change in luminance.

[0009] FIG. 1 is a configuration diagram of a display device in Example 1. FIG. 2 is a configuration diagram of a pixel region of an imaging element in each Example. FIG. 3 is a flowchart showing a control method of a display device in Example 1. FIG. 4 is an explanatory diagram of each threshold value in Example 1. FIG. 5 is a configuration diagram of a display device in Example 2. FIG. 6 is a flowchart showing a control method of an imaging device in Example 2. FIG. 7 is an explanatory diagram of each threshold value in Example 2. FIG. 8 is a configuration diagram of a display device in Example 3. FIG. 9 is a configuration diagram of a display device and a liquid crystal shutter in Example 4. FIG. 10 is a configuration diagram of a display device in Example 5.

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same components are designated by the same reference numerals, and redundant explanations will be omitted.

[0011] First Embodiment First, a display device 10 according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a configuration diagram of the display device 10. The display device 10 is, for example, an AR glass, and is configured to include a light source unit (first light source) 11, a light guide plate (optical system) 13, a processing unit 14, an illuminance detection unit 15, an imaging unit 16, and a timing control unit 31.

[0012] The light source unit 11 is an image light source (first light source) that emits image light 12. In this embodiment, the dominant wavelength (peak illuminance) of the light source unit 11 is a wavelength of 440 nm or more and 660 nm or less. The light guide plate 13 is an optical system that guides the image light 12 emitted from the light source unit 11 to a pupil 17 of a user (observer). Note that in this embodiment, other optical members such as a free-form surface prism may be used instead of or together with the light guide plate 13. Furthermore, in this embodiment, the display device 10 may further include an optical element such as a relay unit for forming the pupil 17.

[0013] The processing unit 14 has a microprocessor such as a CPU or MPU, and generates image information. The illuminance detection unit 15 is a detection unit (measurement unit) that detects (measures) the illuminance of the outside world. The imaging unit 16 has an imaging optical system 161 and an imaging element 162. The imaging element 162 photoelectrically converts an image of the outside world (optical image) formed by the imaging optical system 161 to obtain image data. The imaging element 162 is controlled to start and end exposure in response to receiving a control signal transmitted from the timing control unit 31.

[0014] In this embodiment, the imaging unit 16 may be configured to detect illuminance. That is, the imaging unit 16 may be configured to have both the functions of acquiring image data and detecting illuminance (the imaging unit and the illuminance detection unit may be integrated). In this case, there is no need to provide an illuminance detection unit separate from the imaging unit 16, which contributes to reducing the size and weight of the display device 10.

[0015] In this embodiment, the image sensor 162 has a SPAD (Single Photon Avalanche Diode) sensor. The SPAD sensor is a photon-counting image sensor that can measure the number of incident photons and handle the incident light as a digital value. Because the SPAD sensor can directly measure the number of incident photons, random noise is not generated, and the S / N ratio can be improved.

[0016] Here, the pixel region of the image sensor 162 will be described with reference to Fig. 2. Fig. 2 is a configuration diagram of the pixel region of the image sensor 162. In the pixel region, a plurality of SPAD pixels 103 are repeatedly arranged two-dimensionally in the X and Y directions. Each SPAD pixel 103 has a photoelectric conversion unit (avalanche diode) 201, a quenching element 202, a control unit 210, a counter / memory 211, and a readout unit 212. In other words, the image sensor 162 has a SPAD in which a plurality of avalanche diodes are arranged in a two-dimensional plane.

[0017] A potential based on a potential VH higher than the voltage VL supplied to the anode is supplied to the cathode of the photoelectric conversion unit 201. A potential is supplied to the anode and cathode of the photoelectric conversion unit 201 so that a reverse bias is applied so that photons incident on the photoelectric conversion unit 201 are avalanche multiplied. By performing photoelectric conversion with such a reverse bias potential supplied, the charge generated by the incident light undergoes avalanche multiplication, generating an avalanche current.

[0018] When a reverse bias potential is supplied and the potential difference between the anode and the cathode is greater than the breakdown voltage, the photoelectric conversion unit (avalanche diode) 201 operates in Geiger mode. An avalanche diode that uses Geiger mode operation to quickly detect weak signals at the single-photon level is called a SPAD.

[0019] The control unit 210 determines whether to count the output signal from the photoelectric conversion unit 201. For example, the control unit 210 is a switch (gate circuit) provided between the photoelectric conversion unit 201 and the counter / memory 211. The gate of the switch is connected to the pulse line 124, and the control unit 210 is switched on and off in response to a signal input to the pulse line 124. A signal based on a control signal from the timing control unit 31 is input to the pulse line 124. The gates of the switches are controlled collectively for all columns. This controls the start and end of photodetection for all SPAD pixels 103 collectively.

[0020] Furthermore, the control unit 210 may be configured with a logic circuit instead of a switch. For example, if an AND circuit is provided as the logic circuit, and the first input of the AND circuit is the output from the photoelectric conversion unit 201 and the second input is the signal on the pulse line 124, it becomes possible to switch whether or not to count the output signal from the photoelectric conversion unit 201. Note that the control unit 210 does not need to be provided between the photoelectric conversion unit 201 and the counter / memory 211, and may be a circuit that inputs a signal that switches between operating and non-operating the counter of the counter / memory 211.

[0021] The counter / memory 211 counts the number of photons entering the photoelectric conversion unit 201 in accordance with a control signal from a control line 213 and stores the count as digital data. The readout unit 212 is connected to the counter / memory 211 and a readout signal line 123. A control pulse is supplied to the readout unit 212 from the vertical scanning circuit unit via a control line 214, and the readout unit 212 switches whether or not to output the count value of the counter / memory 211 to the readout signal line 123. The readout unit 212 includes, for example, a buffer circuit for outputting a signal.

[0022] The readout signal line 123 is a signal line that outputs from the image sensor 162 to a signal processing unit (not shown). The horizontal scanning circuit unit and the vertical scanning circuit unit may be provided on the substrate on which the SPAD array is provided, or may be provided on a substrate different from the substrate on which the SPAD array is provided.

[0023] In this embodiment, the image sensor 162 includes a SPAD sensor, but is not limited to this. This embodiment can also be applied to other photoelectric conversion elements, such as a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) sensor, instead of a SPAD sensor.

[0024] In this embodiment, the processing unit 14 switches whether or not to generate image information including image data, depending on the illuminance detected by the illuminance detection unit 15. Therefore, the display device 10 of this embodiment can obtain high visibility even when there is a sudden change in external luminance, and is particularly suitable for use in nighttime photography of animals or stars, or for long-term nighttime travel.

[0025] For example, when the illuminance is lower than a first threshold (first illuminance A), the processing unit 14 generates an image (captured image data) of the outside world captured by the imaging unit 16. Then, the processing unit 14 generates image data (third image data) by superimposing the captured image data (first image data) on image data (second image data) related to specific information as image information (image information including the captured image data) (video see-through state). Here, the second image data is digital content (digital information that does not exist in real space), such as a video or 3D data (stereo image data), and is displayed superimposed on the first image data related to the image of the outside world (image in real space).

[0026] Furthermore, the processing unit 14 controls the light source unit 11 so that image light 12 corresponding to the image information is emitted from the light source unit 11. This allows the display device 10 to display, in the video see-through state, the image light 12 corresponding to the image information generated by the processing unit 14 so that it can be viewed by the user. Note that when there is no second image data, the captured image data corresponds to the third image data.

[0027] On the other hand, for example, when the illuminance is higher than the second threshold (second illuminance B), the processing unit 14 does not generate image information including captured image data (first image data) (see-through state). However, this embodiment is not limited to this. If image data (second image data) related to specific information is present, image information not including captured image data (first image data) may be generated. The processing unit 14 also controls the light source unit 11 to emit image light 12 corresponding to the image information. This allows the display device 10 to display, in the see-through state, the image light 12 corresponding to the image information generated by the processing unit 14 so that the user can observe it. In this case, if the second image data is present, the second image data corresponds to the image information, and if the second image data is absent, there is no image information (the light source unit 11 does not emit the image light 12). However, since the display device 10 is configured so that light from the outside world (external light) passes through the light guide plate 13, when the outside world is bright, the user can directly observe a real image of the outside world. In this embodiment, the first threshold and the second threshold may be the same value or different values.

[0028] As shown in FIG. 1 , the light guide plate 13 has a first surface 13a on the side closer to the user's pupil 17 (pupil side), and a second surface 13b opposite the first surface 13a. External light is incident on the second surface 13b. In this embodiment, with respect to light perpendicularly incident on the second surface 13b, the average transmittance of light in the green band (wavelength range of 510 nm to 580 nm) that passes through the first surface 13a and the second surface 13b is preferably 20% or more. More preferably, the average transmittance is 35% or more. Even more preferably, the average transmittance is 50% or more.

[0029] The display device 10 of this embodiment can provide high visibility even when there is a sudden change in external brightness. Furthermore, the SPAD sensor consumes a lot of power and has the characteristic of saturating when the external illuminance is relatively high (bright). Therefore, in order to reduce power consumption and obtain high-quality images, it is preferable not to operate the SPAD sensor (not to capture images) in a high-illuminance environment.

[0030] Next, a control method for the display device 10 in this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the control method for the display device 10. The flow in Fig. 3 is executed when the power of the display device 10 is turned on, and stops when the power of the display device 10 is turned off.

[0031] First, in step S101, the processing unit 14 of the display device 10 sets the display device 10 to a see-through state. That is, the display device 10 does not generate image information that includes captured image data captured by the imaging unit 16. When image data related to specific information is available, the display device 10 generates image information that does not include captured image data.

[0032] Next, in step S102, the processing unit 14 determines whether the illuminance L detected by the illuminance detection unit 15 is lower than the first illuminance A. If the illuminance L is higher than the first illuminance A, the process returns to step S101 and the see-through state is maintained. On the other hand, if the illuminance L is lower than the first illuminance A, the process proceeds to step S103.

[0033] In step S103, the processing unit 14 acquires captured image data (first image data) captured using the imaging unit 16. Subsequently, in step S104, the processing unit 14 generates image data (third image data) by superimposing the captured image data on image data (second image data) relating to specific information as image information (image information including the captured image data). At this time, the light source unit 11 emits image light 12 corresponding to the third image data (video see-through state). Note that if there is no second image data, there is no image data to be superimposed, and therefore the first image data corresponds to the third image data.

[0034] Next, in step S105, the processing unit 14 determines whether the illuminance L detected by the illuminance detection unit 15 is lower than the second illuminance B. If the illuminance L is lower than the second illuminance B, the process returns to step S103, and the display device 10 maintains the video see-through state. On the other hand, if the illuminance L is higher than the second illuminance B, the process returns to step S101, and the display device 10 transitions from the video see-through state to the see-through state. The above flow continues while the power of the display device 10 is on, and stops when the power is turned off.

[0035] In FIG. 3, the state immediately after the power is turned on (initial state) is described as the see-through state, but this is not limiting, and the initial state may also be the video see-through state.

[0036] Here, the relationship between the first illuminance A (first threshold) and the second illuminance B (second threshold) will be described with reference to Fig. 4. Fig. 4 is an explanatory diagram of the first illuminance A and the second illuminance B. In Fig. 4, the horizontal axis represents illuminance L, and the illuminance L increases from left to right. In Fig. 4, the side with high illuminance L (right side) represents a see-through state (a state in which the user is observing a real image of the outside world through the light guide plate 13), and the side with low illuminance L (left side) represents a video see-through state (a state in which the user is observing an image of the outside world captured by the imaging unit 16).

[0037] 4, the processing unit 14 generates image information including captured image data when the illuminance L changes from a state higher than the first illuminance A to a state lower than the first illuminance A. In other words, when the illuminance L changes to be lower than the first illuminance A, the display device 10 transitions from the see-through state to the video see-through state.

[0038] On the other hand, when the illuminance L changes from a state lower than the second illuminance B to a state higher than the second illuminance B, the processing unit 14 does not generate image information including the captured image data, or generates image information not including the captured image data. In other words, when the illuminance L changes to be higher than the second illuminance B, the display device 10 transitions from the video see-through state to the see-through state.

[0039] In this embodiment, it is preferable that the second illuminance B is higher than the first illuminance A (B > A). This makes it possible to reduce the frequency of transitions between the see-through state and the video see-through state. However, this embodiment is not limited to this, and the second illuminance B may be equal to the first illuminance A (B = A).

[0040] In this embodiment, the first illuminance A (lux (lx)) preferably satisfies the condition 1.0≦A≦20.0. The second illuminance B (lx) preferably satisfies the condition A≦B≦40.0.

[0041] As described above, in this embodiment, the display device switches whether or not to generate image information including captured image data depending on the illuminance, and therefore, this embodiment can provide a display device and a control method for the display device that can achieve high visibility even when there is a sudden change in luminance.

[0042] (Example 2) Next, a display device 50 according to Example 2 of the present invention will be described with reference to Fig. 5. Fig. 5 is a configuration diagram of the display device 50. The display device 50 differs from the display device 10 in that the display device 50 has a light source unit 51 instead of the light source unit 11 of the display device 10 described in Example 1 with reference to Fig. 1, and in that the display device 50 has a diffuser plate 52. Note that other configurations of the display device 50 are similar to those of the display device 10 of Example 1, and therefore description thereof will be omitted.

[0043] The light source unit 51 includes a first light source 511 and a second light source 512. The first light source 511 is an image light source that emits image light 12. In this embodiment, the dominant wavelength (peak illuminance) of the first light source 511 is equal to or greater than 440 nm and equal to or less than 660 nm. The second light source 512 is a projection light source that emits projection light such as infrared light (to project to the outside world). Note that the second light source 512 is configured to continuously emit projection light and is different from a light source that emits light for distance measurement. In this embodiment, the dominant wavelength (peak light intensity) of the second light source 512 is equal to or greater than 660 nm and equal to or less than 2000 nm. The diffuser plate 52 is disposed on the second surface 13b of the light guide plate 13 and diffuses the projection light from the second light source 512 toward the outside world.

[0044] In this embodiment, the second light source 512 switches whether to emit projection light depending on the illuminance detected by the illuminance detection unit 15. Specifically, in the video see-through state, the second light source 512 emits projection light when, for example, the illuminance is lower than a third threshold (third illuminance C). On the other hand, the second light source 512 does not emit projection light when, for example, the illuminance is higher than a fourth threshold (fourth illuminance D). Therefore, the display device 10 of this embodiment can achieve high visibility even when there is a sudden change in external brightness and the external brightness is particularly low. Furthermore, by switching the second light source 512 on and off as needed, power consumption can be reduced.

[0045] Next, a control method for the display device 50 in this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the control method for the display device 50. The flow in Fig. 6 is executed when the power of the display device 50 is turned on, and stops when the power of the display device 50 is turned off.

[0046] First, in step S201, the processing unit 14 of the display device 50 sets the display device 50 to a see-through state. That is, the display device 50 does not generate image information that includes captured image data captured by the imaging unit 16. If there is image data related to specific information, the display device 50 generates image information that does not include captured image data.

[0047] Next, in step S202, the processing unit 14 determines whether the illuminance L detected by the illuminance detection unit 15 is lower than the first illuminance A. If the illuminance L is higher than the first illuminance A, the process returns to step S201 and the see-through state is maintained. On the other hand, if the illuminance L is lower than the first illuminance A, the process proceeds to step S203.

[0048] In step S203, the processing unit 14 determines whether the illuminance L detected by the illuminance detection unit 15 is lower than the third illuminance C. If the illuminance L is higher than the third illuminance C, the process proceeds to step S210. On the other hand, if the illuminance L is lower than the third illuminance C, the process proceeds to step S204.

[0049] In step S204, the processing unit 14 turns on the second light source 512, causing projection light to be emitted from the second light source 512 (light-emitting video see-through state). Subsequently, in step S207, the processing unit 14 acquires captured image data (first image data) captured using the imaging unit 16. Subsequently, in step S208, the processing unit 14 generates image data (third image data) by superimposing the captured image data on image data (second image data) relating to specific information as image information (image information including the captured image data). At this time, the first light source 511 emits image light 12 corresponding to the third image data. Note that if there is no second image data, there is no image data to be superimposed, and therefore the first image data corresponds to the third image data.

[0050] Next, in step S205, the processing unit 14 determines whether the illuminance L detected by the illuminance detection unit 15 is lower than the second illuminance B. If the illuminance L is higher than the second illuminance B, the process returns to step S201, and the display device 50 transitions from the light-emitting video see-through state to the see-through state. At this time, the second light source 512 is turned off. On the other hand, if the illuminance L is lower than the second illuminance B, the process proceeds to step S206.

[0051] In step S206, the processing unit 14 determines whether the illuminance L detected by the illuminance detection unit 15 is lower than the fourth illuminance D. If the illuminance L is lower than the fourth illuminance D, the process proceeds to step S207, where the display device 50 maintains the luminous video see-through state. On the other hand, if the illuminance L is higher than the fourth illuminance D, the process proceeds to step S209.

[0052] In step S209, the processing unit 14 turns off the second light source 512. Then, in steps S210 and S211, the processing unit 14 executes the same processes as in steps S207 and S208, respectively (non-emission video see-through state).

[0053] Next, in step S212, the processing unit 14 determines whether the illuminance L detected by the illuminance detection unit 15 is lower than the second illuminance B. If the illuminance L is higher than the second illuminance B, the process returns to step S201, and the display device 50 transitions from the non-emission video see-through state to the see-through state. On the other hand, if the illuminance L is lower than the second illuminance B, the process proceeds to step S213.

[0054] In step S213, processing unit 14 determines whether illuminance L detected by illuminance detection unit 15 is lower than the third illuminance C. If illuminance L is higher than the third illuminance C, processing proceeds to step S210. On the other hand, if illuminance L is lower than the third illuminance C, processing proceeds to step S204, where display device 50 transitions from the non-emission video see-through state to the emission video see-through state. The above flow continues while display device 50 is powered on and stops when the power is turned off.

[0055] Here, referring to FIG. 7 , the relationship between the first illuminance A (first threshold), the second illuminance B (second threshold), the third illuminance C (third threshold), and the fourth illuminance D (fourth threshold) will be described. FIG. 7 is an explanatory diagram of the first illuminance A, the second illuminance B, the third illuminance C, and the fourth illuminance D. In FIG. 7 , the horizontal axis represents illuminance L, and the illuminance L increases from left to right. In FIG. 7 , the area on the side with high illuminance L (right side) represents a see-through state (a state in which a user observes a real image of the outside world through the light guide plate 13). The area on the side with low illuminance L (left side) represents a light-emitting video see-through state (a state in which a user observes an image of the outside world captured by the image capture unit 16 while emitting projection light). The central area represents a non-light-emitting video see-through state (a state in which a user observes an image of the outside world captured by the image capture unit 16 without emitting projection light). In FIG. 7, the relationship between the first illuminance A and the second illuminance B is the same as in FIG. 4, and therefore the description thereof will be omitted.

[0056] 7 , when the illuminance L changes from a state higher than the third illuminance C to a state lower than the third illuminance C in the video see-through state, the processing unit 14 turns on the second light source 512 to emit projection light from the second light source 512. In other words, when the illuminance L changes to be lower than the third illuminance C, the display device 50 transitions from the non-illuminated video see-through state to the illuminated video see-through state.

[0057] On the other hand, when the illuminance L changes from a state lower than the fourth illuminance D to a state higher than the fourth illuminance D, the processing unit 14 turns off the second light source 512. In other words, when the illuminance L changes to be higher than the fourth illuminance D, the display device 50 transitions from the luminous video see-through state to the non-luminous video see-through state.

[0058] In this embodiment, it is preferable that the fourth illuminance D is higher than the third illuminance C (D > C). This reduces the frequency of transitions between the luminous video see-through state and the non-luminous video see-through state. However, this embodiment is not limited to this, and the fourth illuminance D may be equal to the third illuminance C (D = C).

[0059] In this embodiment, the third illuminance C (lx) preferably satisfies the condition 0.02≦C≦1.0, and the fourth illuminance D (lx) preferably satisfies the condition C≦D≦2.0.

[0060] Furthermore, in this embodiment, it is preferable that the third illuminance C is lower than the first illuminance A (C<A). Furthermore, in this embodiment, it is preferable that the fourth illuminance D is lower than the second illuminance B (D<B). However, this embodiment is not limited to this, and at least two illuminances selected from the first illuminance A, the second illuminance B, the third illuminance C, and the fourth illuminance D may be equal. For example, the first illuminance A, the second illuminance B, the third illuminance C, and the fourth illuminance D may all be equal. In this case, there is no non-emissive video see-through state, and only a transition between the see-through state and the emissive video see-through state occurs.

[0061] As described above, in this embodiment, when generating image information including captured image data, the display device switches between emitting and not emitting projection light depending on the illuminance of the external environment. Therefore, this embodiment can provide a display device and a control method for the display device that can achieve high visibility even in a dark external environment (when brightness is low).

[0062] In this embodiment, the first light source 511 and the second light source 512 are configured as an integrated light source unit 51, but this is not limiting. For example, the first light source 511 and the second light source 512 may be configured as separate light source units that are separated from each other. Alternatively, the first light source 511 and the second light source 512 may be configured as a single common light source. In this case, the common light source emits both visible light and infrared light.

[0063] (Example 3) Next, a display device 80 according to Example 3 of the present invention will be described with reference to Fig. 8. Fig. 8 is a configuration diagram of the display device 80. Each of the components constituting the display device 10 of Example 1 corresponds to one eye (either the left or right eye) of the user. On the other hand, the display device 80 of this example is a glasses-type display device, and has components corresponding to both eyes of the user. That is, the display device 80 has two of each of the components constituting the display device 10 of Example 1 (light source unit 11, light guide plate 13, and imaging unit 16) for the left eye and the right eye.

[0064] According to this embodiment, since it becomes easier for the user to grasp the sense of distance, a display device such as 3D-compatible AR glasses can be realized. Note that this embodiment may include two of each of the components constituting the display device 50 of the second embodiment.

[0065] (Example 4) Next, a display device 90 according to Example 4 of the present invention will be described with reference to Figures 9(a) and (b). Figure 9(a) is a configuration diagram of the display device 90. Figure 9(b) is a configuration diagram of a liquid crystal shutter 91. The display device 90 differs from the display device 80 of Example 3 in that it has a liquid crystal shutter 91 in addition to the components of the display device 80. Note that other configurations of the display device 90 are similar to those of the display device 80, and therefore description thereof will be omitted.

[0066] 9A, the liquid crystal shutter 91 is disposed in front of the light guide plate 13 (farther from the pupil 17 than the light guide plate 13) and adjusts the amount of light from the outside (amount of external light). As shown in FIG. 9B, the liquid crystal shutter 91 is configured by sandwiching a liquid crystal 911 between two polarizing plates 912.

[0067] In this embodiment, the transmittance of the liquid crystal shutter 91 changes in accordance with the illuminance detected by the illuminance detection unit 15. Note that in this embodiment, the transmittance of the liquid crystal shutter 91 changes, for example, linearly, but is not limited to this.

[0068] According to this embodiment, it is possible to improve the visibility of the outside world in a bright environment, and also to protect the user's eyes when the environment changes from a dark environment to a bright environment.

[0069] Fifth Embodiment Next, a display device 100 according to a fifth embodiment of the present invention will be described with reference to FIG. 10 . FIG. 10 is a configuration diagram of the display device 100. The display device 100 is a modified example of the display device 80 described in the third embodiment with reference to FIG. 8 . The display device 100 of this embodiment differs from the display device 80 in that it has an infrared light source 26 instead of one of the two image capturing units 16. That is, the display device 100 is provided with an image capturing unit 16 having a SPAD sensor for one eye (either the left or right eye) of the user, and an infrared light source 26 for projection for the other eye (the other of the left or right eye).

[0070] The infrared light source 26 is turned on at the timing of image capture by the imaging unit 16. The SPAD sensor receives reflected light of the light pulses projected by the infrared light source 26, thereby measuring the time of flight and acquiring distance information. The display device 100 generates a parallax image from the distance information and projects light corresponding to the parallax image from the light source unit 11, making it applicable as 3D-compatible AR glasses. By projecting infrared light from the infrared light source 26, the display device 100 can acquire distance information and generate parallax images even for darker subjects, compared to the display device 80 of Example 3.

[0071] In this embodiment, the darker the subject, the shorter the interval at which the infrared light source 26 projects the pulses is preferably set, which increases the number of photons received per unit time by the SPAD sensor, thereby enabling a clearer image to be obtained.

[0072] The present invention can also be realized by a process in which a program that realizes one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

[0073] According to each embodiment, it is possible to provide a display device, a control method for a display method, and a program that can obtain high visibility even when there is a sudden change in luminance.

[0074] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications and changes are possible within the scope of the invention.

Claims

1. A display device comprising: a processing unit that generates image information; a first light source that emits image light based on the image information; an optical system that directs the image light to a pupil; a detection unit that detects the illuminance of the outside world; and an imaging unit that photoelectrically converts an image of the outside world to obtain image data, wherein the processing unit switches whether or not to generate the image information including the image data depending on the illuminance.

2. The display device according to claim 1, characterized in that the processing unit generates the image information including the image data when the illuminance is lower than the first illuminance, and does not generate the image information including the image data when the illuminance is higher than the second illuminance.

3. The display device according to claim 1 or 2, characterized in that the processing unit generates the image information including the image data when the illuminance is lower than the first illuminance, and generates the image information not including the image data when the illuminance is higher than the second illuminance.

4. The display device according to claim 2 or 3, wherein the second illuminance is equal to the first illuminance.

5. The display device according to claim 2 or 3, wherein the second illuminance is higher than the first illuminance.

6. The display device according to any one of claims 1 to 5, characterized in that the imaging section has an imaging element in which a plurality of avalanche diodes are arranged in a two-dimensional plane.

7. A display device according to any one of claims 1 to 6, further comprising a second light source that emits projection light to the outside world, wherein the second light source switches whether or not to emit the projection light depending on the illuminance.

8. The display device according to claim 7, characterized in that the second light source emits the projection light when the illuminance is lower than the third illuminance, and does not emit the projection light when the illuminance is higher than the fourth illuminance.

9. The display device according to claim 8, wherein the fourth illuminance is equal to the third illuminance.

10. The display device according to claim 8, wherein the fourth illuminance is higher than the third illuminance.

11. A display device as described in any one of claims 8 to 10, characterized in that the processing unit generates the image information including the image data when the illuminance is lower than the first illuminance, and the third illuminance is lower than the first illuminance.

12. A display device as described in any one of claims 8 to 11, characterized in that when the illuminance is higher than the second illuminance, the processing unit does not generate the image information including the image data, or generates the image information not including the image data, and the fourth illuminance is lower than the second illuminance.

13. A display device according to any one of claims 1 to 12, characterized in that the peak illuminance of the first light source is at a wavelength of 440 nm or more and 660 nm or less.

14. A display device according to any one of claims 7 to 12, characterized in that the peak of the light intensity of the second light source is between 660 nm and 2000 nm.

15. A display device according to any one of claims 1 to 14, further comprising a liquid crystal shutter whose transmittance changes in accordance with the illuminance.

16. The display device according to any one of claims 1 to 15, characterized in that the display device is a glasses-type display device.

17. A display device according to any one of claims 1 to 16, characterized in that the optical system has a first surface on the pupil side and a second surface opposite the first surface, and the external light is incident on the second surface.

18. The display device according to claim 17, characterized in that, for light incident perpendicularly on the second surface, the average transmittance of the light in the wavelength range of 510 nm to 580 nm that is transmitted through the first surface and the second surface is 20% or more.

19. The display device according to any one of claims 1 to 18, characterized in that the optical system has at least one of a light guide plate and a free-form surface prism.

20. A control method for a display device, comprising: a first step of generating image information; a second step of emitting image light based on the image information from a first light source and guiding the image light to a pupil using an optical system; a third step of detecting the illuminance of the outside world; and a fourth step of photoelectrically converting the image of the outside world to obtain image data, wherein in the first step, whether or not to generate the image information including the image data is switched depending on the illuminance.

21. A program causing a computer to execute the display device control method according to claim 20.

Images