Display device and display method

Abstract

The purpose of the present invention is to provide a display device with which the controllability of video presentation corresponding to the eyeball characteristic of a user can be further improved.　Provided is a display device equipped with: a light source; a processing unit that processes the characteristic distribution of an eyeball; a monitoring unit that monitors the state of the eyeball; a matching unit that matches the characteristic distribution of the eyeball and the state of the eyeball; and an irradiation unit that irradiates a predetermined spot of a retina with video display light emitted from the light source.

Description

[Technical Field]

[0001] The present technology relates to a display device and a display method. [Background technology]

[0002] In recent years, attention has been focused on technology that displays images superimposed on scenes from the outside world, such as real landscapes. This technology is also known as augmented reality (AR) technology. One example of a product that uses this technology is a head-mounted display. A head-mounted display is worn on the user's head. In a method of displaying images using a head-mounted display, for example, light from the head-mounted display reaches the user's eyes in addition to light from the outside world, and the user perceives the image created by the light from the display as being superimposed on an image of the outside world.

[0003] For example, Patent Document 1 proposes an image display device that can detect the irradiation position on the retina of light emitted from a light source and adjust the irradiation position on the retina. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2012 / 169064 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology proposed in Patent Document 1 may not be able to further improve the controllability of video presentation according to the eyeball characteristics of the user.

[0006] Therefore, the present technology has been developed in consideration of such circumstances, and its main purpose is to provide a display device and a display method that can further improve the controllability of image presentation according to the user's eyeball characteristics. [Means for solving the problem]

[0007] The inventors conducted intensive research to achieve the above-mentioned objectives, and as a result, surprisingly, they succeeded in further improving the controllability of image presentation in accordance with the user's eye characteristics, thereby completing the present technology.

[0008] That is, the present technology provides, as a first aspect, A light source and a processing unit for processing the eyeball characteristic distribution; a monitoring unit that monitors the state of the eyeball; a matching unit that matches the eyeball characteristic distribution with the eyeball state; an irradiation unit that irradiates a predetermined location on the retina with image display light emitted from the light source; A display device is provided, comprising:

[0009] A display device according to a first aspect of the present technology includes: The eyeball detection apparatus may further include an acquisition unit that acquires the characteristic distribution of the eyeball, In this case, the acquisition unit may be composed of at least one selected from the group consisting of a fundus camera, an OCT, a refractometer, and a light detection device that detects return light from an IR scan.

[0010] In the display device according to the first aspect of the present technology, The monitoring unit can monitor the state of the eyeball using a corneal reflex or a fundus reflex.

[0011] A display device according to a first aspect of the present technology includes: The display device may further include a tracking unit that causes the image display light to follow the movement of the eyeball, In this case, the tracking unit may be composed of at least one selected from the group consisting of a combiner, a relay system driving unit, a mirror driving unit, and a retardation panel.

[0012] In the display device according to the first aspect of the present technology, defining a coordinate system based on the property distribution of the eye; In this case, the matching unit can match the eyeball characteristic distribution with the state of the eyeball to which an image is to be presented via the coordinate system. The coordinate system may be defined based on at least two selected from the group consisting of a first fovea of ​​the right eye, a first blind spot of the right eye, a second fovea of ​​the left eye, and a second blind spot of the left eye.

[0013] In the display device according to the first aspect of the present technology, The light source may be a laser light source.

[0014] A display device according to a first aspect of the present technology includes: It may further comprise a scanning mirror, The scanning mirror can irradiate the image display light onto the retina.

[0015] In the display device according to the first aspect of the present technology, The irradiation unit may further include a member placed in front of the eye, The member may have see-through properties, In this case, the member may be a first optical element including a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, or a metasurface, or a second optical element that reflects predetermined light and transmits light other than the predetermined light.

[0016] A display device according to a first aspect of the present technology includes: may further include a wavelength dispersion compensation member, In this case, the wavelength dispersion compensation member may be a first optical element including a reflection or transmission type volume hologram, a reflection or transmission type relief hologram, or a metasurface.

[0017] In addition, as a second aspect, this technology: Processing a distribution of eyeball characteristics; monitoring the condition of the eye; matching the characteristic distribution of the eyeball with a state of the eyeball; Illuminating a predetermined location on the retina with image display light emitted from a light source; A display method is provided, including:

[0018] The display method according to the second aspect of the present technology includes: The method may further include obtaining the property distribution of the eyeball.

[0019] According to the present technology, it is possible to further improve the controllability of image presentation according to the eyeball characteristics of the user. Note that the effects described herein are not necessarily limited to those described herein and may be any of the effects described in the present disclosure. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram showing an example of a flow of displaying an image by a display device to which the present technology is applied. [Figure 2] FIG. 2 is a diagram for explaining that the positions of the fovea and blind spot change with left and right rotation of the eyeball of the right eye. [Figure 3] FIG. 3 is a diagram for explaining the results of matching the fundus map with the state of the eyeball to which an image is to be presented. [Figure 4] FIG. 4 is a diagram for explaining the results of matching the interpolated fundus map with the state of the eyeball to which an image is to be presented. [Figure 5] FIG. 5 is a diagram showing another example of the flow of video display by a display device to which the present technology is applied. [Figure 6] FIG. 6 is a diagram for explaining normal vision, distorted vision (metamorphopsia), and central darkness (central scotoma). [Figure 7] FIG. 7 is a diagram for explaining how eyeball distortion is corrected. [Figure 8] FIG. 8 is a diagram for explaining correction of eyeball unevenness. [Figure 9] FIG. 9 is a top view showing a configuration example of a display device to which the present technology is applied. [Figure 10]FIG. 10 is a front view showing a configuration example of a display device to which the present technology is applied. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of a display device according to a first embodiment to which the present technology is applied. [Figure 12] FIG. 12 is a block diagram showing an example of the configuration of a display device according to a second embodiment to which the present technology is applied. [Figure 13] FIG. 13 is a diagram showing an example of a member arranged in front of the eyes that is included in a display device to which the present technology is applied. [Figure 14] FIG. 14 is a diagram showing another example of a member provided in a display device to which the present technology is applied and placed in front of the user's eyes. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, preferred embodiments for implementing the present technology will be described. The embodiments described below are examples of typical embodiments of the present technology, and the scope of the present technology will not be interpreted narrowly by them. In the description using the drawings, the same or equivalent elements or members are denoted by the same reference numerals, and redundant description will be omitted.

[0022] The explanation will be given in the following order. 1. Overview of this technology 2. First Embodiment (Example 1 of Display Device) 3. Second Embodiment (Example 2 of Display Device) 4. Third embodiment (Example 1 of display method) 5. Fourth embodiment (Example 2 of display method)

[0023] <1. Overview of this technology> First, an overview of the present technology will be described. The present technology relates to a display device and a display method.

[0024] This technology acquires the characteristic distribution of the eyeball (shape and optical characteristics) and matches the coordinates specified on the distribution with the results of EyeSense, making it possible to display an image corresponding to the characteristic distribution of the eyeball at a specified position on the retina.

[0025] First, an example of a technique other than the present technique will be described.

[0026] Example 1 of other technology is flat-panel magnification eyewear. Because the focal plane is fixed, it may not be possible to precisely identify the stimulation site on the retina. Example 2 of other technology is laser retinal direct imaging. Using a half mirror in this technology may result in poor see-through performance, making it unsuitable for focusing on external objects. Using a holographic optical element (HOE) may result in a narrow field of view and wavelength dispersion, making it difficult to control the range and accuracy of stimulus presentation. Example 3 of other technology is position adjustment (tracking). This technology may use pattern matching (patterns such as retinal patterns, such as blood vessels) to adjust position, which can result in large amounts of reference data and heavy processing.

[0027] The present technology has been developed in view of the above circumstances.

[0028] According to the present technology, it is possible to realize further improvement in the controllability of video presentation according to the eyeball characteristics of a user. Also, according to the present technology, it is possible to realize further improvement in the controllability of video presentation according to the eyeball movement (eye movement) of a user.

[0029] Specifically, in this technology, eyeball characteristic distribution information is acquired, a coordinate system is defined, and a display image and display position are defined in the coordinate system, so less data is required than when a pattern map is used, and the display image (presented image) and display position (presentation position) are controlled based on eyeball movement information (e.g., eyeball rotation information), so processing is lighter than when pattern matching is used. The coordinate system is defined based on at least two selected from the group consisting of, for example, the fovea of ​​the right eye (the fovea of ​​the right eye may be referred to as the first fovea), the blind spot of the right eye (the blind spot of the right eye may be referred to as the first blind spot), the second fovea of ​​the left eye (the fovea of ​​the left eye may be referred to as the second fovea), and the second blind spot of the left eye (the blind spot of the left eye may be referred to as the second blind spot).

[0030] In addition, in this technology, a laser light source is preferably used, making it possible to present stimuli of a specific wavelength, and the component placed in front of the eye is preferably see-through, making it possible to focus on objects in the outside world and present unconscious stimuli, and further, a wavelength dispersion compensation component is preferably used, making it possible to compensate for wavelength dispersion and accurately stimulate a specific point (any point) on the retina.

[0031] Examples of the component placed in front of the eye include a reflection or transmission volume hologram, a reflection or transmission relief hologram, a first optical element including a metasurface, a second optical element that reflects predetermined light and transmits light other than the predetermined light, etc. The first optical element including a metasurface may have a structure in which a dielectric or metal is periodically arranged at narrow intervals on the surface of the first optical element, and can deflect light in a specific frequency band.

[0032] Examples of the second optical element include a half mirror (preferably with a light transmittance of 50% or more), a bandpass filter that reflects only specific wavelengths (wavelengths of the light source) and transmits all others, and a polarizing beam splitter that reflects specific polarized light.

[0033] Examples of wavelength dispersion compensation members include a reflection or transmission volume hologram, a reflection or transmission relief hologram, a first optical element including a metasurface, etc. As described above, the first optical element including a metasurface as a wavelength dispersion compensation member may also have a structure in which, for example, dielectrics or metals are periodically arranged at narrow intervals on the surface of the first optical element, and can deflect light in a specific frequency band.

[0034] Hereinafter, preferred embodiments for carrying out the present technology will be described in detail with reference to the drawings. The embodiments described below are examples of typical embodiments of the present technology, and the scope of the present technology should not be interpreted as being narrow.

[0035] 2. First Embodiment (Example 1 of Display Device) A display device according to a first embodiment (example 1 of the display device) of the present technology is a display device including a light source, a processing unit that processes a characteristic distribution of an eyeball, a monitoring unit that monitors a state of the eyeball, a matching unit that matches the characteristic distribution of the eyeball with the state of the eyeball, and an irradiation unit that irradiates a predetermined location on the retina with image display light emitted from the light source. The display device according to the first embodiment of the present technology may further include a tracking unit. The display device according to the first embodiment of the present technology can be applied to, for example, eyewear displays, head-mounted displays, etc.

[0036] The processing unit processes eyeball characteristic distribution information obtained from an external device (e.g., a fundus camera, OCT, a light detection device that detects return light from an IR scan, a refractometer, etc.). The monitoring unit can monitor the state of the eyeball using, for example, the corneal reflex or fundus reflex and acquire the optical axis. The matching unit can acquire the deviation between the visual axis and the optical axis (Gaze Calibration) and create a map reflecting the offset. The illumination unit can modulate light according to the laser light source, image display (projector), and eyeball characteristic distribution, adjust the light intensity according to external light, and perform control including distortion correction. The tracking unit can track the displayed image in response to eye movement using eye tracking, steering the light beam to change the illuminated image and illumination position based on eyeball rotation information.

[0037] First, an example of a flow of video display by a display device according to a first embodiment of the present technology will be described with reference to FIG.

[0038] A fundus map is created in steps S101 to S103 shown in Fig. 1. More specifically, in step S101, a fundus camera or the like is used to take a frontal fundus photograph, in step S102, a fundus camera or the like is used to take fundus photographs while rotating up, down, left, and right, and in step S103, the positions of the fovea and blind spot are detected in each of the fundus photographs taken in steps S101 and S102.

[0039] Steps S101 to S103 will be described in detail with reference to FIG. 2. FIG. 2 is a diagram illustrating the change in the positions of the fovea and the blind spot as the right eyeball rotates left and right. FIG. 2A-2 in FIG. 2 is a fundus photograph of the right eye in frontal view taken in step S101, and FIG. 2B-2 is a top view schematically illustrating the right eyeball 20 in frontal view. FIG. 2A-1 in FIG. 2 is a fundus photograph of the right eye when rotated left in step S102, and FIG. 2B-1 is a top view schematically illustrating the right eyeball 20 when rotated left. FIG. 2A-3 in FIG. 2 is a fundus photograph of the right eye when rotated right in step S102, and FIG. 2B-3 is a top view schematically illustrating the right eyeball 20 when rotated right.

[0040] Comparing Figures 2A-1 and 2A-2, the fovea 30-1 shown in Figure 2A-1 moves to the left relative to the fovea 30-2 shown in Figure 2A-2, and similarly, the blind spot 31-1 shown in Figure 2A-1 also moves to the left relative to the blind spot 31-2 shown in Figure 2A-2.

[0041] Note that the positional relationship between the fovea 30-2 and the blind spot 31-2 shown in Figure 2A-2 (the distance between the fovea 30-2 and the blind spot 31-2) is approximately the same as the positional relationship between the fovea 30-1 and the blind spot 31-1 shown in Figure 2A-1 (the distance between the fovea 30-1 and the blind spot 31-1).

[0042] Comparing Figure 2A-3 with Figure 2A-2, the fovea 30-3 shown in Figure 2A-3 moves to a position to the right compared to the fovea 30-2 shown in Figure 2A-2, and similarly, the blind spot 31-3 shown in Figure 2A-3 also moves to a position to the right compared to the blind spot 31-2 shown in Figure 2A-2.

[0043] Note that the positional relationship between the fovea 30-2 and the blind spot 31-2 shown in Figure 2A-2 (the distance between the fovea 30-2 and the blind spot 31-2) is approximately the same as the positional relationship between the fovea 30-3 and the blind spot 31-3 shown in Figure 2A-3 (the distance between the fovea 30-3 and the blind spot 31-3).

[0044] 1, the eye-tracking result is matched with the fundus map. More specifically, in step S104, Gaze Calibration is performed using the display device according to the first embodiment of the present technology, and in step S105, the deviation between the optical axis center of the display system and the fovea is calculated.

[0045] Steps S104 and S105 will be described in detail with reference to Fig. 3. Fig. 3 is a diagram for explaining the result of matching the fundus map with the state of the eyeball to which an image is to be presented. As shown in Fig. 3A, an optical axis 1 (optical vector) and a visual axis (gaze vector) 2 are offset by an angle of θ. Optical axis 1 is a corneal normal passing through the center of pupil 10, and visual axis 2 is an axis connecting the nodal point (the central posterior surface of the crystalline lens) and fovea centralis 30.

[0046] As shown in FIG. 3B , the blind spot 31 exists across the first quadrant 51 and the fourth quadrant 54 of the image display range (angle of view) 50, which are made up of the first quadrant 51 (for example, a region where the X axis is positive and the Y axis is positive), the second quadrant 52 (for example, a region where the X axis is negative and the Y axis is positive), the third quadrant 53 (for example, a region where the X axis is negative and the Y axis is negative), and the fourth quadrant 54 (for example, a region where the X axis is positive and the Y axis is negative).

[0047] Steps S106 to S108 shown in FIG. 1 are performed while an image is being presented. More specifically, in step S106, gaze tracking is performed. For example, in step S106, the movement of the eyeball is tracked to detect the pupil position, calculate the angle of view of the desired point, and infrared light is placed on the imaging optical axis to detect the fundus reflection (the blind spot and the fovea have different reflectances). In step S107, an interpolated fundus map is generated. The interpolated fundus map is generated by interpolating between vertical and horizontal rotations of the eyeball. For example, the interpolated fundus map is generated by using the fundus map to estimate the positions of the fovea and blind spot when the eyeball is rotated vertically and horizontally at a level smaller than the vertical and horizontal rotation level used in the fundus map generated in steps S101 to S103. Note that the interpolated fundus map may be generated using position information of a specific eyeball or information on the angle of view deviation. Then, in step S108, an image is generated. After the image is generated, a signal of the generated image is sent to an image display unit including, for example, a light source (e.g., a laser light source), a mirror, a scanning mirror (e.g., a MEMS mirror), a relay system driver (projection optical system), and a see-through member placed in front of the eye (e.g., a holographic optical element (hereinafter sometimes referred to as an HOE)). Examples of holographic optical elements include a reflective or transmissive volume hologram, and a reflective or transmissive relief hologram (also called a surface relief hologram).

[0048] Steps S106 to S108 will be specifically described with reference to Fig. 4. Fig. 4 is a diagram for explaining the result (correction result) of matching the interpolated fundus map with the state of the eyeball to which an image is to be presented.

[0049] As shown in FIG. 4, among a first quadrant 51 (for example, a region where the X axis is positive and the Y axis is positive), a second quadrant 52 (for example, a region where the X axis is negative and the Y axis is positive), a third quadrant 53 (for example, a region where the X axis is negative and the Y axis is negative), and a fourth quadrant 54 (for example, a region where the X axis is positive and the Y axis is negative) that make up an image display range (angle of view) 50, the blind spot 31 is located in the first quadrant 51 that is closest to the fourth quadrant 54.

[0050] Steps S106 to S108 are repeated until a desired or predetermined image is generated, as indicated by reference symbol P1 in Fig. 1. Step S106 (Gaze Tracking) and step S107 (generation of an interpolated fundus map) may be performed simultaneously.

[0051] Another example of the flow of video display by the display device according to the first embodiment of the present technology will be described with reference to FIG.

[0052] A fundus map is created in step S201 shown in Fig. 5. The method for creating the fundus map in step S201 is the same as the method for creating the fundus map in steps S101 to S103 shown in Fig. 1 described above, and therefore a detailed description thereof will be omitted here.

[0053] In step S202 shown in FIG. 5, ocular characteristics are acquired. The ocular characteristics are acquired using, for example, OCT (Optical Coherence Tomography), a refractometer (objective refraction test), a light detection device that detects light returning from an IR scan, or the like. Using OCT, a cross section of the retina can be photographed. Using a refractometer, optical characteristics such as aberration (e.g., data related to ocular distortion) can be obtained, allowing for measurement of, for example, the refractive power of the eye and the curvature of the cornea. Using a light detection device that detects light returning from an IR scan, the shape of the fundus can be measured. By acquiring the ocular characteristics, it is possible to determine whether the user's eyes are affected by, for example, age-related macular degeneration. Age-related macular degeneration is a disease in which the macula, the central part of the retina, is damaged due to age-related changes, light damage, etc., resulting in impaired vision. For example, as shown in Figure 6, normal eyes see normally (Figure 6A), but age-related macular degeneration (metamorphopsia) can distort the retina, causing it to appear distorted, as in distortion 60B (Figure 6B), or age-related macular degeneration (metamorphopsia) can reduce vision, causing the center to appear dark, as in central scotoma 60C (Figure 6C).

[0054] In step S203 shown in Fig. 5, the eye-tracking results are matched with the fundus map. The matching method performed in step S203 is the same as the matching method performed in steps S104 to S105 shown in Fig. 1, as described above, and therefore a detailed description thereof will be omitted here.

[0055] Steps S204 to S208 shown in FIG. 5 are performed while the image is being presented. More specifically, in step S204, gaze tracking is performed. For example, in step S204, the movement of the eyeball is tracked to detect the pupil position, and the angle of view of the desired point is calculated. Also, infrared light is placed on the imaging optical axis to detect the fundus reflection (the blind spot and the fovea have different reflectances). In step S205, an interpolated fundus map is generated. The interpolated fundus map is generated by interpolating between vertical and horizontal rotations of the eyeball. For example, the interpolated fundus map is generated by using the fundus map to estimate the positions of the fovea and blind spot when the eyeball is rotated vertically and horizontally at a level smaller than the vertical and horizontal rotation level used in the fundus map generated in step S201. Note that the interpolated fundus map may be generated using position information of a specific eyeball or information on the angle of view deviation.

[0056] Next, in step S206, the eyeball distortion correction table is reconstructed. Step S206 will be specifically described with reference to Fig. 7. Fig. 7A shows a video (image) to be presented. Fig. 7B shows a video (image) caused by eyeball distortion. Fig. 7C shows an output video (generated video) corrected using the eyeball distortion correction table.

[0057] In step S207, eye movement unevenness is corrected. Step S207 will be specifically described with reference to FIG. 8. FIG. 8A shows a video (image) to be presented. FIG. 8B shows a video (image) caused by eye movement unevenness. FIG. 8C shows an output video (generated video) corrected by eye movement unevenness correction.

[0058] Then, in step S208, an image is generated. After the image is generated, a signal of the generated image is sent to an image display unit including, for example, a light source (for example, a laser light source), a mirror, a scanning mirror (for example, an MEMS mirror), a relay system driver (projection optical system), and a see-through member (for example, a holographic optical element (hereinafter sometimes referred to as an HOE)). Examples of the holographic optical element include a reflective or transmissive volume hologram, and a reflective or transmissive relief hologram (also called a surface relief hologram).

[0059] Steps S204 to S208 are repeated until a desired or predetermined image is generated, as indicated by reference character P5 in Fig. 5. Step S204 (Gaze Tracking) and step S205 (generation of an interpolated fundus map) may be performed simultaneously.

[0060] An example of the configuration of a display device according to a first embodiment of the present technology will be described with reference to FIGS. 9 and 10. FIG. 9 is a top view of a display device according to the present technology worn on a user's head. FIG. 10 is a front view of a display device according to the present technology worn on a user's head. The display device shown in FIG. 9 includes a video display unit (also referred to as an image display unit), a sensor that detects a change in the position of the display device relative to the head (in this specification, a sensor that detects a change in the position of the display device relative to the head is also referred to as a "displacement sensor" or "sensor"), a monitoring unit, a projection position adjustment mechanism, a control unit, and a storage unit. These components will be described below.

[0061] (Video display section) As shown in FIG. 9 , the display device 100 has a eyeglass-like shape and is configured to project image display light (sometimes referred to as image display light) onto each eye. That is, the display device 100 includes an image display unit that projects image display light onto the left eye and an image display unit that projects image display light onto the right eye. The image display unit that projects image display light onto the left eye includes a light source unit 101L, a projection optical system 102L, and a holographic optical element (hereinafter also referred to as HOE) 103L as an irradiation unit. Although not shown, the display device 100 may include a combiner as a tracking unit, and the combiner may structurally include the holographic optical element 103L, a half mirror, etc. A relay system drive unit (not shown), which is the tracking unit, may be included in the projection optical system 102L. Although not shown, the mirror driver and the phase difference panel, which are the tracking units, may be disposed between the light source unit 101L and the projection optical system 102L or between the projection optical system 102L and the holographic optical element 103L.

[0062] The light source unit 101L emits image display light. As a configuration for emitting the image display light, the light source unit 101L may include, for example, a laser light source 120L, a mirror 121L, and a scanning mirror 122L. The laser light emitted from the laser light source 120L is reflected by the mirror 121L and reaches the scanning mirror 122L. The scanning mirror 122L scans the laser light two-dimensionally. The scanning mirror 122L may be, for example, a MEMS mirror. The scanning mirror 122L can move the direction of the laser light at high speed so that an image is formed on the retina.

[0063] The projection optical system 102L adjusts the direction of the image display light so that the image display light reaches a desired region and / or position on the HOE 103L. For example, the image display light scanned by the scanning mirror 122L is converted into parallel light.

[0064] The HOE 103L diffracts the image display light so that it is focused near the user's pupil and illuminated on the retina. The HOE 103L may be, for example, a reflective diffraction element. The HOE 103L may have optical properties such that it acts as a lens for light within the wavelength range of the image display light and transmits light with wavelengths outside that range. These optical properties allow the user to recognize, for example, a scene ahead in the line of sight through the HOE 103L and recognize an image formed by the image display light. In other words, the image formed by the image display light can be superimposed on the external scenery. The HOE 103L may be, for example, a hologram lens, preferably a film-type hologram lens, and more preferably a transparent film-type hologram lens. The film-type hologram lens may be attached to, for example, glass. Desired optical properties can be imparted to the hologram lens using techniques known in the art. The HOE 103L may also be, for example, a volume hologram or a surface relief hologram. The hologram lens may be a commercially available hologram lens, or may be manufactured by a technique known in the art.

[0065] As described above, the light source unit 101L, the projection optical system 102L, and the HOE 103L cause image display light to reach the left eye of the user.

[0066] The display device 100 has temples 109L and rims 108L that are part of a pair of eyeglasses. A light source 101L and a projection optical system 102L are disposed in the temples 109L. An HOE 103L is held in the rims 108L. More specifically, an inner rim 106L is held in the rims 108L via a projection position adjustment mechanism 105L-2 serving as an irradiation unit, and an HOE 103L is held in the inner rim 106L via a projection position adjustment mechanism 105L-1 serving as an irradiation unit.

[0067] The image display unit that projects image display light onto the user's right eye includes a light source unit 101R, a projection optical system 102R, and an HOE 103R as an illumination unit. Although not shown, a combiner may be included in the display device 100 as a tracking unit, and the combiner may structurally include the HOE 103R, a half mirror, etc. A relay system driver (not shown), which is a tracking unit, may be included in the projection optical system 102R. Although not shown, a mirror driver and a retardation panel, which are tracking units, may be disposed between the light source unit 101R and the projection optical system 102R or between the projection optical system 102R and the HOE 103R.

[0068] The explanations regarding the light source unit 101L, the projection optical system 102L, and the HOE 103L also apply to the light source unit 101R, the projection optical system 102R, and the HOE 103R.

[0069] Similar to the image display unit for the left eye, light source unit 101R and projection optical system 102R are arranged in temple unit 109R. HOE 103R is held in rim unit 108R. More specifically, inner rim unit 106R is held in rim unit 108R via projection position adjustment mechanism 105R-2 as an irradiation unit, and HOE 103R is held in inner rim unit 106R via projection position adjustment mechanism 105R-1 as an irradiation unit.

[0070] The rim portions 108L and 108R of the display device 100 are connected via a bridge portion 110. The bridge portion 110 is the portion that rests on the user's nose when the user wears the display device 100. In addition, both the rim portions 108L and 108R of the display device 100 are connected to a headband portion 111. The headband portion 111 is the portion that comes into contact with the top of the user's head when the user wears the display device 100, as shown in FIG. 10 .

[0071] Although the light source unit 101L shown in FIG. 9 includes one laser light source 120L, the number of laser light sources included in the light source unit 101L may be two or more, for example, two to five. These multiple laser light sources may output laser light of different wavelengths. Similarly, the light source unit 101R includes one laser light source 120R, but the number of laser light sources included in the light source unit 101R may be two or more, for example, two to five. These multiple laser light sources may output laser light of different wavelengths. By using the laser light source 120L and the laser light source 120R, a stimulus of a specific wavelength can be presented.

[0072] Although not shown, the display device 100 may further include a wavelength dispersion compensation member. The wavelength dispersion compensation member may be, for example, a reflection or transmission volume hologram, a reflection or transmission relief hologram, a first optical element including a metasurface, or the like. The wavelength dispersion compensation member may be disposed around the mirror 121L and / or 121R, for example, between the mirror 121L and the scanning mirror 122L and / or between the mirror 121R and the scanning mirror 122R. When the wavelength dispersion compensation member is used in the display device 100, wavelength dispersion is compensated for, so that an arbitrary point (predetermined point) on the retina can be accurately stimulated.

[0073] (sensor) The display device 100 further includes sensors 104L, 104R, 104C, and 104T that detect a change in the position of the display device 100 relative to the user's head. The change in position detected by these sensors may be, for example, the direction of the change in position and / or the amount of the change in position. Note that, in this specification, the sensors 104L, 104R, 104C, and 104T may be collectively referred to as sensors 104.

[0074] Sensors 104L and 104R detect a change in the horizontal position of display device 100 relative to the user's head, sensor 104C detects a change in the front-to-back position of display device 100 relative to the user's head, and sensor 104T detects a change in the up-to-down position of display device 100 relative to the user's head. This makes it possible to grasp the wearing misalignment three-dimensionally.

[0075] (Monitoring unit (gaze detection device)) The display device 100 includes monitoring units 107L and 107R that detect the user's line of sight. In this specification, the monitoring units 107L and 107R are sometimes collectively referred to as the monitoring unit 107. The monitoring unit 107 can monitor the state of the eyeball using corneal reflex or fundus reflex. The inclusion of these monitoring units in the display device 100 enables the position of an image presented to the user to be adjusted to a more appropriate position. For example, when an image presented by the display device 100 is superimposed on an image of the outside world, the image can be displayed in a more appropriate position by detecting the user's line of sight. In other words, the inclusion of the monitoring unit 107 is preferable for presenting AR information. The monitoring unit may be configured, for example, as a line of sight detection device.

[0076] The monitoring unit 107 may be, for example, an imaging type monitoring unit or a photodiode type monitoring unit, which will be described in more detail below.

[0077] The monitor 107L detects the line of sight of the user's left eye. The monitor 107L may be provided, for example, at any position on the rim 108L, but may also be provided at any position on another element (such as the inner rim 106L) as long as it can detect the line of sight of the left eye.

[0078] The monitoring unit 107L may be, for example, a photodiode-based monitoring unit. The photodiode-based monitoring unit may include, for example, a combination of a light source and a photodiode. The light source is configured to irradiate light onto the left eye. The light source is preferably an infrared light source. This prevents the light from affecting the user's recognition of images of the outside world and the image display light. The photodiode may be configured to detect light (especially infrared light) emitted from the light source reflected by the eyeball. The photodiode may be capable of detecting, for example, the difference between the amount of light reflected by the iris (pupil) and the amount of light reflected by the white of the eye (sclera). The photodiode-based monitoring unit may detect the line of sight based on, for example, the area ratio of the iris and the area ratio of the white of the eye detected by the photodiode.

[0079] A photodiode-based monitoring unit cannot detect misalignment even if the device is misaligned. Therefore, if misalignment occurs, the gaze detection accuracy of the monitoring unit may decrease. As described above, the display device of the present technology is equipped with a sensor that detects changes in the position of the display device relative to the head, thereby enabling detection of misalignment. By performing gaze correction based on the misalignment detected by the sensor, the gaze detection accuracy of the monitoring unit is improved. The display device of the present technology can detect the gaze with an accuracy of, for example, 3 mm or less, particularly 2 mm or less, and more particularly 1 mm or less. Gaze detection with such accuracy is particularly preferable for image presentation using Maxwellian vision.

[0080] Alternatively, the monitoring unit 107L may be an imaging-type monitoring unit. The imaging-type monitoring unit may include, for example, a combination of a light source and an imaging element. The light source is configured to irradiate the left eye with light, as in the case of a photodiode-type monitoring unit. The light source is preferably an infrared light illumination light source. The imaging element may be configured to obtain, for example, a reflection image of the light source on the eyeball (particularly the cornea) (a so-called Purkinje image) and an image from which the center of gravity of the pupil can be obtained. The imaging element may be, for example, an infrared imaging element. The imaging-type monitoring unit may estimate the optical axis of the eyeball, for example, based on the Purkinje image and the image. The monitoring unit may convert the estimated optical axis into a visual axis to detect the line of sight.

[0081] In gaze detection based on the Purkinje image and the image, if the positional relationship between the light source and the eyeball is fixed, the position of the Purkinje image is fixed. Wearing misalignment causes a misalignment in this positional relationship, which can change the position of the Purkinje image. In addition, gaze detection is easily affected by blinking, hair, eyelashes, etc. Furthermore, while calibration is typically performed to correct for individual differences in gaze detection, recalibration is required if wearing misalignment occurs. As described above, the display device of the present technology includes a sensor that detects changes in the position of the display device relative to the head, thereby enabling wearing misalignment to be detected. Therefore, for example, by preparing a correction value corresponding to the amount of wearing misalignment in advance (e.g., stored in a memory unit, etc.) and performing correction using the correction value when wearing misalignment occurs, gaze detection can be performed with high accuracy. In addition, detection of wearing misalignment is less affected by blinking, hair, eyelashes, etc. Furthermore, performing correction based on detected wearing misalignment can reduce the number of calibrations required.

[0082] (Projection position adjustment mechanism) The irradiation unit included in the display device 100 may further include projection position adjustment mechanisms 105L-1 and 105L-2, and 105R-1 and 105R-2 that adjust the projection position of the image display light emitted from the display device 100. Note that in this specification, these four projection position adjustment mechanisms may be collectively referred to as the projection position adjustment mechanism 105. The projection position adjustment mechanism 105 may be configured to adjust the projection position of the image display light by following the line of sight, for example. The projection position adjustment mechanism 105 can adjust the projection position of the image display light in accordance with a mounting misalignment.

[0083] In addition, the projection position adjustment mechanism 105 can adjust the projection position of the image display light in accordance with the rotational movement of the eyeball or the movement of the line of sight. For example, by including the projection position adjustment mechanism 105 in the display device 100, it is possible to adjust the position of the image presented to the user to a more appropriate position. For example, when an image presented by the display device 100 is superimposed on an image of the outside world, the image can be displayed in a more appropriate position by detecting the user's line of sight. In other words, including the monitoring unit 107 is preferable for presenting AR information. Furthermore, these projection position adjustment mechanisms can also adjust the position at which the image display light is focused in image display using Maxwellian vision.

[0084] The projection position adjustment mechanisms 105L-1 and 105L-2 adjust the projection position of the image display light projected to the left eye. The projection position adjustment mechanism 105L-1 adjusts the positional relationship in the z-axis direction between the inner rim portion 106L and the rim portion 108L. For example, the projection position adjustment mechanism 105L-1 moves the inner rim portion 106L in the z-axis direction relative to the rim portion 108L. This adjusts the position of the HOE 103L in the z-axis direction. The projection position adjustment mechanism 105L-2 adjusts the positional relationship in the x-axis direction between the HOE 103L and the inner rim portion 106L. For example, the projection position adjustment mechanism 105L-2 moves the HOE 103L in the x-axis direction relative to the inner rim portion 106L. This adjusts the position of the HOE 103L in the x-axis direction.

[0085] The driving element for driving the projection position adjustment mechanism 105L-1 to adjust the positional relationship between the inner rim portion 106L and the rim portion 108L in the z-axis direction may be, for example, a piezoelectric element, an actuator, or a bimetal, but is not limited to these. The driving element for driving the projection position adjustment mechanism 105L-2 to adjust the positional relationship between the HOE 103L and the inner rim portion 106L in the x-axis direction may also be, for example, a piezoelectric element, an actuator, or a bimetal, but is not limited to these.

[0086] The projection position adjustment mechanism 105L-1 may adjust the positional relationship between the inner rim portion 106L and the rim portion 108L in the z-axis direction based on, for example, a change in the position of the display device 100 detected by one, two, three, or all four of the sensors 104L, 104R, 104C, and 104T. The projection position adjustment mechanism 105L-1 may also adjust the positional relationship based on the change in position and the line of sight detected by the monitoring unit 107L. The projection position adjustment mechanism 105L-2 may adjust the positional relationship between the HOE 103L and the inner rim portion 106L in the x-axis direction based on, for example, a change in the position of the display device 100 detected by one, two, three, or all four of the sensors 104L, 104R, 104C, and 104T. Furthermore, the projection position adjustment mechanism 105L-2 may adjust the positional relationship based on the change in the position and the line of sight detected by the monitoring unit 107L.

[0087] The projection position adjustment mechanisms 105R-1 and 105R-2 adjust the projection positions of the image display light projected onto the right eye, and this adjustment may be performed in the same manner as the projection position adjustment mechanisms 105L-1 and 105L-2.

[0088] (Control unit and memory unit) The display device 100 includes a control unit 112. As shown in FIG. 11 , which is a block diagram illustrating the main components of the display device 100, the control unit 112 includes an image control unit 181, a projection position control unit 182, a line-of-sight correction unit 183, a processing unit 191, and a matching unit 192. As described above, the display device 100 shown in FIG. 11 includes a light source unit 101, a sensor 104, a projection position adjustment mechanism 105 as an irradiation unit, a monitoring unit (line-of-sight detection mechanism) 107, an irradiation unit 201 including a see-through member (e.g., a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, etc.) arranged in front of the eye, and a tracking unit 202. Note that the display device 100 does not necessarily have to include the tracking unit 202. The storage unit 184 may be included in the display device 100, or may be included in an external device other than the display device 100.

[0089] The members arranged in front of the eyes and provided in the projection unit 201 of the display device 100 have been described above, but the case will be described in more detail using FIGS. 13 and 14.

[0090] Fig. 13 is a diagram showing a second optical element 300, which is an example of a member placed in front of the eye and reflects predetermined light and transmits light other than the predetermined light. Fig. 14 is a diagram showing a second optical element 400, which is another example of a member placed in front of the eye and reflects predetermined light and transmits light other than the predetermined light.

[0091] The device shown in Fig. 13 includes a second optical element 300, a liquid crystal panel 301, and a special liquid crystal lens 302. The second optical element 300 is made up of a polarizing beam splitter (PBS), and although not shown, a λ / 4 plate is disposed in the second optical element (polarizing beam splitter) 300. The device shown in Fig. 13 does not necessarily include the special liquid crystal lens 302; instead, a lens may be disposed between the liquid crystal panel 301 and the second optical element 300.

[0092] As shown in FIG. 13, polarized light (image display light) L31 from the liquid crystal panel 310 is reflected by region H31 of the second optical element 300, passes through regions T31-1 and T31-2 of the special liquid crystal lens 302, and reaches the user's eye 310 (pupil 310-1), and polarized light (image display light) L32 from the liquid crystal panel 301 is reflected by region H32 of the second optical element (polarizing beam splitter) 300, passes through regions T32-1 and T32-2 of the special liquid crystal lens 302, and reaches the user's eye 310 (pupil 310-1), and the user perceives a virtual image (picture) V13 based on the light (image display light (polarized light L31 and L32)) emitted from the liquid crystal panel 301. Then, light L310 and L320 from the outside world passes through the second optical element 300 (polarizing beam splitter), passes through the special liquid crystal lens 302, and reaches the user's eye 310 (pupil 310-1), and the user recognizes a reality video (image) R13 due to the light L310 and 320 from the outside world. That is, the user recognizes (sees) it as if the virtual video (image) V13 and the reality video (image) R13 are superimposed on each other.

[0093] FIG. 14 shows a second optical element 400, an OLED (Organic Light Emitting Diode) panel (organic EL) 401, and a lens 402. The second optical element 400 is an aspherical half mirror and is composed of a first mirror member 400-1 and a second mirror member 400-2. For example, the second optical element 400 may be a combination of a half mirror 400-1 and a half mirror 400-2, or a combination of a polarizing beam splitter (PBS) 400-1 and a half mirror 400-2. When the second optical element 400 is a combination of a polarizing beam splitter (PBS) 400-1 and a half mirror 400-2, a λ / 4 plate may be disposed in the half mirror 400-2. When the first mirror member 400-1 and / or the second mirror member 400-2 are half mirrors, it is preferable that the light transmittance be 50% or more.

[0094] As shown in FIG. 14 , light (image display light) L41 from an OLED (Organic Light Emitting Diode) panel (organic EL) 401 is reflected by a region H41-1 of a first mirror member 400-1 constituting the second optical element 400, then reflected by a region H41-2 of a second mirror member 400-2 constituting the second optical element 400, passes through a region T41 of the first mirror member 400-1, and reaches the user's eye 410 (pupil 410-1). Light (image display light) L42 from the OLED panel 401 is reflected by a region H42-1 of the first mirror member 400-1 constituting the second optical element 400, then reflected by a region H42-2 of the second mirror member 400-2 constituting the second optical element 400, and then Light ray (image display light) L43 from the OLED panel 401 passes through region T42 of the first mirror member 400-1 and reaches the user's eye 410 (pupil 410-1), is reflected by region H43-1 of the first mirror member 400-1 that constitutes the second optical element 400, is then reflected by region H43-2 of the second mirror member 400-2 that constitutes the second optical element 400, passes through region T43 of the first mirror member 400-1, and reaches the user's eye 410 (pupil 410-1), and the user recognizes a virtual image (picture) made up of light (image display light (light rays L41, L42, and L43)) emitted from the OLED panel 401. Although not shown, light from the outside world passes through the second optical element 400 and reaches the user's eye 410 (pupil 410-1), and the user perceives a reality video (image) created by the light from the outside world. The user perceives the virtual video (image) and the reality video (image) as being superimposed on each other.

[0095] As described above, the two examples of components that are provided in the irradiation unit 201 and placed in front of the eyes, which were explained using Figures 13 and 14, can also be applied to the display device 100-1, which is the second embodiment of the present technology described below.

[0096] The control unit 112 will be described below.

[0097] The image control unit 181 controls the projection of image display light by the image display unit. The image control unit 181 drives, for example, the light source units 101L and 101R, particularly the laser light sources and scanning mirrors included in these light source units, to output the image display light. The image control unit 181 may acquire image data stored in, for example, the storage unit 184, and cause the light source units 101L and 101R to output the image display light based on the image data. The image control unit 181 may correct the image data based on a change in the position of the display device 100 relative to the head, detected by the sensor 104. The image control unit 181 may cause the light source units 101L and 101R to output the image display light based on the corrected image data. In other words, the display device 100 may correct the image based on a change in position detected by a sensor that detects a change in the position of the display device relative to the head.

[0098] The projection position control unit 182 controls the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2, thereby controlling the projection position of the image display light. For example, the projection position control unit 182 may adjust the projection position of the image display light by driving one to four of the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2 based on the lines of sight detected by the monitoring units 107L and 107R. For example, the projection position of the image display light may be adjusted to follow the lines of sight. The projection position control unit 182 may adjust the projection position of the image display light by driving one to four of the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2 based on the lines of sight corrected by the line of sight correction unit 183, which will be described later. For example, the projection position of the image display light can be adjusted so as to follow the line of sight after the correction. The projection position control unit 182 may adjust the projection position of the image display light by driving one to four of the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2 based on data relating to a change in the position of the display device 100 relative to the head (hereinafter also referred to as "displacement data") detected by one to four of the sensors 104L, 104R, 104C, and 104T.

[0099] The projection position control unit 182 may calculate the amount of position adjustment by each projection position adjustment mechanism based on, for example, the displacement data and the correction coefficient. The projection position control unit 182 may drive each projection position adjustment mechanism so that the positional relationship is changed by the calculated amount of position adjustment. The projection position control unit 182 may obtain a correction coefficient from, for example, a correction table stored in advance in the storage unit 184 and use the correction coefficient to calculate the amount of position adjustment. The correction table may include, for example, multiple correction coefficients, and the projection position control unit 182 may select a predetermined correction coefficient from among the multiple correction coefficients in accordance with the displacement data. Furthermore, a correction table may be provided for each projection position adjustment mechanism, for example. The correction table may be provided in advance in the display device 100 or may be updated according to the user's use of the display device 100. The accuracy of projection position control can be improved by selecting or updating the correction table or correction coefficient. The projection position control unit 182 may use the line of sight detected by the monitoring unit or the line of sight corrected by the line of sight correction unit 183 to calculate the amount of position adjustment.

[0100] The gaze correction unit 183 corrects the gazes detected by the monitoring units 107L and 107R based on the displacement data. This enables the gaze correction unit 183 to identify the gazes taking into account the wearing misalignment, thereby improving the accuracy of gaze detection. The correction may be performed on the optical axis of the eyeball, the visual axis of the eyeball, or another reference axis. The gaze correction unit 183 may also obtain a correction coefficient from a correction table pre-stored in the storage unit 184, for example, and use the correction coefficient for the gaze correction. The correction table may have, for example, multiple correction coefficients, and the gaze correction unit 183 may select a predetermined correction coefficient from these multiple correction coefficients depending on the displacement data. The correction table may be pre-installed in the display device 100 or may be updated as the user uses the head-mounted display device 100. Selecting or updating the correction table or correction coefficients can improve the accuracy of gaze correction.

[0101] Display device 100 may further include a storage unit 184. The storage unit may store data related to the image display light projected by the image display unit, a correction table used for control of the projection position by projection position control unit 122, and a correction table used for line of sight correction by line of sight correction unit 123.

[0102] The processing unit 191 processes eyeball characteristic distribution information. The eyeball characteristic distribution may be acquired, for example, using a fundus camera, or using a light detection device that detects return light from an OCT, a refractometer, or an IR scan. The eyeball characteristic distribution may be acquired from geometrical information (e.g., the size and shape of the eyeball, the size and shape of the macula (e.g., the shape of the depression), the size and shape of the blind spot (e.g., the shape of the depression), the retinal tomographic shape (e.g., the uneven shape)), qualitative information (e.g., the position of the fovea, the position of the blind spot, characteristics of the lesion site, etc.), or optical information (e.g., refraction within the eyeball, aberration, etc.). The processing unit 191 calculates coordinates based on the eyeball characteristic distribution. The coordinate system can be defined. Note that defining the coordinate system based on the characteristic distribution of the eyeball does not have to be performed by the processing unit 191, and for example, a defining unit for defining the coordinate system may be provided in the control unit 112, independent of the processing unit 191. The coordinate system is defined based on at least two selected from the group consisting of the first fovea of ​​the right eye, the first blind spot of the right eye, the second fovea of ​​the left eye, and the second blind spot of the left eye. Since the display image and display position are defined using the coordinate system, less data is required than when a pattern map is used. Furthermore, the processing unit 191 may control the display image and display position based on eyeball rotation information, in which case the processing load can be reduced compared to pattern matching.

[0103] The matching unit 192 matches the eyeball characteristic distribution with the eyeball state. For example, the matching unit 192 matches the eyeball characteristic distribution with the state of the eyeball to which an image is to be presented (the eyeball to which an image is to be presented may be referred to as the real eye) via the coordinate system described above. The matching unit 192 can acquire the deviation between the visual axis and the optical axis (Gaze Calibration) and create a map that reflects the offset.

[0104] The above description of the display device of the first embodiment (example 1 of display device) according to the present technology can be applied to the display device of the second embodiment (example 2 of display device) according to the present technology, the display method of the third embodiment (example 1 of display method) according to the present technology, and the display method of the fourth embodiment (example 2 of display method) according to the present technology, as described below, unless there is any particular technical contradiction.

[0105] 3. Second Embodiment (Example 2 of Display Device) A display device according to a second embodiment (example 2 of the display device) of the present technology is a display device including a light source, a processing unit that processes a characteristic distribution of the eyeball, a monitoring unit that monitors a state of the eyeball, a matching unit that matches the characteristic distribution of the eyeball with the state of the eyeball, an irradiation unit that irradiates a predetermined location on the retina with image display light emitted from the light source, and further includes an acquisition unit that acquires the characteristic distribution of the eyeball. That is, the display device according to the second embodiment of the present technology is a display device that adds an acquisition unit that acquires the characteristic distribution of the eyeball to the display device according to the first embodiment of the present technology. The display device according to the second embodiment of the present technology may further include a tracking unit. The display device according to the second embodiment of the present technology can be applied to, for example, eyewear displays, head-mounted displays, etc.

[0106] The processing unit processes eyeball characteristic distribution information obtained from an acquisition unit (e.g., a fundus camera, OCT, a light detection device that detects return light from an IR scan, a refractometer, etc.). The monitoring unit can monitor the state of the eyeball using, for example, the corneal reflex or fundus reflex and acquire the optical axis. The matching unit can acquire the deviation between the visual axis and the optical axis (Gaze Calibration) and create a map reflecting the offset. The illumination unit can modulate light according to the laser light source, image display (projector), and eyeball characteristic distribution, adjust the light intensity according to external light, and perform control including distortion correction. The tracking unit can track the displayed image in response to eye movement using eye tracking, steering the light beam to change the illuminated image and illumination position based on eyeball rotation information.

[0107] A configuration example of a display device according to a second embodiment of the present technology will be described with reference to FIG.

[0108] FIG. 12 is a block diagram showing main components of a display device (display device 100-1) according to a second embodiment of the present technology.

[0109] The display device 100-1 includes a light source unit 101, a sensor 104, a projection position adjustment mechanism 105 as an irradiation unit, a monitoring unit (gaze detection mechanism) 107, an irradiation unit 201 including a see-through member (for example, a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, etc.) placed in front of the eye, a tracking unit 202, an acquisition unit 301, and a control unit 112. Note that the display device 100-1 does not necessarily have to include the tracking unit 202.

[0110] The control unit 112 includes an image control unit 181 , a projection position control unit 182 , a line of sight correction unit 183 , a processing unit 191 , and a matching unit 192 .

[0111] The image control unit 181 controls the projection of image display light by the image display unit. The image control unit 181 drives, for example, the light source units 101L and 101R, particularly the laser light sources and scanning mirrors included in these light source units, to output the image display light. The image control unit 181 may acquire image data stored in, for example, the storage unit 184, and cause the light source units 101L and 101R to output the image display light based on the image data. The image control unit 181 may correct the image data based on a change in the position of the display device 100 relative to the head, detected by the sensor 104. The image control unit 181 may cause the light source units 101L and 101R to output the image display light based on the corrected image data. In other words, the display device 100 may correct the image based on a change in position detected by a sensor that detects a change in the position of the display device relative to the head.

[0112] The projection position control unit 182 controls the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2, thereby controlling the projection position of the image display light. For example, the projection position control unit 182 may adjust the projection position of the image display light by driving one to four of the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2 based on the lines of sight detected by the monitoring units 107L and 107R. For example, the projection position of the image display light may be adjusted to follow the lines of sight. The projection position control unit 182 may adjust the projection position of the image display light by driving one to four of the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2 based on the lines of sight corrected by the line of sight correction unit 183, which will be described later. For example, the projection position of the image display light can be adjusted so as to follow the line of sight after the correction. The projection position control unit 182 may adjust the projection position of the image display light by driving one to four of the projection position adjustment mechanisms 105L-1, 105L-2, 105R-1, and 105R-2 based on data relating to a change in the position of the display device 100 relative to the head (hereinafter also referred to as "displacement data") detected by one to four of the sensors 104L, 104R, 104C, and 104T.

[0113] The projection position control unit 182 may calculate the amount of position adjustment by each projection position adjustment mechanism based on, for example, the displacement data and the correction coefficient. The projection position control unit 182 may drive each projection position adjustment mechanism so that the positional relationship is changed by the calculated amount of position adjustment. The projection position control unit 182 may obtain a correction coefficient from, for example, a correction table stored in advance in the storage unit 184 and use the correction coefficient to calculate the amount of position adjustment. The correction table may include, for example, multiple correction coefficients, and the projection position control unit 182 may select a predetermined correction coefficient from among the multiple correction coefficients in accordance with the displacement data. Furthermore, a correction table may be provided for each projection position adjustment mechanism, for example. The correction table may be provided in advance in the display device 100 or may be updated according to the user's use of the display device 100. The accuracy of projection position control can be improved by selecting or updating the correction table or correction coefficient. The projection position control unit 182 may use the line of sight detected by the monitoring unit or the line of sight corrected by the line of sight correction unit 183 to calculate the amount of position adjustment.

[0114] The gaze correction unit 183 corrects the gazes detected by the monitoring units 107L and 107R based on the displacement data. This enables the gaze correction unit 183 to identify the gazes taking into account the wearing misalignment, thereby improving the accuracy of gaze detection. The correction may be performed on the optical axis of the eyeball, the visual axis of the eyeball, or another reference axis. The gaze correction unit 183 may also obtain a correction coefficient from a correction table pre-stored in the storage unit 184, for example, and use the correction coefficient for the gaze correction. The correction table may have, for example, multiple correction coefficients, and the gaze correction unit 183 may select a predetermined correction coefficient from these multiple correction coefficients depending on the displacement data. The correction table may be pre-installed in the display device 100 or may be updated as the user uses the head-mounted display device 100. Selecting or updating the correction table or correction coefficients can improve the accuracy of gaze correction.

[0115] Display device 100 may further include a storage unit 184. The storage unit may store data related to the image display light projected by the image display unit, a correction table used for control of the projection position by projection position control unit 122, and a correction table used for line of sight correction by line of sight correction unit 123.

[0116] The processing unit 191 processes eyeball characteristic distribution information. The eyeball characteristic distribution can be acquired by the acquisition unit 301. For example, the eyeball characteristic distribution can be acquired using a fundus camera included in the acquisition unit 301, or using a light detection device that detects returned light from an OCT, a refractometer, or an IR scan included in the acquisition unit 301. The eyeball characteristic distribution can be acquired from geometrical information (e.g., the size and shape of the eyeball, the size and shape of the macula (e.g., the shape of the depression), the size and shape of the blind spot (e.g., the shape of the depression), and the tomographic shape of the retina (e.g., the uneven shape)), qualitative information (e.g., the position of the fovea, the position of the blind spot, characteristics of the lesion site, etc.), and optical information (e.g., refraction, aberration, etc. within the eyeball). The processing unit 191 can define a coordinate system based on the eyeball characteristic distribution. Note that the definition of the coordinate system based on the eyeball characteristic distribution may not be performed by the processing unit 191, and for example, a definition unit for defining the coordinate system may be provided in the control unit 112, independent of the processing unit 191. The coordinate system is defined based on at least two selected from the group consisting of the first fovea of ​​the right eye, the first blind spot of the right eye, the second fovea of ​​the left eye, and the second blind spot of the left eye. Since the display image and display position are defined using the coordinate system, less data is required than when a pattern map is used. Furthermore, the processing unit 191 may control the display image and display position based on eyeball rotation information, in which case the processing load can be reduced compared to pattern matching.

[0117] The matching unit 192 matches the eyeball characteristic distribution with the eyeball state. For example, the matching unit 192 matches the eyeball characteristic distribution with the state of the eyeball to which an image is to be presented (the eyeball to which an image is to be presented may be referred to as the real eye) via the coordinate system described above. The matching unit 192 can acquire the deviation between the visual axis and the optical axis (Gaze Calibration) and create a map that reflects the offset.

[0118] 4. Third Embodiment (Example 1 of Display Method) A display method according to a third embodiment (first example of display method) of the present technology is a display method including: processing a characteristic distribution of the eyeball; monitoring a state of the eyeball; matching the characteristic distribution of the eyeball with the state of the eyeball; and irradiating a predetermined location on the retina with image display light emitted from a light source.

[0119] The display method of the third embodiment (example 1 of the display method) according to the present technology is executed using the display device (for example, the display device 100) of the first embodiment (example 1 of the display device) according to the present technology.

[0120] 1 and 5 may be applied to the flow of the display method of the third embodiment (example 1 of the display method) according to the present technology. Step S101 (taking a frontal fundus photograph), step S102 (taking fundus photographs when rotated up, down, left, and right), and step S103 (detecting the fovea and blind spot positions in each fundus photograph) shown in Fig. 1, and step S201 (creating a fundus map) and step S202 (acquiring eyeball characteristics) shown in Fig. 5 may be executed using an external device (e.g., a fundus camera, an OCT, a refractometer, etc.) other than the display device (e.g., the display device 100) of the first embodiment (example 1 of the display device) according to the present technology.

[0121] 5. Fourth embodiment (Example 2 of display method) A display method according to a fourth embodiment (example 2 of the display method) of the present technology includes processing an eyeball characteristic distribution, monitoring a state of the eyeball, matching the eyeball characteristic distribution with the state of the eyeball, and irradiating a predetermined location on the retina with image display light emitted from a light source, and further includes acquiring the eyeball characteristic distribution. That is, the display method according to the fourth embodiment of the present technology is a display method that adds acquiring the eyeball characteristic distribution to the display method according to the third embodiment of the present technology.

[0122] The display method according to the fourth embodiment (example 2 of the display method) of the present technology is executed using the display device (for example, the display device 100-1) according to the second embodiment (example 2 of the display device) of the present technology.

[0123] The contents described above with reference to FIGS. 1 and 5 can be applied to the flow of the display method of the fourth embodiment (example 2 of the display method) according to the present technology.

[0124] It should be noted that the embodiments according to the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0125] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0126] The present technology can also be configured as follows. [1] A light source and a processing unit for processing the eyeball characteristic distribution; a monitoring unit that monitors the state of the eyeball; a matching unit that matches the eyeball characteristic distribution with the eyeball state; an irradiation unit that irradiates a predetermined location on the retina with image display light emitted from the light source; A display device comprising: [2] The display device according to [1], further comprising an acquisition unit that acquires the characteristic distribution of the eyeball. [3] The display device according to [2], wherein the acquisition unit is composed of at least one selected from the group consisting of a fundus camera, an OCT, a refractometer, and a light detection device that detects return light from an IR scan. [4] The display device according to any one of [1] to [3], wherein the monitoring unit monitors the state of the eyeball using a corneal reflex or a fundus reflex. [5] The display device according to any one of [1] to [4], further comprising a tracking unit that causes the image display light to track the movement of the eyeball. [6] The display device according to [5], wherein the tracking unit is composed of at least one selected from the group consisting of a combiner, a relay system driving unit, a mirror driving unit, and a retardation panel. [7] defining a coordinate system based on the characteristic distribution of the eyeball; The display device according to any one of [1] to [6], wherein the matching unit matches the characteristic distribution of the eyeball with the state of the eyeball to which the image is to be presented via the coordinate system. [8] The display device described in [7], wherein the coordinate system is defined based on at least two selected from the group consisting of the first fovea of ​​the right eye, the first blind spot of the right eye, the second fovea of ​​the left eye, and the second blind spot of the left eye. [9] The display device according to any one of [1] to [8], wherein the light source is a laser light source.

[10] Further comprising a scanning mirror, The display device according to any one of [1] to [9], wherein the scanning mirror irradiates the image display light onto the retina.

[11] The irradiation unit further includes a member disposed in front of the eye, The display device according to any one of [1] to

[10] , wherein the member has see-through properties.

[12] The display device according to

[11] , wherein the component is a first optical element including a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, or a metasurface, or a second optical element that reflects predetermined light and transmits light other than the predetermined light.

[13] The display device according to any one of [1] to

[12] , further comprising a wavelength dispersion compensation member.

[14] The display device according to

[13] , wherein the wavelength dispersion compensation member is a first optical element including a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, or a metasurface.

[15] Processing a distribution of eyeball characteristics; monitoring the condition of the eye; matching the characteristic distribution of the eyeball with a state of the eyeball; Illuminating a predetermined location on the retina with image display light emitted from a light source; including, how it is displayed.

[16] The display method described in

[15] , further comprising acquiring the characteristic distribution of the eyeball.

[17] The display method described in

[16] , wherein the acquiring of the characteristic distribution of the eyeball includes acquiring the characteristic distribution using at least one device selected from the group consisting of an OCT, a refractometer, and a light detection device that detects returning light from an IR scan.

[18] The display method according to any one of

[15] to

[17] , wherein said monitoring of the state of the eyeball includes monitoring the state of the eyeball using a corneal reflex or a fundus reflex.

[19] The display method according to any one of

[15] to

[18] , further comprising causing the image display light to follow the movement of the eyeball.

[20] The display method according to

[19] , wherein making the image display light follow the movement of the eyeball includes making the light follow the movement of the eyeball using at least one selected from the group consisting of a combiner, a relay system drive unit, a mirror drive unit, and a phase difference panel. [twenty one] defining a coordinate system based on the property distribution of the eye; A display method described in any one of

[15] to

[20] , wherein matching the characteristic distribution of the eyeball with the state of the eyeball includes matching, via the coordinate system, the characteristic distribution of the eyeball with the state of the eyeball to which the image is to be presented. [twenty two] The display method described in

[21] , wherein the coordinate system is defined based on at least two selected from the group consisting of the first fovea of ​​the right eye, the first blind spot of the right eye, the second fovea of ​​the left eye, and the second blind spot of the left eye. [twenty three] The display method according to any one of

[15] to

[22] , wherein the light source is a laser light source. [twenty four] The display method according to any one of

[15] to

[23] , comprising irradiating the image display light onto the retina using a scanning mirror. [twenty five] Preparing a see-through member; The display method according to any one of

[15] to

[24] , further comprising placing the see-through member in front of the user's eyes.

[26] The display method described in

[25] , wherein the component is a first optical element including a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, or a metasurface, or a second optical element that reflects predetermined light and transmits light other than the predetermined light.

[27] The display method according to any one of

[15] to

[26] , further comprising providing a wavelength dispersion compensation member.

[28] The display method according to

[27] , wherein the wavelength dispersion compensation member is a first optical element including a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, or a metasurface. [Explanation of symbols]

[0127] 1...Optical Vector, 2...Gaze Vector, 10...eyes, 20...eyeballs, 30...fovea, 31...blind spot, 100, 100-1...display device, 101...Light source section, 104...sensor, 105...Projection position adjustment mechanism, 107...Monitoring unit (gaze detection device), 112...control unit, 181...Image control unit, 182...Projection position control unit, 183... line of sight correction unit, 191...processing unit, 192...Matching section, 201...irradiation unit, 201...following part, 301...Acquisition Department.

Claims

1. A light source and an acquisition unit that acquires a characteristic distribution of the user's eyeball; a processing unit that generates a user-specific three-dimensional eyeball model including information about the shape of the eyeball and information about the optical characteristics of the eyeball, based on the eyeball characteristic distribution acquired by the acquisition unit; and a monitoring unit that monitors the state of the eyeball; a matching unit that generates correction information for correcting distortion of an image based on optical characteristics of the eyeball by referring to a three-dimensional eyeball model specific to the user in accordance with the state of the eyeball monitored by the monitoring unit; and an illumination unit that applies the correction information to image display light emitted from a light source to generate corrected image display light, and illuminates the corrected image display light at a predetermined location on a retina in accordance with a state of the eyeball; A display device comprising:

2. The display device according to claim 1 , wherein the acquisition unit is composed of at least one selected from the group consisting of a fundus camera, an OCT, a refractometer, and a light detection device that detects return light from an IR scan.

3. The display device according to claim 1 , wherein the monitoring unit monitors the state of the eyeball using a corneal reflex or a fundus reflex.

4. The display device according to claim 1 , further comprising a tracking unit that causes the image display light to follow the movement of the eyeball.

5. 5. The display device according to claim 4, wherein the follower comprises at least one selected from the group consisting of a combiner, a relay driving section, a mirror driving section, and a retardation panel.

6. A display device as described in claim 1, wherein the matching unit uses a coordinate system defined based on at least two selected from the group consisting of the first fovea of ​​the right eye, the first blind spot of the right eye, the second fovea of ​​the left eye, and the second blind spot of the left eye when referring to the three-dimensional eyeball model.

7. The display device according to claim 1 , wherein the light source is a laser light source.

8. Further comprising a scanning mirror, The display device according to claim 1 , wherein the scanning mirror irradiates the image display light onto the retina.

9. The irradiation unit further includes a member disposed in front of the eye, The display device according to claim 1 , wherein the member has a see-through property.

10. 10. The display device according to claim 9, wherein the member is a first optical element including a reflective or transmissive volume hologram, a reflective or transmissive relief hologram, or a metasurface, or a second optical element that reflects predetermined light and transmits light other than the predetermined light.

11. The display device according to claim 1 , further comprising a wavelength dispersion compensation member.

12. 12. The display device according to claim 11, wherein the wavelength dispersion compensation member is a first optical element including a reflection or transmission type volume hologram, a reflection or transmission type relief hologram, or a metasurface.

13. Obtaining a characteristic distribution of a user's eyeball; generating a user-specific three-dimensional eyeball model including information about the shape of the eyeball and information about the optical properties of the eyeball based on the acquired eyeball property distribution; monitoring the condition of the eye; generating correction information for correcting distortion of an image based on optical characteristics of the eyeball, by referring to a three-dimensional eyeball model specific to the user, according to the monitored state of the eyeball; applying the correction information to image display light emitted from a light source to generate corrected image display light, and irradiating the corrected image display light onto a predetermined location on a retina according to a state of the eyeball; including, how it is displayed.

Images