Device and display device

Abstract

An apparatus including a lens unit having a first region corresponding to a visual angle larger than a visual angle θ a , and a second region corresponding to a visual angle equal to or smaller than the visual angle θ a , where a change amount of an image height in the second region is larger than a change amount of an image height in the first region, and an imaging unit including a sensor substrate having a plurality of pixels each including an avalanche diode on which light from the lens unit is incident, and a pixel circuit substrate outputting pixel signals of the plurality of pixels.

Description

Technical Field

[0001] The embodiments relate to a device and a display device. In particular, the embodiments relate to head-mounted display technology. Background Technology

[0002] As one form of imaging display device, Japanese Patent Application Publication No. 2020-167600 discloses a head-mounted display having a fisheye lens for acquiring images with a wide viewing angle and an image sensor. As another form of imaging display device, Japanese Patent Application Publication No. 2024-121777 discloses smart glasses having a photoelectric conversion device, an OLED (organic light-emitting diode) display device, and a lens.

[0003] Typically, image sensors used in imaging display devices can be CMOS (Complementary Metal-Oxide-Semiconductor) sensors. Alternatively, image sensors can be SPAD (Single-Photon Avalanche Diode) sensors. Each of these sensors has a lower limit for photon detection.

[0004] Because image sensors have a lower limit for photon detection, the lower limit of pixel pitch increases as ambient light decreases. This lower limit of pixel pitch can suppress the resolution of imaging display devices. Therefore, this trade-off between ambient light and resolution may limit the use cases of imaging display devices and imaging devices used in imaging display devices. Summary of the Invention

[0005] According to some embodiments, a device includes: a lens unit having a first region and a second region, the first region corresponding to a viewing angle θ. a The second region corresponds to a viewpoint equal to or less than the viewpoint θ. a The viewing angle, wherein the change in image height in the second region is greater than the change in image height in the first region; and an imaging unit, the imaging unit comprising a sensor substrate and a pixel circuit substrate, the sensor substrate having multiple pixels, each pixel including an avalanche diode, light from the lens unit being incident on the avalanche diode, the pixel circuit substrate outputting pixel signals of the multiple pixels, wherein when the maximum viewing angle of the lens unit is θ max The focal length is f, and the maximum angle of view is θ. max The image height at half the viewpoint is y(θ) max When / 2), the following expression (1) is satisfied:

[0006] (1)

[0007] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of the embodiments is by way of example. Attached Figure Description

[0008] Figure 1A and Figure 1B An exemplary configuration of a head-mounted display according to a first embodiment is shown.

[0009] Figure 2 An example of the projection characteristics of the lens unit according to the first embodiment is shown.

[0010] Figure 3 An example of the resolution of the lens unit according to the first embodiment and a conventional lens is shown.

[0011] Figure 4 This is a view illustrating an exemplary configuration of an image sensor according to a first embodiment.

[0012] Figure 5 This is a diagram illustrating an exemplary configuration of a pixel circuit board according to a first embodiment.

[0013] Figure 6 This is a diagram illustrating an exemplary configuration of the charge processing unit according to the first embodiment.

[0014] Figure 7 An example of the angular pixel density of a head-mounted display according to Example 1 is shown.

[0015] Figure 8 An example of the angular pixel density of a head-mounted display according to Example 2 is shown.

[0016] Figure 9 This is a diagram illustrating an exemplary configuration of a head-mounted display according to a second embodiment.

[0017] Figure 10 This is a diagram illustrating an operational example of the video signal conversion unit according to the second embodiment.

[0018] Figure 11 This is a diagram illustrating an exemplary configuration of a head-mounted display according to a third embodiment.

[0019] Figure 12 This is a diagram illustrating an exemplary configuration of a head-mounted display according to a fourth embodiment.

[0020] Figure 13 This is a diagram illustrating an exemplary configuration of a head-mounted display according to a fifth embodiment.

[0021] Figure 14 An exemplary relationship between pixel pitch and the number of photons detected in an image sensor is shown. Detailed Implementation

[0022] The following embodiments are provided merely to illustrate the technical concepts of this disclosure and are not intended to limit the technology of this disclosure. For clarity, the dimensions and positional relationships of the components shown in the drawings are not necessarily drawn to scale. In the following description, the same components are represented by the same reference numerals, and detailed descriptions may not be provided.

[0023] This disclosure relates to an imaging device and an imaging display device using the imaging device. For example, the imaging device may be a camera device or a video camera device. For example, night vision goggles or head-mounted displays may be used as imaging display devices. In particular, a head-mounted display will be described as an example of an imaging display device according to an embodiment of this disclosure. A camera system built into a head-mounted display will be described as an example of an imaging device. Specifically, to expand the uses of head-mounted displays, the embodiments described below relate to head-mounted displays suitable for use in low-light environments. The devices to which the embodiments can be applied are not limited to those described above.

[0024] The head-mounted display disclosed herein is a video-perspective head-mounted display that includes at least a stereo camera configuration. For example, in the following embodiments, low illumination refers to ambient illumination in the range of 10 mlx to 100 mlx. Typically, a person's field of vision is divided into a central field of vision (which allows for high-resolution viewing), an effective field of vision (which allows for viewing through eye movements), and a peripheral field of vision (where the shape and color of objects cannot be clearly distinguished). In this disclosure, it is assumed that the central field of vision has a 10-degree viewing angle, the effective field of vision has a 60-degree viewing angle, and the peripheral field of vision has a 120-degree viewing angle.

[0025] The head-mounted display disclosed herein provides a user with peripheral video in low-light environments and provides a resolution that does not compromise user usability, at least within the effective field of view. In the following embodiments, a corner pixel density of 20 ppd is assumed to be a resolution level that does not compromise user usability. This corner pixel density corresponds to the corner pixel density when peripheral video is provided to the user via a so-called 2K×2K display panel. It should be noted that this corner pixel density value is a typical example, and the corner pixel density applicable in each embodiment is not limited to this value.

[0026] Figure 14 An exemplary relationship between the pixel pitch of an image sensor and the number of photons detected by a pixel is shown when using a wide-angle imaging lens in an imaging device. Figure 14As shown in the graph, the number of photons detected and the lower limit for photon detection vary depending on the imaging lens or exposure time (frame rate). The relative relationships are essentially the same for CMOS sensors and SPAD sensors. This disclosure provides techniques for ensuring the desired resolution in imaging devices and imaging display devices using the imaging device, even under reduced ambient light conditions.

[0027] First Embodiment

[0028] The configuration of the head-mounted display according to the first embodiment will be described with reference to the accompanying drawings.

[0029] Figure 1A The configuration of a portion of the head-mounted display is shown. Figure 1B It is along Figure 1A The image shows a cross-sectional view of the head-mounted display 100 taken along line A-A'. The head-mounted display 100 includes a lens unit 101 and a housing 102. The lens unit 101 includes a low-resolution region and a high-resolution region, the low-resolution region being a viewing angle greater than θ. a The first region, the high-resolution region, is where the viewing angle is equal to or less than θ. a The second region. For the change in image height y(θ) of lens unit 101 relative to the viewing angle, the change in the high-resolution region is set to be greater than the change in the low-resolution region.

[0030] The head-mounted display 100 includes an image sensor 103 and a display panel 105. The image sensor 103 and the display panel 105 are disposed on a substrate 104. The substrate 104 may include circuitry for driving the image sensor 103 and the display panel 105. The substrate 104 may also include multiple substrates.

[0031] The lens unit 101 projects images of objects outside the head-mounted display 100 onto the image sensor 103. The head-mounted display 100 according to this embodiment includes a camera system 110, which includes the lens unit 101 and the image sensor 103.

[0032] The head-mounted display 100 also includes a display optics unit 106. Video displayed on the display panel 105 is formed on the retina of the eyeball 109 via the display optics unit 106 and the user's pupil 108, thereby presenting a virtual image to the user. Specifically, the display panel 105 displays video based on images acquired by the imaging unit.

[0033] To reduce user discomfort from the video displayed on the display panel 105, the parallax video is preferably projected onto the user's left and right eyeballs 109. To acquire the parallax video, multiple imaging units are preferably provided. For example, some of the imaging units are preferably arranged in a stereo camera configuration, wherein the lens unit 101, image sensor 103, and eyeballs 109 are aligned along the optical axis 107. More preferably, the center of the display panel 105 and the display optics unit 106 are aligned along the optical axis 107.

[0034] Figure 2 The projection characteristics of lens unit 101 are shown, illustrating the relationship between the lens's half-angle and the image height. The lens used to acquire an image with the assumed angle of view in this embodiment can be, for example, a fisheye lens, which is a wide-angle lens. In this embodiment, a wide-angle lens including a fisheye lens can also be referred to as a regular lens.

[0035] The projection characteristics of a common lens can be expressed as fsin(θ / 2), where f represents the focal length of the lens and θ / 2 represents the half-angle. A wide-angle image can be obtained by distorting the image on the wide-angle side. Typically, in a common lens, when the image height at the half-angle θ / 2 is represented by y(θ / 2), the lens characteristics can be expressed by the following expression (1):

[0036] (1)

[0037] Meanwhile, the lens unit 101 according to this embodiment has the following projection characteristics: in these projection characteristics, when equal to or less than θ a Within its field of view (high-resolution region), it exhibits image height characteristics greater than those of a conventional lens, and greater than θ. a The image height characteristics of the lens unit 101 are smaller than those of a conventional lens within its viewing angle range (low-resolution region). When the maximum viewing angle of the lens unit 101 is θ... max Indicates and the perspective θ max The image height at half the height is determined by y(θ) max When / 2) is represented, the characteristics of lens unit 101 satisfy the following expression (2):

[0038] (2)

[0039] When the lens is designed to be at its maximum angle θ max When the same image height is obtained, expression (2) allows lens unit 101 to select a focal length that is larger than that of a normal lens. By increasing the focal length, the resolution in the high-resolution region can be increased more easily than in the case of a fisheye lens as a normal lens.

[0040] For example, Japanese Patent Application Publication No. 2024-78751 discloses a lens unit 101 having the aforementioned features. The problem addressed by the technology in Japanese Patent Application Publication No. 2024-78751 is to reduce the processing delay time caused by the distortion correction processing required when using a fisheye lens as a common lens. To solve this problem, Japanese Patent Application Publication No. 2024-78751 discloses a design example of a lens unit having two regions (i.e., a high-resolution region and a low-resolution region). Japanese Patent Application Publication No. 2024-78751 also discloses a device configuration capable of acquiring a sufficiently identifiable wide-angle image without distortion correction processing.

[0041] The problem to be solved in this embodiment is different from the problem in Japanese Patent Application Publication No. 2024-78751. However, in this embodiment, in order to provide a head-mounted display that ensures high resolution without sacrificing viewing angle even under reduced ambient light, a lens unit having high-resolution and low-resolution regions is incorporated into the head-mounted display.

[0042] As disclosed in Japanese Patent Application Publication No. 2024-78751, the ratio θ a / θ max Preferably, the ratio falls within the range of 0.15 to 0.35, where θ is the angle of view of lens unit 101. a and maximum angle θ max The ratio between them. In the embodiment, the lens unit 101 with optical properties is also referred to as the different viewing angle lens unit.

[0043] Figure 3 This is a graph illustrating an example of angular pixel density, showing the resolution in the case where the lens unit 101 according to this embodiment is incorporated into the head-mounted display 100 and in the case where a fisheye lens, as a conventional lens, is incorporated into the head-mounted display 100. Here, we assume a head-mounted display where the image sensor 103 has a pixel pitch of 4 micrometers, the lens has an F-number of 2.0, the imaging angle is 120 degrees, and the video is displayed at 90 fps. The lens unit 101 and the conventional lens are designed to achieve substantially the same level of angular pixel density. In this case, while the focal length of the conventional lens is 1.4, the focal length of the lens unit 101 can be increased to 5.7. With the increase in focal length, the angular pixel density is expected to increase from approximately 10 ppd to approximately 30 ppd.

[0044] As mentioned above, in Figure 1A and Figure 1BIn the illustrated head-mounted display 100, the imaging unit includes at least a lens unit 101 and an image sensor 103. The head-mounted display 100 includes multiple imaging units. In one embodiment, the high-resolution area of ​​the lens unit 101 corresponds to the effective visual field area of ​​the eyeball 109. The low-resolution area of ​​the lens unit 101 corresponds to the peripheral visual field area of ​​the eyeball 109. In the head-mounted display 100, the high-resolution area of ​​the lens unit 101 is allocated to the effective visual field, which is the range within the human visual field that can be viewed through eye movements. Furthermore, in the head-mounted display 100, the low-resolution area of ​​the lens unit 101 is allocated to the peripheral visual field, which is the range in which the shape and color of objects cannot be clearly distinguished.

[0045] Based on the above-described allocation of resolution regions, the low-resolution region of lens unit 101 is assigned to a field of view that is more difficult to distinguish than other regions in the user's multiple field of view regions, making it less likely for the user to differentiate resolution differences among the multiple field of view regions. In this way, in the embodiment, even when lens unit 101 with multiple resolution regions including high-resolution and low-resolution regions is used in the imaging unit, discomfort felt by the user with the video can be easily suppressed.

[0046] Figure 4 This is a view of an exemplary configuration of the image sensor 103 according to this embodiment. Figure 4 As shown, the image sensor 103 includes a sensor substrate 211 and a pixel circuit substrate 216. The sensor substrate receives incident light from the lens unit 101. The sensor substrate 211 has a plurality of pixels 214 arranged in a matrix in a plan view of the sensor substrate 211. Each pixel 214 includes an avalanche diode 215 that converts light into an electrical signal. The avalanche diode 215 detects the charge from the light and generates an avalanche current by causing avalanche multiplication. The pixel circuit substrate 216 detects the avalanche current through a charge processing unit 219 and outputs the signal of the pixel 214. In one embodiment, the charge processing units 219 of the pixel circuit substrate 216 are configured to correspond to the pixels 214 of the sensor substrate 211 respectively. The charge processing units 219 are arranged in a matrix on the pixel circuit substrate 216.

[0047] The sensor substrate 211 has a plurality of pixels 214 divided into a first pixel region 212 and a second pixel region 213. The charge processing unit 219 of the pixel circuit substrate 216 is also divided into a first pixel circuit region 217 and a second pixel circuit region 218, corresponding to the first pixel region 212 and the second pixel region 213, respectively. Pixels 214 from the first pixel region 212 are connected to the charge processing unit 219 of the first pixel circuit region 217, and pixels 214 from the second pixel region 213 are connected to the charge processing unit 219 of the second pixel circuit region 218. In one embodiment, the first pixel region 212 includes at least a low-resolution region, and the second pixel region 213 includes at least a high-resolution region. The first pixel region 212 is located outside the second pixel region 213. In another embodiment, the first pixel region 212 may surround the second pixel region 213, or it may be a region other than the second pixel region 213.

[0048] exist Figure 4 In the example, the pixel region of sensor substrate 211 is divided into two pixel regions, but the pixel region is not limited to this and can be divided into two or more pixel regions. Similarly, the pixel circuit region of pixel circuit substrate 216 is divided into two pixel circuit regions, but the pixel circuit region is not limited to this and can be divided into two or more pixel circuit regions. For example, as an intermediate pixel region between the first pixel region 212 and the second pixel region 213, a third pixel region smaller than the first pixel region 212 and larger than the second pixel region 213 can be provided. Similarly, for the pixel circuit region, an intermediate pixel circuit region can be provided between the first pixel circuit region 217 and the second pixel circuit region 218 as the third pixel circuit region.

[0049] Figure 5 This is a diagram illustrating an exemplary configuration of the pixel circuit board 216 according to this embodiment. Figure 5 As shown, the pixel circuit board 216 includes a plurality of charge processing units 219, which detect the avalanche current of the avalanche diodes 215 constituting the pixel 214 and output signals. The charge processing units 219 are arranged in a matrix on the pixel circuit board 216. The pixel circuit board 216 also includes at least a vertical scanning circuit unit 321, a horizontal scanning circuit unit 322, a column circuit 323, a control pulse generation unit 324, and an output circuit 325.

[0050] The vertical scanning circuit unit 321 receives control pulses supplied from the control pulse generation unit 324 and supplies the control pulses to the charge processing unit 219 via the vertical control line 326. The vertical control line 326 includes at least a reset line and a select line. The reset line supplies control pulses for resetting the charge processing unit 219, and the select line supplies control pulses for selecting the charge processing unit 219.

[0051] The vertical control line 326 may also include an exposure control line that controls the exposure time of the avalanche diode 215. Logic circuitry such as shift registers and address decoders is used for the vertical scan circuit unit 321. The signal line 328 supplies the signal output from the charge processing unit 219 selected by the vertical scan circuit unit 321 (this signal is also referred to as the signal of pixel 214) to the subsequent column circuit 323.

[0052] The signal from pixel 214 is input to column circuit 323 via signal line 328, and column circuit 323 performs prescribed processing on the input signal. Examples of processing include noise removal and amplification of the input signal, and converting the input signal into a form suitable for output to the external image sensor 103. For example, a parallel-to-serial conversion circuit is used in column circuit 323.

[0053] The horizontal scanning circuit unit 322 supplies control pulses to the column circuit 323 for sequentially outputting the signals processed by the column circuit 323 to the output circuit 325. The output circuit 325 includes a buffer amplifier, a differential amplifier, and the like, and outputs the signals output from the column circuit 323 as pixel signals 331 to the outside of the image sensor 103.

[0054] Figure 6 This is a diagram of an exemplary configuration of the charge processing unit 219. Figure 6 As shown, the charge processing unit 219 includes a quenching element 431, a waveform shaping unit 432, and a counter circuit 433. The quenching element 431 is connected to a power supply voltage VH, which is higher than the voltage VL described later, and is also connected to an avalanche diode 215 on the sensor substrate 211. A voltage VL, lower than VH, is supplied to the anode of the avalanche diode 215. The cathode of the avalanche diode 215 is connected to the quenching element 431 and the waveform shaping unit 432. A reverse bias voltage is supplied to the avalanche diode 215. When light is incident on the avalanche diode 215 and a charge from the incident light is generated in the avalanche diode 215, the charge causes avalanche multiplication, resulting in the generation of an avalanche current.

[0055] When a reverse bias voltage is supplied to the avalanche diode 215, the avalanche diode 215 operates in Geiger mode and linear mode. In Geiger mode, the avalanche diode 215 operates when the potential difference between the anode and cathode is greater than the breakdown voltage. In linear mode, the avalanche diode 215 operates when the potential difference between the anode and cathode is close to or below the breakdown voltage.

[0056] An avalanche diode 215 operating in Geiger mode is called a SPAD (single-photon avalanche diode). In this case, for example, the voltage VL is −30 V and the voltage VH is 1 V. The avalanche diode 215 can operate in linear mode or Geiger mode. In one embodiment, the avalanche diode 215 is a SPAD because the potential difference is greater than that in linear mode, and the avalanche multiplication effect becomes more significant in the case of a SPAD.

[0057] During signal amplification via avalanche multiplication, quenching element 431 functions as a load circuit (quenching circuit). Quenching element 431 suppresses avalanche multiplication (quenching operation) by limiting the voltage supplied to avalanche diode 215. Quenching element 431 restores the voltage supplied to avalanche diode 215 to voltage VH (charging operation) by allowing current flow corresponding to the voltage drop caused by the quenching operation. Quenching element 431 can be, for example, a resistive element.

[0058] The quenching element 431 may include a switching element that turns on and off the voltage supply path to the avalanche diode 215. For example, the switching element of the quenching element 431 may include a transistor, and the exposure time may be adjusted via an exposure control line (not shown) of vertical control line 326. During the period in which the switching element is closed, voltage is supplied to the avalanche diode 215, and the detection of charge from the incident light is performed.

[0059] The waveform shaping unit 432 shapes the potential change at the cathode of the avalanche diode 215 obtained during charge detection and outputs a pulse signal. For example, an inverter circuit can be used as the waveform shaping unit 432. Figure 6 An example is shown in which a single inverter is used as waveform shaping unit 432, but a circuit including multiple inverters connected in series can be used as waveform shaping unit 432, and other circuits with waveform shaping effects can also be used.

[0060] The counter circuit 433 includes at least a counter for counting the pulse signals output from the waveform shaping unit 432 and a selection circuit. The counter in the counter circuit 433 counts the pulse signals output from the waveform shaping unit 432. When the counter is an N-bit counter (N: a positive integer), the counter circuit 433 can count the pulse signals output from the waveform shaping unit 432 up to approximately 2... N One pulse. The signal counted by the counter circuit 433 is a digital signal and is retained as the signal for pixel 214.

[0061] When a control pulse is supplied from the vertical scanning circuit unit 321 via a reset line included in the vertical control line 326, the counter of the counter circuit 433 resets the signal of the held pixel 214. The counter circuit 433 may include a switching circuit that opens and closes the signal path of the pulse signal output from the waveform shaping unit 432. The switching circuit of the counter circuit 433 opens and closes in response to a signal from the exposure control line (not shown) of the vertical control line 326, and, for example, in one embodiment, the counting operation is performed only during the period in which the switching circuit is closed.

[0062] The selection circuit of counter circuit 433 switches between an electrical connection and a non-connection between the counter and signal line 328 of counter circuit 433 in response to a control pulse supplied from vertical scan circuit unit 321 via a selection line included in vertical control line 326. The selection circuit of counter circuit 433 may include a transistor, a buffer circuit, or the like.

[0063] The circuit configurations of the avalanche diode 215 and the image sensor 103 described above are disclosed, for example, in Japanese Patent Application Publication No. 2024-121777. However, the circuit configurations of the avalanche diode 215 and the image sensor 103 are not limited to those described above.

[0064] Reference Figure 5 The pixel circuit board 216 according to this embodiment is further described. For example, unlike the circuit configuration disclosed in Japanese Patent Application Publication No. 2024-121777, the circuit configuration of the pixel circuit board 216 according to this embodiment includes a pixel region setting unit 329. The pixel region setting unit 329 is an allocation unit that controls the control pulse generation unit 324 to allocate the region of the charge processing unit 219 corresponding to the pixel signal 331. More specifically, the vertical scanning circuit unit 321 selects the charge processing unit 219 in the row direction, and the horizontal scanning circuit unit 322 selects the charge processing unit 219 in the column direction. The signal from the region selected by the pixel region setting unit 329 is output as the pixel signal 331 via the output circuit 325. The pixel region setting unit 329 refers to the region control signal 330 input from the outside and selects or switches the region of the pixel 214. In other words, in response to the region control signal 330, the pixel region setting unit 329 of the pixel circuit board 216 of the image sensor 103 changes the region of the pixel 214 from which the pixel signal 331 is generated. Therefore, the pixel region setting unit 329 assigns a plurality of pixels of the sensor board to a first pixel region that includes at least a low-resolution region and a second pixel region that includes at least a high-resolution region.

[0065] For example, the pixel circuit board 216 can alternately select the signal of the first pixel circuit region 217 and the signal of the second pixel circuit region 218, and selectively and sequentially output the selected signal as the pixel signal 331. Figure 6 The counter circuit 433 in the middle maintains the counter value unless the reset line of the vertical control line 326 switches to an active state. Therefore, the charge processing unit 219 can read the counter value multiple times, and the counter value is the signal of pixel 214. This is different from the pixel circuit configuration of a CMOS sensor, in which the pixel signal value is reset when the pixel signal is read out. Due to this feature, for example, pixel signal 331 from the first pixel region 212 corresponding to the first pixel circuit region 217 and pixel signal 331 from the second pixel region 213 corresponding to the second pixel circuit region 218 can be acquired at different timings from the start of exposure.

[0066] In this embodiment, the viewing angle of lens unit 101 is assumed to be, for example, 120 degrees. Since lens unit 101 is a wide-angle lens, the amount of light in the periphery is reduced, and the peripheral field of view becomes darker than the effective field of view. In this embodiment's configuration, signals from the first pixel circuit region 217 corresponding to the peripheral field of view and signals from the second pixel circuit region 218 corresponding to the effective field of view can be extracted at different timings from the start of exposure.

[0067] For example, photon counting begins in both the effective field of view and the peripheral field of view at the start of exposure, and pixel signals from the effective field of view are extracted after t1 seconds. Subsequently, photon counts in each region are recovered, and pixel signals from the peripheral field of view are extracted after t2 seconds. In this embodiment, the difference between t1 and t2 seconds corresponds to a difference in exposure time, enabling the suppression of brightness differences between the effective and peripheral field of view. Therefore, in the head-mounted display 100 according to this embodiment, processing related to correction of reduced peripheral light intensity can be reduced, and improved processing latency performed by the head-mounted display 100 is also expected.

[0068] According to the head-mounted display 100 of this embodiment, the avalanche diode 215 can reduce the photon detection limit, and the resolution of the effective field of view can be improved by the lens unit 101. Therefore, a head-mounted display can be provided that ensures the required resolution even under reduced ambient light, thereby expanding the application scenarios of conventional head-mounted displays.

[0069] Example 1

[0070] Next, the configuration of the head-mounted display according to Example 1 of the first embodiment will be described with reference to the accompanying drawings. In the following description, the same components as those in the first embodiment are indicated by the same reference numerals and will not be provided in detail.

[0071] In the lens unit 101 of this example, the viewing angle θ corresponds to the high-resolution region. a The angle is 60 degrees, and the maximum angle of view is θ. max The field of view is 180 degrees, with a focal length f of 5 mm, and the high-resolution area is set within the effective field of view of the eyeball 109. Since the viewing angle of the head-mounted display 100 is assumed to be 120 degrees, the viewing angle of the lens unit 101 is a viewing angle that includes the assumed viewing angle for the head-mounted display 100. The viewing angle θ of the lens unit 101... a and maximum angle θ max The ratio θ between a / θ max The value is 0.34, and the characteristic of lens unit 101 is fsin(θ) max / 2) / y(θ max / 2) is 1.16.

[0072] Figure 7 The angular pixel density of the head-mounted display 100 relative to the pixel pitch of the image sensor 103 is shown when the lens unit 101 described above is used. More specifically, estimates of the angular pixel density are plotted for the central field of view, the effective field of view, and the peripheral field of view. Considering the use of the head-mounted display 100 in low ambient light conditions, it is assumed that the image sensor 103 is a sensor using SPADs as pixels to expand the application examples of the head-mounted display 100. Furthermore, it is assumed that the display panel 105 is a so-called 4K×4K panel using OLEDs (Organic Light Emitting Diodes) or LEDs (Light Emitting Diodes) as light-emitting elements for the pixels.

[0073] exist Figure 7 In this embodiment, by setting the high-resolution area of ​​the lens unit 101 within the effective field of view, the characteristics of the effective field of view and the central field of view exhibit substantially the same characteristics relative to the pixel pitch of the SPAD. As described above, assuming a 20 ppd angular pixel density is a resolution level that does not impair the user usability of the head-mounted display 100, in one embodiment, the pixel pitch of the image sensor 103 is less than 4.5 micrometers in both the effective and central fields of view. For example, in Figure 14 In the graph shown, when the ambient light exposure setting has a lower limit of 10 mlx and the SPAD is assumed to be image sensor 103, the pixel pitch is limited to 4 micrometers or more due to the detection limit. That is, the pixel pitch of multiple pixels of image sensor 103 is at least 4 micrometers and no more than 4.5 micrometers.

[0074] To achieve the above characteristics, in one embodiment, the image circle of the lens (which reflects the image height characteristics of the lens unit 101) is approximately Φ8mm or larger, and the sensor size is half-size or larger. Considering that the lens unit 101 is integrated into the head-mounted display 100, and considering that the imaging unit including the lens unit 101 is arranged within the interpupillary distance of the left and right eyes, the image circle is set to Φ16mm, and the upper limit of the sensor size is set to a type 1 sensor.

[0075] The head-mounted display 100 according to this embodiment can provide a head-mounted display that ensures the required resolution even under reduced ambient light, thereby expanding the application examples of conventional head-mounted displays.

[0076] Example 2

[0077] Next, the configuration of the head-mounted display according to Example 2 of the first embodiment will be described with reference to the accompanying drawings. In the following description, the same components as those in the first embodiment are indicated by the same reference numerals and will not be provided in detail.

[0078] In the lens unit 101 according to this example, the viewing angle θ corresponds to the high-resolution region. a The angle is 20 degrees, and the maximum angle of view is θ. max The viewing angle is 120 degrees, and the focal length f is 10 mm, with the high-resolution area positioned in the central visual field of the eyeball 109. Since the viewing angle of the head-mounted display 100 is assumed to be 120 degrees, the viewing angle of the lens unit 101 is essentially equal to the assumed viewing angle for the head-mounted display 100. The viewing angle θ of the lens unit 101... a and maximum angle θ max The ratio θ between a / θ max The value is 0.16, and the characteristic of lens unit 101 is fsin(θ) max / 2) / y(θ max / 2) is 1.52.

[0079] Figure 8 The angular pixel density of the head-mounted display 100 relative to the pixel pitch of the image sensor 103 is shown when the lens unit 101 described above is used. More specifically, estimates of the angular pixel density are plotted for the central field of view, the effective field of view, and the peripheral field of view. Considering that the head-mounted display 100 will be used in low ambient light conditions, and to broaden the applications of the head-mounted display 100, it is assumed that the image sensor 103 is a sensor with SPADs as pixels. It is assumed that the display panel 105 is a so-called 4K×4K panel with OLED or LED light-emitting elements as pixels.

[0080] exist Figure 8 In this embodiment, the angular pixel density of the head-mounted display 100 satisfies the relationship expressed as "(angular pixel density of the central field of view) > (angular pixel density of the effective field of view) > (angular pixel density of the peripheral field of view)". The upper limit of the angular pixel density of the central field of view and the effective field of view is 32 ppd, which is the display limit of the 4K×4K panel used as the display panel 105. As described above, assuming an angular pixel density of 20 ppd does not impair the resolution level that makes the head-mounted display 100 usable by the user, in one embodiment, the image sensor 103 has a pixel pitch of less than 9 micrometers in the central field of view. In the effective field of view, the pixel pitch of the image sensor 103 is less than 4.2 micrometers.

[0081] As described above, when SPADs are assumed to be pixels of image sensor 103, the lower limit of the pixel pitch is 4 micrometers due to the detection limit. That is, in one embodiment, in the central field of view, the pixel pitch of the plurality of pixels of image sensor 103 is at least 4 micrometers and no more than 9 micrometers. In the effective field of view, the pixel pitch of the plurality of pixels of image sensor 103 is at least 4 micrometers and no more than 4.2 micrometers.

[0082] To achieve the above features, in one embodiment, the image circle of the lens in the effective field of view (which reflects the image height characteristics of the lens unit 101) is approximately φ12mm or larger, and the sensor size is 2 / 3 or larger. Figure 8 As shown, in the central viewing area, due to the display characteristics of the display panel 105, the corner pixel density in the 4K×4K panel remains constant until the pixel pitch reaches 5.5 micrometers.

[0083] Therefore, considering the lower limit of the pixel pitch of the SPAD (which is 4 micrometers), in one embodiment, the image circle is approximately Φ9mm or larger, and the sensor size is 1 / 1.8 type or larger. Considering that the lens unit 101 is incorporated into the head-mounted display 100, the imaging unit including the lens unit 101 is arranged within the interpupillary distance of the left and right eyes. Therefore, the image circle of the imaging unit including the lens unit 101 has an upper limit of Φ16mm, and its sensor size has an upper limit of a type 1 sensor.

[0084] Based on the characteristics of the sensor size in Examples 1 and 2, in one embodiment, a plurality of pixels of substrate 104 fall within a sensor size ranging from type 1 / 2 to type 1. A first pixel region 212 and a second pixel region 213 are arranged to be inscribed within the image circle of lens unit 101, circumscribed outside the image circle of lens unit 101, or disposed inside the image circle of lens unit 101, the image circle being in the range from Φ8 to Φ16.

[0085] The head-mounted display 100 provided according to this embodiment ensures the required resolution even under reduced ambient light conditions, thereby enabling the expansion of applications of conventional head-mounted displays.

[0086] Second Embodiment

[0087] Next, the configuration of the head-mounted display according to the second embodiment will be described with reference to the accompanying drawings. In the following description, components identical to those in the first embodiment are indicated by the same reference numerals and will not be provided in detail.

[0088] Figure 9 This is a diagram illustrating an exemplary system configuration of a head-mounted display 200 according to this embodiment. The head-mounted display 200 includes a first imaging unit 501 and a second imaging unit 502 that detects photons 542 reflected from an object 541. The first imaging unit 501 outputs a first video signal 545. The second imaging unit 502 outputs a second video signal 546.

[0089] The first imaging unit 501 and the second imaging unit 502 each include a lens unit 101 and an image sensor 103 constituting the imaging unit of the first embodiment. The image sensor 103 includes a sensor substrate 211 and a pixel circuit substrate 216. The pixel circuit substrate 216 includes a pixel region setting unit 329. The pixel region setting unit 329 assigns a plurality of pixels 214 of the sensor substrate 211 to a first pixel region 212 that includes at least a low-resolution region and a second pixel region 213 that includes at least a high-resolution region.

[0090] The first imaging unit 501 and the second imaging unit 502 each include a video signal conversion unit 544, which converts the data of the pixel signal 331 output from the pixel circuit board 216 of the image sensor 103 into a video signal. The video signal conversion unit 544 is a first conversion unit that converts the pixel signal into first video data from the pixel signal of the first pixel region 212 and second video data from the pixel signal of the second pixel region 213. The first imaging unit 501 and the second imaging unit 502 each include a video signal conversion unit 544. The video signal conversion unit 544 of the first imaging unit 501 outputs a first video signal 545, including the first video data and the second video data, to the user's right eye. The video signal conversion unit 544 of the second imaging unit 502 outputs a second video signal 546, including the first video data and the second video data, to the user's left eye.

[0091] The head-mounted display 200 includes a control unit 543. The control unit 543 outputs a region control signal 330. In response to the region control signal 330, the image sensor 103 selects or switches the region from which it generates pixel signals 331. The video signal conversion unit 544 references the region control signal 330 to obtain information about the arrangement of pixels 214 in the specified region (e.g., the number of pixels in the row direction and the number of pixels in the column direction), and generates a video signal from the pixel signals 331. In one embodiment, the video signal is a signal conforming to a specified interface standard. Examples of applicable interface standards include the USB standard, the HDMI® standard, or the MIPI standard. However, applicable interface standards are not limited to those listed above.

[0092] Figure 10 This is a diagram illustrating an operational example of the video signal conversion unit 544. The head-mounted display 200 switches between a time period in which it outputs a pixel signal 331 corresponding to a first pixel region 212 and a time period in which it outputs a pixel signal 331 corresponding to a second pixel region 213 via the control unit 543. The first pixel region 212 is, for example, the peripheral field of view corresponding to a low-resolution region of the lens unit 101. The second pixel region 213 is the effective field of view corresponding to a high-resolution region of the lens unit 101.

[0093] The video signal conversion unit 544 references the arrangement information of the pixels 214 included in the first pixel region 212 and the pixel signal 331 output from the image sensor 103, and converts the pixel signal into a first video signal, which includes first video data 651 from the pixel signal 331 of the first pixel region 212. The video signal conversion unit 544 references the arrangement information of the pixels 214 included in the second pixel region 213 and the pixel signal 331 output from the image sensor 103, and converts the pixel signal into a second video signal, which includes second video data 652 from the pixel signal 331 of the second pixel region 213.

[0094] like Figure 4 As shown, since the first pixel region 212 is larger than the second pixel region 213, the number of pixels 214 contained in the first pixel region 212 is greater than the number of pixels 214 contained in the second pixel region 213. Therefore, as Figure 10 As shown, the first video data 651 has a larger data size than the second video data 652, and the first video data transmission period 653 is longer than the second video data transmission period 654.

[0095] The first video data 651 includes video data of the peripheral field of view and video data of the effective field of view. The peripheral field of view is the range within which the user cannot clearly distinguish the shape and color of objects, while the effective field of view is the range that can be viewed through the user's eye movements. The second video data 652 includes video data of the effective field of view, which is the range that can be viewed through the user's eye movements.

[0096] By utilizing the low visibility of the peripheral field of view for the user in the head-mounted display 200, the head-mounted display 200 can be driven in the following manner. For example, after updating the first video data 651 and presenting the user with an image including the peripheral field of view, in one embodiment, only the video of the effective field of view near the central field of view, which is the second video data 652, is updated. Because the user has low visibility of the image of the peripheral field of view corresponding to the difference between the first video data 651 and the second video data 652, the user's discomfort with the video image is unlikely to increase compared to a scenario where the entire video presented to the user by the first video data 651 is updated.

[0097] exist Figure 10 In the example, after transmitting the first video data 651, the second video data 652 is transmitted twice consecutively. This corresponds to updating the video including the peripheral field of view in the head-mounted display 200, followed by two consecutive updates of the video only within the effective field of view. In other words, in the head-mounted display 200, the video signal conversion unit 544 outputs the first video data 651 and the second video data 652 at different frame rates.

[0098] According to the head-mounted display 200 of this embodiment, in one embodiment, the duration of a first video data transmission period 653 for transmitting video including the peripheral field of view is different from the duration of a second video data transmission period 654 for transmitting only the effective field of view. Therefore, updating the peripheral and effective fields of view at different frame rates reduces the total time required for data transmission compared to the case where the video including the peripheral field of view is completely updated. Thus, in the head-mounted display 200, the power consumption required for data transmission can be reduced compared to conventional head-mounted displays, thereby enabling a reduction in the power consumption of the head-mounted display.

[0099] Reference Figure 9 Further description of the head-mounted display 200. The imaging unit of the head-mounted display 200 includes a first imaging unit 501 that outputs a first video signal 545 for the user's right eye, the first video signal being a video signal conforming to an interface standard. The imaging unit of the head-mounted display 200 also includes a second imaging unit 502 that outputs a second video signal 546 for the user's left eye, the second video signal being a video signal conforming to an interface standard. Figure 9In this configuration, the first imaging unit 501 primarily captures the video to be displayed on the right eye 109a. The second imaging unit 502 primarily captures the video to be displayed on the left eye 109b. Figure 9 The diagram shows a configuration in which the head-mounted display 200 includes two imaging units, but the head-mounted display 200 may include three or more imaging units.

[0100] The head-mounted display 200 also includes a processing unit 547, which converts the first video signal 545 and the second video signal 546 into a first parallax video signal 548 for the user's right eye 109a and a second parallax video signal 549 for the user's left eye 109b. The processing unit 547 is a second conversion unit that converts the first video signal output from the first imaging unit into the first parallax video signal for the user's right eye, and converts the second video signal output from the second imaging unit into the second parallax video signal for the user's left eye.

[0101] In one embodiment, the first parallax video signal 548 and the second parallax video signal 549 are video signals conforming to an interface standard. The first parallax video signal 548 and the second parallax video signal 549 represent parallax video corresponding to the interpupillary distance between the user's right eye 109a and left eye 109b. Therefore, the first imaging unit 501 and the second imaging unit 502 are arranged at an interval substantially equal to the interpupillary distance.

[0102] The processing unit 547 may include an image processing unit. The image processing unit may, for example, perform correction processing on performance deviations between pixels 214 of the sensor substrate 211 of the image sensor 103, or on defects in the pixels 214. The image processing unit may also perform distortion correction processing and peripheral light attenuation correction processing on the lens unit 101. The image processing unit may also adjust alignment errors between the lens unit 101 and the image sensor 103. The image processing unit may also adjust pose misalignment between the first imaging unit 501 and the second imaging unit 502. The image processing unit may also perform averaging processing to improve the signal-to-noise ratio. Averaging processing includes frame addition processing and averaging or smoothing processing of adjacent pixels.

[0103] The various types of processing performed by the image processing unit described above can be performed on the entire video signal or on a portion of the video signal. For example, the image processing unit can perform different kinds of processing on a portion of the video corresponding to the peripheral field of view and a portion of the video corresponding to the effective field of view, or perform processing under different conditions. The processing performed by the processing unit 547 is not limited to the processing described above, and may include processing typically performed on a head-mounted display.

[0104] The head-mounted display 200 includes a first display unit 503 that projects video using a first parallax video signal 548 onto the right eye 109a, and a second display unit 504 that projects video using a second parallax video signal 549 onto the left eye 109b. Similar to an imaging unit, in one embodiment, the first display unit 503 and the second display unit 504 are arranged at a distance substantially equal to the interpupillary distance. In the following description, the first display unit 503 and the second display unit 504 may be collectively referred to as display units.

[0105] The first display unit 503 includes a display panel 105 that projects video using a first parallax video signal 548 to the right eye 109a via a display optical unit 106, and a display driver 550 that drives the display panel 105. The second display unit 504 includes a display panel 105 that projects video using a second parallax video signal 549 to the left eye 109b via a display optical unit 106, and a display driver 550 that drives the display panel 105.

[0106] The display driver 550 can update all pixels of the display panel 105 at once, or it can update pixels partially. For example, according to Figure 10 The transmission sequence shown can update pixels corresponding to the high-resolution region (effective field of view) and pixels corresponding to the low-resolution region (peripheral field of view) at different frame rates.

[0107] According to the head-mounted display 200 of this embodiment, by utilizing the low visibility of the user's peripheral field of vision, the pixel update rate can be adjusted for each region without increasing the user's discomfort with the video. According to the head-mounted display 200, the update rate of pixels near the peripheral region of the field of vision can be lower than the update rate of pixels near the central region of the field of vision. Since the head-mounted display 200 can adjust the video update rate for the regions divided as described above, the power consumption for driving the display panel 105 can be suppressed compared to conventional head-mounted displays.

[0108] Third Embodiment

[0109] Next, the configuration of the head-mounted display according to the third embodiment will be described with reference to the accompanying drawings. In the following description, components identical to those in the first and second embodiments are indicated by the same reference numerals and will not be provided in detail.

[0110] Figure 11This diagram illustrates an exemplary system configuration of a head-mounted display 300 according to this embodiment. Compared to the head-mounted display 200 according to the second embodiment, the head-mounted display 300 differs in the number of signal paths from the imaging unit to the processing unit 547 and includes multiple signal paths. The first imaging unit 501 of the head-mounted display 300 outputs a first effective field-of-view video signal 1061 and a first peripheral field-of-view video signal 1062 in parallel. The second imaging unit 502 of the head-mounted display 300 outputs a second effective field-of-view video signal 1063 and a second peripheral field-of-view video signal 1064 in parallel. The first imaging unit 501 outputs the first effective field-of-view video signal 1061 and the first peripheral field-of-view video signal 1062 at different frame rates. The second imaging unit 502 outputs the second effective field-of-view video signal 1063 and the second peripheral field-of-view video signal 1064 at different frame rates.

[0111] The first effective field-of-view video signal 1061 and the second effective field-of-view video signal 1063 are primarily related to the second pixel region 213 of the image sensor 103, and are video signals used for the central field-of-view region or the effective field-of-view region. The first peripheral field-of-view video signal 1062 and the second peripheral field-of-view video signal 1064 are primarily related to the first pixel region 212 of the image sensor 103, and are video signals used for the peripheral field-of-view region.

[0112] In the second embodiment, as Figure 10 As shown, serial transmission corresponds to first video data 651 of the peripheral field-of-view video signal and second video data 652 of the effective field-of-view video signal. Conversely, in this embodiment, since the paths for these data segments are set to parallel, data transmission can be performed in a shorter time than in the head-mounted display 200 according to the second embodiment, and the latency of the head-mounted display is expected to be improved. In this way, it is anticipated that in the head-mounted display 300, even in conditions of reduced ambient light, the processing to ensure the required resolution can be performed more appropriately.

[0113] The head-mounted display 300 according to this embodiment eliminates the need for processing to extract image signals from various regions of the video signal in signal processing performed by the processing unit 547 or image processing performed by the image processing unit of the processing unit 547. Therefore, compared to the first and second embodiments, the head-mounted display 300 further reduces the processing load and improves the latency of the head-mounted display. In this way, it is contemplated that in the head-mounted display 300, even in conditions of reduced ambient light, processing to ensure the required resolution can be performed more appropriately.

[0114] Fourth embodiment

[0115] Next, the configuration of the head-mounted display according to the fourth embodiment will be described with reference to the accompanying drawings. In the following description, components identical to those in the first to third embodiments are indicated by the same reference numerals and will not be provided in detail.

[0116] Figure 12 This diagram illustrates an exemplary system configuration of a head-mounted display 400 according to this embodiment. Compared to the head-mounted display 300 according to the third embodiment, the head-mounted display 400 differs in the number of signal paths from the processing unit 547 to the display driver 550, and has multiple signal paths. The processing unit 547 of the head-mounted display 400 outputs a first effective field-of-view parallax video signal 1171 and a first peripheral field-of-view parallax video signal 1172 in parallel to the display driver 550 of the first display unit 503. The processing unit 547 of the head-mounted display 400 outputs a second effective field-of-view parallax video signal 1173 and a second peripheral field-of-view parallax video signal 1174 in parallel to the display driver 550 of the second display unit 504. Therefore, the processing unit 547 outputs in parallel the parallax video signal corresponding to the first video data and the parallax video signal corresponding to the second video data from the first parallax video signal. The processing unit 547 also outputs in parallel the parallax video signal corresponding to the first video data and the parallax video signal corresponding to the second video data from the second parallax video signal. Processing unit 547 outputs a first effective field-of-view parallax video signal 1171 and a first peripheral field-of-view parallax video signal 1172 at different frame rates. Processing unit 547 also outputs a second effective field-of-view parallax video signal 1173 and a second peripheral field-of-view parallax video signal 1174 at different frame rates.

[0117] The first effective field-of-view parallax video signal 1171 and the second effective field-of-view parallax video signal 1173 are primarily signals related to the images projected onto the user's effective field of view and central field of view. The first peripheral field-of-view parallax video signal 1172 and the second effective field-of-view parallax video signal 1173 are primarily signals related to the video projected onto the user's peripheral field of view. The video signal conversion unit 544 outputs the effective field-of-view video signal and the peripheral field-of-view video signal in parallel, but similar to the first video signal 545 in the head-mounted display 200 according to the second embodiment, these video signals can be output serially.

[0118] According to the head-mounted display 400 of this embodiment, the parallax video signal input to the corresponding display driver 550 is, for example, divided for each field of view and transmitted in parallel. Therefore, the head-mounted display 400 enables data transmission to be performed in a shorter time than the head-mounted display according to the above embodiment, and an improvement in latency is expected. In this way, it is anticipated that in the head-mounted display 400, even in conditions of reduced ambient light, the processing for ensuring the required resolution can be performed more appropriately.

[0119] Fifth Embodiment

[0120] Next, the configuration of the head-mounted display according to the fifth embodiment will be described with reference to the accompanying drawings. In the following description, components identical to those in the first to fourth embodiments are indicated by the same reference numerals and will not be described in detail. The head-mounted display 500 according to this embodiment corresponds to a modification of the head-mounted display 200 according to the second embodiment.

[0121] Figure 13 This diagram illustrates an exemplary system configuration of a head-mounted display 500 according to this embodiment. The head-mounted display 500 includes a gaze detection unit 1281 that detects the gaze of a user's right eye 109a and left eye 109b. The gaze detection unit 1281 calculates the area where the user is gazing based on the detected gaze of each eye and outputs gaze area coordinates 1282 as the calculation result. The gaze area coordinates 1282 are the center coordinates of the second pixel circuit region 218 of the pixel circuit board 216 of the image sensor 103. The gaze area coordinates 1282 are also the center coordinates of the second pixel region 213 of the image sensor 103.

[0122] The control unit 543, referencing the gaze region coordinates 1282 output from the gaze detection unit 1281, controls the pixel region setting unit 329 and dynamically changes the second pixel circuit region 218 and / or the second pixel region 213 according to the detected gaze changes. The control unit 543 then projects peripheral vision video and effective vision or central vision video centered on the gaze to the user's right eye 109a and left eye 109b. In one embodiment, when the pixel region setting unit 329 is controlled by the control unit 543, the dynamic change of the second pixel circuit region 218 and / or the second pixel region 213 is performed within the high-resolution region of the lens unit 101.

[0123] Therefore, using the head-mounted display 500 according to this embodiment, the effective field of view or central field of view is dynamically changed based on the user's gaze area calculated using detected gaze. In this way, it is expected that high-resolution video can be provided to the user more stably in the head-mounted display 500, and user discomfort with the video can be further suppressed.

[0124] The above embodiments are merely examples of specific implementations of the technology disclosed herein, and should not be construed as limiting the scope of the technology disclosed based on these embodiments. That is, the technology disclosed herein can be implemented in various forms without departing from its technical concept or essential characteristics. The above embodiments can be implemented in appropriate combinations.

[0125] According to this disclosure, the imaging device and the imaging display device using the imaging device can ensure the required resolution even in the case of reduced ambient light, thereby expanding the application scenarios of the imaging device and the imaging display device.

[0126] While this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the appended claims is to be given the broadest interpretation in order to cover all such modifications and equivalent structures and functions.

Claims

1. An apparatus comprising: The lens unit has a first region and a second region, the first region corresponding to a viewing angle θ. a The second region corresponds to a viewpoint equal to or smaller than the viewpoint θ. a From the perspective of, the change in image height in the second region is greater than the change in image height in the first region; as well as An imaging unit is provided, comprising a sensor substrate and a pixel circuit substrate. The sensor substrate has multiple pixels, each pixel including an avalanche diode. Light from the lens unit is incident on the avalanche diode, and the pixel circuit substrate outputs pixel signals from the multiple pixels. Where the maximum angle of view of the lens unit is θ max The focal length is f, and the maximum angle of view is θ. max The image height at half the viewpoint is y(θ) max When / 2), the following expression (1) is satisfied: (1)。 2. The device according to claim 1, wherein the first region includes a peripheral field of view region, and the second region includes an effective field of view region.

3. The device according to claim 1 or 2, wherein the plurality of pixels have a pixel pitch of at least 4 micrometers and no more than 9 micrometers.

4. The device according to claim 1 or 2, wherein the plurality of pixels have a pixel pitch of at least 4 micrometers and no more than 4.5 micrometers.

5. The device according to claim 1 or 2, wherein the plurality of pixels of the sensor substrate have sensor sizes ranging from type 1 / 2 to type 1.

6. A display device comprising the device according to claim 1 or 2, The pixel circuit substrate includes an allocation unit that allocates the plurality of pixels of the sensor substrate to a first pixel region and a second pixel region, wherein the first pixel region includes at least the first region, and the second pixel region includes at least the second region. The display device further includes a first conversion unit, which converts the pixel signal into first video data from the pixel signal of the first pixel region and second video data from the pixel signal of the second pixel region. The first conversion unit outputs the first video data and the second video data at different frame rates.

7. A display device comprising the device according to claim 3, The pixel circuit substrate includes an allocation unit that allocates the plurality of pixels of the sensor substrate to a first pixel region and a second pixel region, wherein the first pixel region includes at least the first region, and the second pixel region includes at least the second region. The display device further includes a first conversion unit, which converts the pixel signal into first video data from the pixel signal of the first pixel region and second video data from the pixel signal of the second pixel region. The first conversion unit outputs the first video data and the second video data at different frame rates.

8. A display device comprising the device according to claim 4, The pixel circuit substrate includes an allocation unit that allocates the plurality of pixels of the sensor substrate to a first pixel region and a second pixel region, wherein the first pixel region includes at least the first region, and the second pixel region includes at least the second region. The display device further includes a first conversion unit, which converts the pixel signal into first video data from the pixel signal of the first pixel region and second video data from the pixel signal of the second pixel region. The first conversion unit outputs the first video data and the second video data at different frame rates.

9. The display device according to claim 6, wherein the imaging unit comprises a first unit and a second unit, the first unit outputting a first video signal to the user's right eye, the first video signal comprising first video data and second video data, and the second unit outputting a second video signal to the user's left eye, the second video signal comprising the first video data and the second video data. The display device further includes: The second conversion unit converts the first video signal output from the first unit into a first parallax video signal for the right eye, and converts the second video signal output from the second unit into a second parallax video signal for the left eye. The display panel is configured to display video to the right eye and the left eye respectively using the first parallax video signal and the second parallax video signal converted by the second conversion unit; as well as The display driver drives the display panel. The display driver updates the pixels corresponding to the first and second regions of the display panel at different frame rates.

10. The display device according to claim 9, wherein the first unit and the second unit output the first video data and the second video data in parallel.

11. The display device according to claim 9, wherein the second conversion unit outputs in parallel a parallax video signal corresponding to the first video data and the second video data derived from the first parallax video signal, and outputs in parallel a parallax video signal corresponding to the first video data and the second video data derived from the second parallax video signal.

12. The display device according to claim 6, further comprising a gaze detection unit, the gaze detection unit detecting the user's gaze. The allocation unit changes the position of the second pixel region based on the detected change in the line of sight.

13. The display device according to claim 6, wherein the first pixel region and the second pixel region are arranged to be inscribed within an image circle of diameter Φ8 mm to Φ16 mm of the lens unit, circumscribed outside the image circle, or disposed on the inner side of the image circle.

Images