Imaging device and imaging display device

Abstract

In imaging devices and imaging display devices using imaging devices, resolution is ensured even when ambient light is reduced. [Solution] The imaging device has a field of view θ a A larger first region and the field of view θ a An imaging unit having a lens unit having a second region which is as follows, wherein the amount of change in image height in the second region is greater than the amount of change in image height in the first region; a sensor substrate having a plurality of pixels composed of avalanche diodes into which light from the lens unit is incident; and a pixel circuit board which outputs pixel signals of the plurality of pixels, wherein the maximum angle of view of the lens unit is θ max , with focal length f and the maximum angle of view θ max The image height at half the field of view is y(θ). max If we set it to / 2, then 1 <f×sin(θ max / 2) / y(θ max (satisfying / 2)

Description

[Technical Field]

[0001] This invention relates to an imaging device and an imaging display device. In particular, this invention relates to head-mounted display technology. [Background technology]

[0002] As one form of an imaging display device, Patent Document 1 discloses a head-mounted display having a fisheye lens for acquiring wide-angle images and an image sensor. Furthermore, as another form of an imaging display device, Patent Document 2 discloses smart glasses having a photoelectric converter, an OLED (Organic Light Emitting Diode) display device, and lenses. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-167600 [Patent Document 2] Japanese Patent Publication No. 2024-121777 [Overview of the project] [Problems that the invention aims to solve]

[0004] Generally, CMOS (Complementary Metal Oxide Semiconductor) sensors are used as image sensors in imaging display devices. Alternatively, SPAD (Single Photon Avalanche Diode) sensors are also used. These sensors also have a lower limit for photon detection. Figure 14 shows an example of the relationship between the pixel pitch of an image sensor and the number of photons detected by a pixel, using a wide-angle imaging lens. As shown in the graph in Figure 14, the number of detected photons and the lower limit of photon detection change depending on the imaging lens or exposure time (frame rate). The relative relationship between CMOS sensors and SPAD sensors is generally consistent.

[0005] Furthermore, due to the characteristics of the sensor's lower photon detection limit, the lower limit of the image sensor's pixel pitch increases as ambient light decreases. This lower pixel pitch limit can potentially suppress the resolution of images in imaging display devices. Therefore, this trade-off between ambient light and resolution may limit the use cases of imaging display devices and the imaging devices used in them.

[0006] The present invention has been made in view of the above-mentioned problems, and its purpose is to provide a technology for an imaging device and an imaging display device using an imaging device that ensures resolution even when ambient light is reduced. [Means for solving the problem]

[0007] A first aspect of this disclosure is the field of view θ a A larger first region and the field of view θ a An imaging unit having a lens unit having a second region which is as follows, wherein the amount of change in image height in the second region is greater than the amount of change in image height in the first region; a sensor substrate having a plurality of pixels composed of avalanche diodes into which light from the lens unit is incident; and a pixel circuit board which outputs pixel signals of the plurality of pixels, wherein the maximum angle of view of the lens unit is θ max , with focal length f and the maximum angle of view θ max The image height at half the field of view is y(θ). max If we assume ( / 2), then the following equation (1) is satisfied.

number

[0008] According to the present invention, by ensuring resolution even when ambient light is reduced, the use cases of the imaging device and the imaging display device can be expanded. [Brief explanation of the drawing]

[0009] [Figure 1] Diagram showing a configuration example of the head-mounted display according to the first embodiment [Figure 2] Diagram showing an example of projection characteristics of the lens unit according to the first embodiment [Figure 3] Diagram showing an example of resolution of the lens unit according to the first embodiment and a normal lens [Figure 4] Diagram showing a configuration example of the image sensor according to the first embodiment [Figure 5] Diagram showing a configuration example of the pixel circuit board according to the first embodiment [Figure 6] Diagram showing a configuration example of the charge processing unit according to the first embodiment [Figure 7] Diagram showing an example of angular pixel density of the head-mounted display according to Example 1 [Figure 8] Diagram showing an example of angular pixel density of the head-mounted display according to Example 2 [Figure 9] Diagram showing a configuration example of the head-mounted display according to the second embodiment [Figure 10] Diagram showing an operation example of the video signal conversion unit according to the second embodiment [Figure 11] Diagram showing a configuration example of the head-mounted display according to the third embodiment [Figure 12] Diagram showing a configuration example of the head-mounted display according to the fourth embodiment [Figure 13] Diagram showing a configuration example of the head-mounted display according to the fifth embodiment [Figure 14] Diagram showing an example of the relationship between the pixel pitch of the image sensor and the number of detected photons [Description of Embodiments]

[0010] The embodiments described below are intended to embody the technical idea of the present invention, and are not intended to limit the present invention. The size and positional relationship of members shown in each drawing may be exaggerated for clarity of explanation. In the following description, the same components may be denoted by the same reference numerals and repeated descriptions thereof may be omitted.

[0011] This disclosure relates to an imaging device and an imaging display device using the imaging device. For example, a camera device or a video camera device can be used as the imaging device. For example, night vision goggles or a head-mounted display can be used as the imaging display device. In particular, the embodiments of this disclosure will be described using a head-mounted display as an example of the imaging display device. Furthermore, the embodiments will be described using a camera system incorporated into a head-mounted display as an example of the imaging device. In detail, the embodiments described below relate to a head-mounted display that is suitable for use in low-light environments, with the aim of expanding the use cases of head-mounted displays. Note that the devices to which each embodiment can be applied are not limited to the examples above.

[0012] The head-mounted display of this disclosure is a video see-through type head-mounted display equipped with at least a stereo camera configuration. For example, the low light assumed in the following embodiments means an ambient light of 10 mlx to 100 mlx. Generally, a person's field of view is divided into the central field of view, which is the range that can be seen with high resolution; the effective field of view, which is the range that can be seen with eye movements; and the peripheral field of view, which is the range in which the shape and color of objects cannot be clearly distinguished. In this disclosure, the central field of view is defined as a range with a field of view of 10 degrees, the effective field of view as a range with a field of view of 60 degrees, and the peripheral field of view as a field of view We assume a range of 120 degrees.

[0013] Furthermore, the head-mounted display of this disclosure provides the user with peripheral vision images in low-light environments and provides a resolution that does not impair the user's workability, at least in the effective field of view area. In the following embodiments, an angular pixel density of 20 ppd is assumed as a resolution that does not impair the user's workability. This angular pixel density corresponds to the angular pixel density when providing peripheral vision images to the user with a so-called 2K2K display panel. Note that this angular pixel density is a typical example value, and the angular pixel density applicable to each embodiment is not limited to this value.

[0014] <First Embodiment> The configuration of the head-mounted display according to the first embodiment will be described below with reference to the drawings.

[0015] Figure 1A is a diagram illustrating a part of the configuration of a head-mounted display. Figure 1B shows a cross-section of the head-mounted display 100 along the line A-A' in Figure 1A. The head-mounted display 100 has a lens unit 101 and a housing 102. The lens unit 101 has a field of view θ a The first region is a larger low-resolution region, and the field of view θ a The system has the following second region, which is the high-resolution region. The change in the image height y(θ) of the lens unit 101 with respect to the field of view is set such that the change in the high-resolution region is greater than the change in the low-resolution region.

[0016] The head-mounted display 100 also includes an image sensor 103 and a display panel 105. The image sensor 103 and the display panel 105 are arranged on a substrate 104. The substrate 104 may include circuits for driving the image sensor 103 and the display panel 105. The substrate 104 may also consist of multiple substrates.

[0017] The lens unit 101 projects an image of an object outside the head-mounted display 100 onto the image sensor 103. The head-mounted display 100 according to this embodiment has a camera system 110 including the lens unit 101 and the image sensor 103.

[0018] The head-mounted display 100 also has a display optical unit 106. The image displayed by the display panel 105 is formed on the retina of the eyeball 109 via the display optical unit 106 and the user's pupil 108, presenting a virtual image to the user. Specifically, the display panel 105 displays an image based on the image acquired by the imaging unit.

[0019] To reduce the user's sense of discomfort with the image displayed on the display panel 105, it is desirable to project parallax images to the user's left and right eyeballs 109. To acquire parallax images, it is desirable that there are a plurality of imaging units. For example, some of the plurality of imaging units desirably have a stereo camera arrangement in which the lens unit 101, the image sensor 103, and the eyeball 109 are aligned along the optical axis 107. Further, it is more desirable to adopt an arrangement in which the center of the display panel 105 and the display optical unit 106 are aligned along this optical axis 107.

[0020] FIG. 2 is a diagram for explaining an image of projection characteristics of the lens unit 101, and shows the projection characteristics that are the relationship between the half angle of view of the lens and the image height. Examples of the lens that acquires an image with an angle of view assumed in the present embodiment include a fisheye lens which is a type of wide-angle lens. In the present embodiment, a wide-angle lens including a fisheye lens may be referred to as a normal lens.

[0021] The projection characteristic of a normal lens can be expressed as f sin(θ / 2), where f is the focal length of the lens and θ / 2 is the half angle of view. By distorting the image on the wide angle of view side, acquisition of a wide angle of view image becomes possible. In general, in a normal lens, when the image height at a half angle of view θ / 2 is defined as y(θ / 2), the characteristic of the lens can be expressed by the following formula (1). [Formula]

[0022] On the other hand, the lens unit 101 of the present embodiment has an angle of view θ a In the following angles of view (high resolution region), it exhibits larger image height characteristics than a normal lens, and at angles of view larger than the angle of view θ a (low resolution region), it has projection characteristics that exhibit smaller image height characteristics than a normal lens. Then, the maximum angle of view θ of the lens unit 101 max is defined, the angle of view θ max When the image height at an angle of view of 1 / 2 is defined as y(θ max / 2), the characteristic of the lens unit 101 satisfies the following formula (2).

number

[0023] Maximum field of view θ max When designing a lens to obtain the same image height, according to equation (2), the lens unit 101 can be configured with a larger focal length than a typical lens. By increasing the focal length, it becomes easier to achieve higher resolution in the high-resolution region compared to a typical fisheye lens.

[0024] An example of a lens unit 101 having such characteristics is disclosed in Japanese Patent Application Publication No. 2024-78751 (hereinafter also referred to as Patent Document 3). Patent Document 3 addresses the issue of processing delay time that occurs with distortion correction processing required when using a fisheye lens, which is a normal lens. To solve this problem, Patent Document 3 discloses a design example of a lens unit having two regions: a high-resolution region and a low-resolution region. Patent Document 3 also discloses an apparatus configuration for acquiring a wide-angle image that can be identified even without distortion correction processing.

[0025] The problems in this embodiment and the problems in Patent Document 3 are different. However, in this embodiment, in order to provide a head-mounted display that ensures high resolution even when ambient light is reduced without compromising the field of view, a lens unit having a high-resolution area and a low-resolution area is incorporated into the head-mounted display.

[0026] As disclosed in Patent Document 3, the angle of view θ of the lens unit 101 a and maximum field of view θ max θ is the ratio of a / θ max It is desirable that this falls within the range of 0.15 to 0.35. In this embodiment, a lens unit 101 having such optical characteristics is also called a different angle of view lens unit.

[0027] Figure 3 is a graph showing an example of angular pixel density, which is the resolution, when the lens unit 101 of this embodiment is incorporated into the head-mounted display 100, and when a conventional fisheye lens is incorporated into the head-mounted display 100. Here, the pixel pitch of the image sensor 103 is 4 μm, the lens F-number is 2.0, and the imaging angle of view is 120 This assumes a head-mounted display that shows video at 90fps. Furthermore, the lens unit 101 and the standard lens are designed to achieve similar angular pixel density. In this case, while the standard lens has a focal length of 1.4, the lens unit 101 can have a focal length up to 5.7. Also, as the focal length increases, The angular pixel density is expected to increase from approximately 10 ppd to approximately 30 ppd.

[0028] As described above, in the head-mounted display 100 of Figure 1, the imaging unit is composed of at least a lens unit 101 and an image sensor 103. The head-mounted display 100 also has multiple imaging units. Preferably, the high-resolution area of ​​the lens unit 101 is the effective field of view area of ​​the eyeball 109. The low-resolution area of ​​the lens unit 101 is the peripheral field of view area of ​​the eyeball 109. In the head-mounted display 100, it is desirable that the high-resolution area of ​​the lens unit 101 be allocated to the effective field of view, which is the range of the human field of view that can be seen by eye movements. In the head-mounted display 100, it is also desirable that the low-resolution area of ​​the lens unit 101 be allocated to the peripheral field of view, which is the range in which the shape and color of objects cannot be clearly distinguished.

[0029] According to the above resolution region allocation, the low-resolution region of the lens unit 101 is assigned to the field of view region of the user's multiple field of view regions that has lower discriminatory ability than other regions. As a result, it becomes difficult for the user to distinguish between the differences in resolution between the multiple field of view regions. Therefore, in this embodiment, even when a lens unit 101 having multiple resolution regions, such as high-resolution and low-resolution regions, is used in the imaging unit, it becomes easy to suppress the user's discomfort with the image.

[0030] Figure 4 illustrates an example configuration of the image sensor 103 of this embodiment. As shown in Figure 4, the image sensor 103 has a sensor substrate 211 into which light from the lens unit 101 is incident, and a pixel circuit board 216. The sensor substrate 211 has a plurality of pixels 214 arranged in a matrix in a plan view of the sensor substrate 211. The pixels 214 are composed of avalanche diodes 215 that convert light into electrical signals. The avalanche diodes 215 detect the charge originating from light and generate an avalanche current by causing avalanche amplification. The pixel circuit board 216 detects the avalanche current with a charge processing unit 219 and outputs a signal from the pixels 214. It is desirable that the charge processing unit 219 of the pixel circuit board 216 be provided so as to correspond to each pixel 214 of the sensor substrate 211. The charge processing unit 219 is arranged in a matrix on the pixel circuit board 216.

[0031] Multiple pixels 214 on the sensor substrate 211 are divided into a first pixel region 212 and a second pixel region 213. The charge processing unit 219 on the pixel circuit board 216 is divided into a first pixel circuit region 217 and a second pixel circuit region 218, corresponding to the first pixel region 212 and the second pixel region 213, respectively. Pixels 214 in the first pixel region 212 are connected to the charge processing unit 219 of the first pixel circuit region 217, and pixels 214 in the second pixel region 213 are connected to the charge processing unit 219 of the second pixel circuit region 218. It is desirable that the first pixel region 212 includes at least a low-resolution region, and the second pixel region 213 includes at least a high-resolution region. It is also desirable that the first pixel region 212 is located outside the second pixel region 213. In this embodiment, the first pixel region 212 may include the second pixel region 213, or it may be a region excluding the second pixel region 213.

[0032] In the example shown in Figure 4, the pixel region of the sensor substrate 211 is divided into two pixel regions, but it is not limited to this, and may be divided into two or more pixel regions. Similarly, the pixel circuit region of the pixel circuit substrate 216 is divided into two pixel circuit regions, but it is not limited to this, and may be divided into two or more pixel circuit regions. For example, a third pixel region may be provided as an intermediate pixel region between the first pixel region 212 and the second pixel region 213, which is smaller than the first pixel region 212 and larger than the second pixel region 213. Likewise, a third pixel region may be provided as an intermediate pixel circuit region between the first pixel circuit region 217 and the second pixel circuit region 218.

[0033] Figure 5 is a diagram illustrating an example of the configuration of the pixel circuit board 216 in this embodiment. The pixel circuit board 216 has a plurality of charge processing units 219 that detect the avalanche current of the avalanche diode 215 constituting the pixel 214 and output a signal. The charge processing units 219 are arranged in a matrix on the pixel circuit board 216. The pixel circuit board 216 also has at least a vertical scanning circuit unit 321, a horizontal scanning circuit unit 322, a column circuit 323, a control pulse generation unit 324, and an output circuit 325.

[0034] The vertical scanning circuit unit 321 receives control pulses supplied from the control pulse generation unit 324 and supplies control pulses to the charge processing unit 219 via the vertical control line 326. The vertical control line 326 includes at least a reset line that supplies a control pulse to reset the charge processing unit 219 and a selection line that supplies a control pulse to select the charge processing unit 219.

[0035] Furthermore, the vertical control line 326 may include an exposure control line that controls the exposure time of the avalanche diode 215. Logic circuits such as a shift register and an address decoder are used in the vertical scanning circuit section 321. The signal line 328 supplies the signal (also called the pixel 214 signal) output from the charge processing unit 219 selected by the vertical scanning circuit section 321 to the subsequent column circuit 323.

[0036] The column circuit 323 receives the signal from the pixel 214 via the signal line 328, and the column circuit 323 performs predetermined processing on the input signal. This processing includes noise reduction and amplification of the input signal, and converting the input signal into a format that can be output outside the image sensor 103. For example, a parallel / serial conversion circuit is used in the column circuit 323.

[0037] The horizontal scanning circuit 322 supplies control pulses to the column circuit 323 for sequentially outputting the signals processed by the column circuit 323 to the output circuit 325. The output circuit 325 consists of a buffer amplifier, a differential amplifier, etc., and outputs the signals output from the column circuit 323 to the outside of the image sensor 103 as pixel signals 331.

[0038] Figure 6 illustrates an example configuration of the charge processing unit 219. As shown in Figure 6, the charge processing unit 219 includes a quench element 431, a waveform shaping unit 432, and a counter circuit 433. The quench element 431 is connected to a power supply voltage that provides a potential VH higher than the potential VL described later, and to the avalanche diode 215 of the sensor substrate 211. In addition, a potential VL lower than the potential VH is supplied to the anode of the avalanche diode 215. The cathode of the avalanche diode 215 is connected to the quench element 431 and the waveform shaping unit 432. A reverse bias voltage is supplied to the avalanche diode 215. When light is incident on the avalanche diode 215 and a charge is generated in the avalanche diode 215 due to the incident light, this charge undergoes avalanche multiplication, resulting in the generation of an avalanche current.

[0039] When a reverse bias voltage is supplied to the avalanche diode 215, there are two operating modes for the avalanche diode 215: Geiger mode and linear mode. Geiger mode is a mode in which the avalanche diode 215 operates when the potential difference between the anode and cathode is greater than the breakdown voltage. Linear mode is a mode in which the avalanche diode 215 operates when the potential difference between the anode and cathode is near or below the breakdown voltage.

[0040] An avalanche diode 215 operating in Geiger mode is connected to SPAD (Single This is called a Photon Avalanche Diode. In this case, for example, the voltage VL is -30V and the voltage VH is 1V. The avalanche diode 215 may be operated in linear mode or Geiger mode. In the case of SPAD, the potential difference is larger compared to linear mode and the avalanche multiplication effect becomes more pronounced, so it is desirable for the avalanche diode 215 to be SPAD.

[0041] The quench element 431 functions as a load circuit (quench circuit) when the signal is multiplied by avalanche multiplication. The quench element 431 suppresses the voltage supplied to the avalanche diode 215, thereby suppressing avalanche multiplication (quench operation). The quench element 431 also restores the voltage supplied to the avalanche diode 215 to voltage VH by flowing a current corresponding to the voltage drop caused by the quench operation (recharge operation). The quench element 431 may be, for example, a resistor.

[0042] Furthermore, the quench element 431 may have a switch element that opens and closes the voltage path supplied to the avalanche diode 215. For example, the switch element of the quench element 431 may be a transistor, and the exposure time can be adjusted by an exposure control line (not shown) of the vertical control line 326. When the switch element is closed, voltage is supplied to the avalanche diode 215, and the charge due to incident light is detected.

[0043] The waveform shaping unit 432 shapes the potential change at the cathode of the avalanche diode 215 obtained when charge is detected and outputs a pulse signal. For example, an inverter circuit can be used as the waveform shaping unit 432. Figure 6 shows an example in which one inverter is used as the waveform shaping unit 432, but the waveform shaping unit 432 may be a circuit in which multiple inverters are connected in series, or other circuits that have a waveform shaping effect may be used.

[0044] The counter circuit 433 includes at least a counter and a selection circuit for counting pulse signals output from the waveform shaping unit 432. The pulse signals output from the waveform shaping unit 432 are counted by the counter in the counter circuit 433. If the counter is an N-bit counter (N: positive integer), the counter circuit 433 can count up to approximately 2 to the power of N pulse signals output from the waveform shaping unit 432. The signals counted by the counter circuit 433 are digital signals and are held as signals in the pixel 214.

[0045] When a control pulse is supplied from the vertical scanning circuit unit 321 via the reset line that constitutes the vertical control line 326, the counter in the counter circuit 433 resets the signal of the pixel 214 it is holding. The counter circuit 433 may also include a switch circuit for opening and closing the signal path of the pulse signal output from the waveform shaping unit 432. The switch circuit of the counter circuit 433 opens and closes based on the signal from the exposure control line of the vertical control line 326 (not shown), and for example, counting is performed only when the switch circuit is closed.

[0046] Furthermore, the selection circuit of the counter circuit 433 switches the electrical connection between the counter of the counter circuit 433 and the signal line 328 by control pulses from the vertical scanning circuit section 321 supplied via the selection line that constitutes the vertical control line 326. For example, a transistor or a buffer circuit may be used for the selection circuit of the counter circuit 433.

[0047] The circuit configurations of the avalanche diode 215 and the image sensor 103 described above are disclosed, for example, in Japanese Patent Application Publication No. 2024-121777 (hereinafter also referred to as Patent Document 4). However, the circuit configurations constituting the avalanche diode 215 and the image sensor 103 are not limited to the above configurations.

[0048] The pixel circuit board 216 of this embodiment will be further described with reference to Figure 5. For example, the circuit configuration of the pixel circuit board 216 of this embodiment differs from the circuit configuration disclosed in Patent Document 4 in that it has a pixel area setting unit 329. The pixel area setting unit 329 is an allocation unit that controls the control pulse generation unit 324 to allocate the area of ​​the charge processing unit 219 that constitutes the pixel signal 331. In detail, the vertical scanning circuit unit 321 selects the charge processing unit 219 in the row direction, and the horizontal scanning circuit unit 322 selects the charge processing unit 219 in the column direction. The signal of the region selected by the region setting unit 329 is output as a pixel signal 331 via the output circuit 325. The pixel region setting unit 329 refers to the region control signal 330 input from an external source and selects or switches the region of the pixel 214. In other words, the pixel region setting unit 329 of the image sensor 103's pixel circuit board 216 changes the region of the pixel 214 that constitutes the pixel signal 331 based on the region control signal 330. As a result, the pixel region setting unit 329 assigns multiple pixels on the sensor board to a first pixel region that includes at least a low-resolution region and a second pixel region that includes at least a high-resolution region.

[0049] For example, the pixel circuit board 216 can alternately select signals from the first pixel circuit region 217 and the second pixel circuit region 218, and sequentially send them out selectively as pixel signals 331. The counter circuit 433 in Figure 6 holds the counter value as long as the reset line of the vertical control line 326 does not transition effectively. Therefore, the charge processing unit 219 can read out the counter value, which is the signal of the pixel 214, multiple times. This differs from the pixel circuit configuration of a CMOS sensor, where the value is reset when the pixel signal is read out. Due to this feature, for example, the pixel signal 331 of the first pixel region 212, which is the first pixel circuit region 217, and the pixel signal 331 of the second pixel region 213, which is the second pixel circuit region 218, can be extracted at different timings with respect to the start of exposure.

[0050] In this embodiment, the field of view of the lens unit 101 is assumed to be, for example, 120 degrees. Because the lens unit 101 is a wide-angle lens, a decrease in peripheral light occurs, and the peripheral field of view becomes darker compared to the effective field of view. In the configuration of this embodiment, the signal from the first pixel circuit area 217 corresponding to the peripheral field of view and the signal from the second pixel circuit area 218 corresponding to the effective field of view can be extracted at different timings from the start of exposure.

[0051] For example, upon exposure commencement, photon counting begins in both the effective field of view and the peripheral field of view, and after t1 seconds, the pixel signal from the effective field of view is extracted. Subsequently, photon counting in each region resumes, and after t2 seconds, the pixel signal from the peripheral field of view is extracted. In this embodiment... The difference between t1 and t2 seconds represents the difference in exposure time, and the difference in brightness between the effective field of view and the peripheral field of view. This makes it possible to suppress the reduction of ambient light intensity. Therefore, in the head-mounted display 100 according to this embodiment, the processing related to correcting the decrease in ambient light intensity can be reduced, and thus it is expected that the latency of the processing performed by the head-mounted display 100 can be improved.

[0052] According to the head-mounted display 100 of this embodiment, the avalanche diode 215 lowers the photon detection limit, and the lens unit 101 increases the resolution of the effective field of view. Therefore, it is possible to provide a head-mounted display that guarantees resolution even when ambient light is reduced, thereby expanding the use cases of conventional head-mounted displays.

[0053] <Example 1> Next, the configuration of the head-mounted display according to Embodiment 1 of the first embodiment will be described with reference to the drawings. In the following description, components similar to those in the first embodiment will be denoted by the same reference numerals, and detailed descriptions will be omitted.

[0054] In the lens unit 101 of this embodiment, the field of view θ is the high-resolution region. aThe angle is 60 degrees, with a maximum field of view of θ. max The field of view is 180 degrees, the focal length f is 5 mm, and the high-resolution area is set within the effective field of view of the eyeball 109. Since the field of view of the head-mounted display 100 is assumed to be 120 degrees, the field of view of the lens unit 101 is the field of view that encompasses the field of view assumed by the head-mounted display 100. Field of view θ of the lens unit 101 a and maximum field of view θ max θ is the ratio of a / θ max It is 0.34, and the characteristic of lens unit 101 is fsin (θ max / 2) / y(θ max ( / 2) is 1.16.

[0055] Figure 7 illustrates the angular pixel density of the head-mounted display 100 with respect to the pixel pitch of the image sensor 103 when using the lens unit 101 described above. In detail, estimated values ​​of the angular pixel density are plotted for the central field of view, effective field of view, and peripheral field of view regions. Considering that the head-mounted display 100 will be used in low ambient light conditions, the image sensor 103 is assumed to be a sensor with SPAD pixels to expand the use cases of the head-mounted display 100. The display panel 105 is either an OLED (Organic Light Emitting Diode) or an LED (Light This assumes a so-called 4K4K panel where the light-emitting elements (Emitting Diodes) are used as pixels.

[0056] In Figure 7, by setting the high-resolution region of the lens unit 101 in the effective field of view region, the characteristics of the effective field of view and the central field of view exhibit approximately the same characteristics as the pixel pitch of the SPAD. As described above, assuming an angular pixel density of 20 ppd as a resolution that does not impair the user's workability of the head-mounted display 100, it is desirable that the pixel pitch of the image sensor 103 in the effective field of view and the central field of view be 4.5 μm or less. For example, in the graph shown in Figure 14, when the ambient illuminance is set to a lower limit of 10 mlx, and assuming the SPAD is the image sensor 103, the lower limit of the pixel pitch is 4 μm due to the detection limit. In other words, it is desirable that the pixel pitch of multiple pixels of the image sensor 103 be between 4 μm and 4.5 μm.

[0057] Furthermore, in order to achieve the above characteristics, it is desirable that the image circle of the lens unit 101 be approximately Φ8 mm or larger, and the sensor size be 1 / 2 inch or larger, reflecting the image height characteristics of the lens unit 101. Considering the integration of the lens unit 101 into the head-mounted display 100, it is desirable that the imaging unit including the lens unit 101 be positioned at the interpupillary distance of the left and right eyes, with an image circle of Φ16 mm and a sensor size of 1 inch as the upper limit.

[0058] The head-mounted display 100 according to this embodiment can provide a head-mounted display that maintains resolution even when ambient light is reduced, thereby expanding the use cases of conventional head-mounted displays.

[0059] <Example 2> Next, the configuration of the head-mounted display according to Embodiment 2 of the first embodiment will be described with reference to the drawings. In the following description, components similar to those in the first embodiment will be denoted by the same reference numerals, and detailed descriptions will be omitted.

[0060] In the lens unit 101 of this embodiment, the field of view θ is the high-resolution region. a The angle is 20 degrees, and the maximum field of view is θ. maxThe field of view is 120 degrees, the focal length f is 10 mm, and the high-resolution area is set in the central field of view of the eyeball 109. Since the field of view of the head-mounted display 100 is assumed to be 120 degrees, the field of view of the lens unit 101 will be approximately equal to the field of view assumed by the head-mounted display 100. Field of view θ of lens unit 101 a and maximum field of view θ max θ is the ratio of a / θ max It is 0.16, and is a characteristic of lens unit 101, fsi n(θ max / 2) / y(θ max ( / 2) is 1.52.

[0061] Figure 8 illustrates the angular pixel density of the head-mounted display 100 relative to the pixel pitch of the image sensor 103 when using the lens unit 101 described above. In detail, estimated values ​​of the angular pixel density are plotted for the central field of view, effective field of view, and peripheral field of view regions. Considering that the head-mounted display 100 will be used under low ambient light conditions, To expand the use cases of the head-mount display 100, the image sensor 103 is assumed to be a sensor with SPADs as pixels. The display panel 105 is assumed to be a so-called 4K4K panel with OLED or LED light-emitting elements as pixels.

[0062] In Figure 8, the angular pixel density of the head-mounted display 100 satisfies the relationship "(angular pixel density in the central field of view) > (angular pixel density in the effective field of view) > (angular pixel density in the peripheral field of view)". The upper limit of the angular pixel density in the central and effective fields of view is 32 ppd, which is the display limit of the 4K4K panel, which is the display panel 105. As described above, assuming an angular pixel density of 20 ppd as a resolution that does not impair the user's workability with the head-mounted display 100, it is desirable that the pixel pitch of the image sensor 103 in the central field of view be 9 μm or less. Also, in the effective field of view, the pixel pitch of the image sensor 103 is 4.2 It is desirable that the size be less than or equal to μm.

[0063] As described above, assuming SPAD as the image sensor 103, the lower limit of the pixel pitch is 4 μm due to the detection limit. That is, in the central field of view, it is desirable that the pixel pitch of multiple pixels of the image sensor 103 be between 4 μm and 9 μm. Furthermore, in the effective field of view, the pixel pitch of multiple pixels of the image sensor 103 is between 4 μm and 4.2 μm. It is preferable that it be lower.

[0064] Furthermore, in order to achieve the above characteristics, the image circle of the lens in the effective field of view region should be approximately Φ12 mm or larger, reflecting the image height characteristics of the lens unit 101, and it is desirable that the sensor size be 2 / 3 type or larger. As shown in Figure 8, in the central field of view region, based on the display characteristics of the display panel 105, the pixel pitch for a 4K4K panel is 5.5 μm. The same angular pixel density is maintained up to that point.

[0065] Therefore, considering the lower limit of the SPAD's pixel pitch of 4 μm, the image circle is approximately Φ9 mm or larger, and the sensor size is 1 / 1.8 inch or larger. This is desirable. Furthermore, considering the integration of the lens unit 101 into the head-mounted display 100, the imaging unit including the lens unit 101 will be positioned at the interpupillary distance between the left and right eyes. Therefore, it is desirable that the imaging unit including the lens unit 101 have an image circle of Φ16 mm and a sensor size of 1 inch as its upper limit.

[0066] Based on the sensor size characteristics shown in Examples 1 and 2, it is desirable that the multiple pixels on the substrate 104 fit within a sensor size of 1 / 2 type to 1 type. Furthermore, it is desirable that the first pixel region 212 and the second pixel region 213 be inscribed, circumscribed, or inside the image circle of the lens unit 101, which is Φ8 to Φ16.

[0067] The head-mounted display 100 according to this embodiment can provide a head-mounted display that maintains resolution even when ambient light is reduced, thereby expanding the use cases of conventional head-mounted displays.

[0068] <Second Embodiment> Next, the configuration of the head-mounted display according to the second embodiment will be described with reference to the drawings. In the following description, components similar to those in the first embodiment will be denoted by the same reference numerals, and detailed descriptions will be omitted.

[0069] Figure 9 is a diagram illustrating an example of the system configuration of the head-mounted display 200 of this embodiment. The head-mounted display 200 has a first imaging unit 501 and a second imaging unit 502 that detect photons 542 reflected from a subject 541. The first imaging unit 501 outputs a first video signal 545. The second imaging unit 502 outputs a second video signal 546.

[0070] The first imaging unit 501 and the second imaging unit 502 each have a lens unit 101 and an image sensor 103, which are imaging units as described in the first embodiment. The image sensor 103 has a sensor substrate 211 and a pixel circuit board 216. The pixel circuit board 216 has a pixel area setting unit 329. The pixel area setting unit 329 assigns a plurality of pixels 214 of the sensor substrate 211 to a first pixel area 212 which includes at least a low-resolution area and a second pixel area 213 which includes at least a high-resolution area.

[0071] The first imaging unit 501 and the second imaging unit 502 each have a video signal conversion unit 544 that converts data consisting of pixel signals 331 output by the pixel circuit board 216 of the image sensor 103 into a video signal. The video signal conversion unit 544 is a first conversion unit that converts the pixel signal into first video data consisting of pixel signals from a first pixel region 212 and second video data consisting of pixel signals from a second pixel region 213. The first imaging unit 501 and the second imaging unit 502 each also have a video signal conversion unit 544. The video signal conversion unit 544 of the first imaging unit 501 outputs a first video signal 545 containing first video data and second video data for the user's right eye. The video signal conversion unit 544 of the second imaging unit 502 outputs a second video signal 546 containing first video data and second video data for the user's left eye.

[0072] The head-mounted display 200 has a control unit 543. The control unit 543 outputs a region control signal 330. Based on the region control signal 330, the image sensor 103 selects or switches regions of pixels 214 that constitute the pixel signal 331. The video signal conversion unit 544 also refers to the region control signal 330, obtains information on the arrangement of pixels 214 in the specified region (for example, the number of pixels in the row direction and the number of pixels in the column direction), and generates a video signal from the pixel signal 331. It is desirable that the video signal conforms to a predetermined interface standard. For example, USB standards, HDMI® standards, and MIPI standards can be applied as interface standards. However, the standards that can be adopted as interface standards are not limited to these.

[0073] Figure 10 is a diagram illustrating an example of the operation of the video signal conversion unit 544. The head-mounted display 200 switches between outputting a pixel signal 331 corresponding to the first pixel area 212 and outputting a pixel signal 331 corresponding to the second pixel area 213, using the control unit 543. The first pixel area 212 is, for example, the peripheral field of view area corresponding to the low-resolution area of ​​the lens unit 101. The second pixel area 213 is the effective field of view area corresponding to the high-resolution area of ​​the lens unit 101.

[0074] The video signal conversion unit 544 refers to the arrangement information of pixels 214 included in the first pixel area 212 and the pixel signals 331 output by the image sensor 103, and converts it into a first video signal that includes first video data 651 consisting of the pixel signals 331 of the first pixel area 212. The video signal conversion unit 544 refers to the arrangement information of pixels 214 included in the second pixel area 213 and the pixel signals 331 output by the image sensor 103, and converts it into a second video signal that includes second video data 652 consisting of the pixel signals 331 of the second pixel area 213.

[0075] As shown in Figure 4, the first pixel region 212 is larger than the second pixel region 213, so the number of pixels 214 in the first pixel region 212 is greater than the number of pixels 214 in the second pixel region 213. Therefore, as illustrated in Figure 10, the data size of the first video data 651 is larger than the data size of the second video data 652, and the transmission period of the first video data 653 is longer than the transmission period of the second video data 654.

[0076] The first video data 651 includes video data of the peripheral field of view, which is the range in which the user cannot clearly distinguish the shape and color of objects, and the effective field of view, which is the range in which the user can see with eye movements. The video data 652 includes video data of the effective field of view, which is the range that the user can see with eye movements.

[0077] By taking advantage of the low visibility of the user's peripheral vision in the head-mounted display 200, the head-mounted display 200 can be driven in the following way: For example, after updating the first video data 651 and presenting the user with an image that includes the peripheral vision, only the image of the effective field of view near the center of the field of view, which is the second video data 652, can be updated. Because the user's visibility of the peripheral vision image corresponding to the difference between the first video data 651 and the second video data 652 is low, the user's discomfort with the image is less likely to be exacerbated compared to when the entire image presented to the user in the first video data 651 is updated.

[0078] In the example shown in Figure 10, the first video data 651 is transmitted, followed by the second video data 652 being transmitted twice in a row. This corresponds to updating the video including the peripheral field of view in the head-mounted display 200, and then updating only the video within the effective field of view twice in a row. In other words, in the head-mounted display 200, the video signal conversion unit 544 outputs the first video data 651 and the second video data 652 at different frame rates.

[0079] In the head-mounted display 200 of this embodiment, the duration of the first video data transmission period 653, which transmits video including the peripheral field of view, and the duration of the second video data transmission period 654, which transmits video only within the effective field of view, are different. Therefore, compared to a mode in which all video including the peripheral field of view is updated, the mode in which the peripheral field of view and the effective field of view are updated at different frame rates can shorten the total time required for data transfer. Consequently, the head-mounted display 200 can reduce the power required for data transfer compared to conventional head-mounted displays, thereby achieving power saving for the head-mounted display.

[0080] The head-mounted display 200 will be further described with reference to Figure 9. The imaging unit of the head-mounted display 200 has a first imaging unit 501 that outputs a first video signal 545, which is a video signal compliant with the interface standard, for the user's right eye. The imaging unit of the head-mounted display 200 also has a second imaging unit 502 that outputs a second video signal 546, which is a video signal compliant with the interface standard, for the user's left eye. In Figure 9, the first imaging unit 501 mainly captures the image displayed to the right eye 109a. The second imaging unit 502 mainly captures the image displayed to the left eye 109b. Although Figure 9 shows a configuration in which the head-mounted display 200 has two imaging units, the head-mounted display 200 may have three or more imaging units.

[0081] The head-mounted display 200 has a processing unit 547 that uses a first video signal 545 and a second video signal 546 to convert them into a first parallax video signal 548 for the user's right eye 109a and a second parallax video signal 549 for the user's left eye 109b. The processing unit 547 is a second conversion unit that converts the first video signal output from the first imaging unit into a first parallax video signal for the user's right eye, and converts the second video signal output from the second imaging unit into a second parallax video signal for the user's left eye.

[0082] It is desirable that the first parallax video signal 548 and the second parallax video signal 549 are video signals that conform to the interface standard. Furthermore, the first parallax video signal 548 and the second parallax video signal 549 are images with parallax corresponding to the distance between the user's right eye 109a and left eye 109b (also called interpupillary distance). For this reason, it is desirable that the first imaging unit 501 and the second imaging unit 502 be arranged at an interval approximately equal to the interpupillary distance.

[0083] The processing unit 547 may include an image processing unit. The image processing unit may, for example, perform correction processing for performance variations between pixels 214 of the sensor substrate 211 of the image sensor 103, or pixel The image processing unit may perform a correction process for defects 214. The image processing unit may also perform distortion correction and peripheral light falloff correction processes for the lens unit 101. Furthermore, the image processing unit may adjust the alignment error between the lens unit 101 and the image sensor 103. The image processing unit may also adjust the orientation misalignment between the first imaging unit 501 and the second imaging unit 502. Finally, the image processing unit may perform an averaging process to improve the signal-to-noise ratio. The averaging process includes frame addition and averaging or smoothing with adjacent pixels.

[0084] The various processes performed by the image processing unit described above may be performed on the entire video signal or on only a portion of the video signal. For example, the image processing unit may perform different processes or under different conditions on the video corresponding to the peripheral field of view and the video corresponding to the effective field of view. The processes performed by the processing unit 547 are not limited to those described above and may include processes that are commonly performed on head-mounted displays.

[0085] The head-mounted display 200 includes a first display unit 503 that projects an image using a first parallax video signal 548 onto the right eye 109a, and a second display unit 504 that projects an image using a second parallax video signal 549 onto the left eye 109b. Similar to the imaging unit, it is desirable that the first display unit 503 and the second display unit 504 be arranged at an interval approximately equal to the interpupillary distance. In the following description, the first display unit 503 and the second display unit 504 may be collectively referred to as the display unit.

[0086] The first display unit 503 includes a display panel 105 that projects an image using a first parallax video signal 548 onto the right eye 109a via a display optical unit 106, and a display driver 550 that drives the display panel 105. The second display unit 504 includes a display panel 105 that projects an image using a second parallax video signal 549 onto the left eye 109b via a display optical unit 106, and a display driver 550 that drives the display panel 105.

[0087] The display driver 550 may update all pixels constituting the display panel 105 at once, or it may update them partially. For example, according to the transmission sequence in Figure 10, pixels corresponding to the high-resolution area (effective field of view) and the low-resolution area (peripheral field of view) may be updated at different frame rates.

[0088] According to the head-mounted display 200 of this embodiment, by utilizing the low visibility of the user's peripheral vision, the pixel update rate can be adjusted for each region without exacerbating the user's discomfort with the image. Furthermore, with the head-mounted display 200, the pixel update rate near the periphery of the field of view can be made smaller compared to the pixel update rate near the center of the field of view. And, because the head-mounted display 200 can adjust the image update rate for the divided regions as described above, the power consumption related to driving the display panel 105 can be reduced compared to conventional head-mounted displays.

[0089] <Third Embodiment> Next, the configuration of the head-mounted display according to the third embodiment will be described with reference to the drawings. In the following description, components similar to those in the first and second embodiments will be denoted by the same reference numerals, and detailed descriptions will be omitted.

[0090] Figure 11 is a diagram illustrating an example of the system configuration of the head-mounted display 300 of this embodiment. Compared to the head-mounted display 200 of the second embodiment, the head-mounted display 300 has a different number of signal paths from the imaging unit to the processing unit 547, and has multiple signal paths. The first imaging unit 501 of the head-mounted display 300 outputs a first effective field of view video signal 1061 and a first peripheral field of view video signal 1062 in parallel. The second imaging unit 502 of the head-mounted display 300 outputs a second effective field of view video signal 1063 and a second peripheral field of view video signal 1064 in parallel. The first imaging unit 501 outputs a first effective field of view video signal 1061 and a first peripheral field of view video signal 1062 at different frame rates. The second imaging unit 502 also outputs a second effective field of view video signal 1063 and a second peripheral field of view video signal 1064 at different frame rates.

[0091] The first effective field of view video signal 1061 and the second effective field of view video signal 1063 are signals mainly related to the second pixel area 213 of the image sensor 103, and are video signals targeting the central field of view or the effective field of view. The first peripheral field of view video signal 1062 and the second peripheral field of view video signal 1064 are signals mainly related to the first pixel area 212 of the image sensor 103, and are video signals targeting the peripheral field of view.

[0092] In the second embodiment, as shown in Figure 10, first video data 651 corresponding to the peripheral field of view video signal and second video data 652 corresponding to the effective field of view video signal are transmitted in series. On the other hand, in this embodiment, since the paths for these data are arranged in parallel, data transmission can be performed in a shorter time compared to the head-mounted display 200 according to the second embodiment, and an improvement in the latency of the head-mounted display can be expected. As a result, it is expected that the head-mounted display 300 can more effectively perform processing to ensure resolution even when ambient light is reduced.

[0093] Furthermore, in the head-mounted display 300 according to this embodiment, the process of extracting image signals from each region from the video signal is unnecessary in the signal processing in the processing unit 547 or in the image processing in the image processing unit of the processing unit 547. Therefore, the processing load is further reduced compared to the first and second embodiments, and an improvement in the latency of the head-mounted display can be expected. As a result, it is expected that the head-mounted display 300 can more effectively perform the processing required to ensure resolution even when ambient light is reduced.

[0094] <Fourth Embodiment> Next, the configuration of the head-mounted display according to the fourth embodiment will be described with reference to the drawings. In the following description, components similar to those in the first to third embodiments will be denoted by the same reference numerals, and detailed descriptions will be omitted.

[0095] Figure 12 is a diagram illustrating an example of the system configuration of the head-mounted display 400 of this embodiment. Compared to the head-mounted display 300 of the third embodiment, the head-mounted display 400 has a different number of signal paths from the processing unit 547 to the display driver 550, and has multiple signal paths. The processing unit 547 of the head-mounted display 400 outputs the parallax video signal 1171 of the first effective field of view and the parallax video signal 1172 of the first peripheral field of view in parallel to the display driver 550 of the first display unit 503. In addition, the processing unit 547 of the head-mounted display 400 outputs the parallax video signal 1173 of the second effective field of view and the parallax video signal 1174 of the second peripheral field of view in parallel to the display driver 550 of the second display unit 504. Therefore, the processing unit 547 outputs the parallax video signal corresponding to the first video data and the parallax video signal corresponding to the second video data in parallel from the first parallax video signal. Furthermore, the processing unit 547 outputs in parallel the parallax video signal corresponding to the first video data and the parallax video signal corresponding to the second video data from the second parallax video signal. Also, the processing unit 547 outputs the parallax video signal 1171 of the first effective field of view and the parallax video signal 1172 of the first peripheral field of view at different frame rates. Furthermore, the processing unit 547 outputs the parallax video signal 1173 of the second effective field of view and the parallax video signal 1174 of the second peripheral field of view at different frame rates.

[0096] The parallax video signal 1171 of the first effective field of view and the parallax video signal 1173 of the second effective field of view are These are primarily signals related to images projected into the user's effective field of view and central field of view. The parallax video signal 1172 for the first peripheral field of view and the parallax video signal 1173 for the second effective field of view are also primarily signals related to images projected into the user's peripheral field of view. The video signal conversion unit 544 outputs the effective field of view video signal and the peripheral field of view video signal in parallel, but these video signals may also be output in series, similar to the first video signal 545 in the head-mounted display 200 according to the second embodiment.

[0097] In the head-mounted display 400 according to this embodiment, the parallax video signals input to each display driver 550 are, for example, divided and parallelized for each field of view and transmitted. Therefore, the head-mounted display 400 enables shorter data transmission times compared to the head-mounted display according to the above embodiment, and an improvement in the latency of the head-mounted display can be expected. As a result, it is expected that the head-mounted display 400 can more effectively perform processing to ensure resolution even when ambient light is reduced.

[0098] <Fifth Embodiment> Next, the configuration of the head-mounted display according to the fifth embodiment will be described with reference to the drawings. In the following description, components similar to those in the first to fourth embodiments will be denoted by the same reference numerals, and detailed descriptions will be omitted. The head-mounted display 500 according to this embodiment corresponds to a modified version of the head-mounted display 200 according to the second embodiment.

[0099] Figure 13 illustrates an example of the system configuration of the head-mounted display 500 of this embodiment. The head-mounted display 500 has a gaze detection unit 1281 that detects the gaze of the user's right eye 109a and left eye 109b. Based on the detected gaze of each eye, the gaze detection unit 1281 calculates the area that the user is fixated on and outputs the fixation area coordinates 1282 as a calculation result. The fixation area coordinates 1282 are the center coordinates of the second pixel circuit area 218 of the pixel circuit board 216 of the image sensor 103. In addition, the fixation area coordinates 1282 are the center coordinates of the second pixel area 213 of the image sensor 103.

[0100] The control unit 543 refers to the gaze area coordinates 1282 output from the gaze detection unit 1281 and controls the pixel area setting unit 329 to dynamically change the second pixel circuit area 218 and / or the second pixel area 213 in accordance with the change in the detected gaze. The control unit 543 then projects peripheral vision images and effective field of view or central vision images centered on the gaze onto the user's right eye 109a and left eye 109b. It is desirable that the dynamic changes of the second pixel circuit area 218 and / or the second pixel area 213 based on the control unit 543's control of the pixel area setting unit 329 be performed on the inside of the high-resolution area of ​​the lens unit 101.

[0101] Therefore, according to the head-mounted display 500 of this embodiment, the effective field of view or central field of view is dynamically changed based on the user's gaze area calculated using the detected gaze. As a result, the head-mounted display 500 is expected to provide the user with higher resolution images more stably and to further reduce the user's discomfort with the images.

[0102] The embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features. Furthermore, the above embodiments may be combined as appropriate.

[0103] This embodiment includes the following configuration. (Composition 1) Angle of view θ a A larger first region and the field of view θ a A lens unit having a second region which is as follows, wherein the amount of change in image height in the second region is greater than the amount of change in image height in the first region, An imaging unit having a sensor substrate having a plurality of pixels composed of avalanche diodes into which light from the lens unit is incident, and a pixel circuit board that outputs pixel signals from the plurality of pixels, It has, The maximum angle of view of the lens unit is θ max , with focal length f and the maximum angle of view θ max The image height at half the field of view is y(θ). max If we assume ( / 2), then the following equation (1) is satisfied.

number

[0104] 101 Lens unit, 103 Image sensor,

Claims

1. Angle of view θ ａ A larger first region and the field of view θ ａ A lens unit having a second region which is as follows, wherein the amount of change in image height in the second region is greater than the amount of change in image height in the first region, An imaging unit having a sensor substrate having a plurality of pixels composed of avalanche diodes into which light from the lens unit is incident, and a pixel circuit board that outputs pixel signals from the plurality of pixels, It has, The maximum angle of view of the lens unit is θ ｍａｘ , with focal length f and the maximum angle of view θ ｍａｘ The image height at half the field of view is y(θ). ｍａｘ If we assume / 2, then the following equation (1) is satisfied. [Math 1] An imaging device characterized by the following features.

2. The imaging apparatus according to claim 1, characterized in that the first region includes the peripheral field of view region and the second region includes the effective field of view region.

3. The imaging apparatus according to claim 1, characterized in that the pixel pitch of the plurality of pixels is 4 μm or more and 9 μm or less.

4. The imaging apparatus according to claim 1, characterized in that the pixel pitch of the plurality of pixels is 4 μm or more and 4.5 μm or less.

5. The imaging apparatus according to claim 1, characterized in that the sensor size of the plurality of pixels on the sensor substrate is a sensor size of 1 / 2 type to 1 type.

6. Having an imaging device according to any one of claims 1 to 5, The pixel circuit board has an allocation unit that assigns the plurality of pixels of the sensor board to a first pixel region including at least the first region and a second pixel region including at least the second region. The imaging device further includes a first conversion unit that converts the pixel signal into first video data consisting of pixel signals from the first pixel region and second video data consisting of pixel signals from the second pixel region. The first conversion unit outputs the first video data and the second video data at different frame rates. An imaging display device characterized by the following:

7. The imaging unit includes a first imaging unit that outputs a first video signal including the first video data and the second video data for the user's right eye, and a second imaging unit that outputs a second video signal including the first video data and the second video data for the user's left eye. The aforementioned imaging display device is A second conversion unit converts the first video signal output from the first imaging unit into a first parallax video signal for the right eye, and converts the second video signal output from the second imaging unit into a second parallax video signal for the left eye, The first parallax video signal and the second parallax video signal converted by the second conversion unit A display panel for displaying images using the above to the right eye and the left eye, respectively, A display driver that drives the aforementioned display panel, It further possesses, The display driver updates the pixels corresponding to the first and second regions of the display panel at different frame rates. The imaging display device according to feature 6.

8. The imaging display device according to claim 7, characterized in that the first imaging unit and the second imaging unit output the first video data and the second video data in parallel.

9. The imaging display device according to claim 7, characterized in that the second conversion unit outputs in parallel a parallax video signal corresponding to the first video data and a parallax video signal corresponding to the second video data from the first parallax video signal, and outputs in parallel a parallax video signal corresponding to the first video data and a parallax video signal corresponding to the second video data from the second parallax video signal.

10. It further includes an eye-tracking unit that detects the user's gaze, The imaging display device according to claim 6, characterized in that the allocation unit changes the position of the second pixel region in accordance with the change in the line of sight detected by the line of sight detection unit.

11. The imaging display device according to claim 6, characterized in that the first pixel region and the second pixel region are arranged to be inscribed within, circumscribed around, or inside the image circle of the lens unit, which has a diameter of Φ8 mm to Φ16 mm.

Images