Imaging device equipped with a liquid crystal panel

Abstract

The target object is measured using an encoded aperture pattern, and this imaging data is used to obtain accurate image data for measuring the distance between the lens center and each part of the target object. [Solution] To achieve this, the present invention has the following configuration: An imaging device having a lens 10, a light sensor 50, and an aperture 20 disposed between the lens 10 and the object to be imaged 40, wherein the aperture 20 has a first aperture that defines the imaging range and a second aperture having an encoded aperture pattern, and when capturing a first image of the object to be imaged 40, the center of the first aperture is shifted by a first distance in a first direction perpendicular to the optical axis of the lens 10, and when capturing a second image of the object to be imaged 40, the center of the first aperture is shifted by a first distance in the opposite direction to the first direction with respect to the optical axis of the lens 10, and the shape of the encoded aperture pattern is the same when capturing the first image and when capturing the second image.

Description

Technical Field

[0001] The present invention relates to an imaging device using an encoded imaging method.

Background Art

[0002] Imaging by a camera is to capture a two-dimensional image from a three-dimensional world. In a normal camera, the image of the focused part is clearly projected, but as the position moves away from the focus, blurring occurs.

[0003] On the other hand, there are demands for full-image display that can display a clear image on the entire screen, or for obtaining a three-dimensional image. To realize such demands, information on the distance between each position of the imaging target and the lens is required.

[0004] Non-Patent Document 1 describes a technique for measuring and calculating distance information together with camera shooting using a coded aperture of a special shape. Non-Patent Document 2 describes a technique using a pair of a pattern for countering image blurring and a pattern for obtaining distance information as a coded aperture.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] One imaging technique that uses a specially shaped aperture pattern (hereinafter also called an encoded aperture pattern) to obtain distance data for forming a 3D image or a fully focused image by simply taking a photograph, by measuring the distance from the lens to the subject. In other words, this method allows for the calculation of the distance from the lens to the pixels by taking a photograph using this encoded aperture pattern.

[0007] However, this method can make it difficult to capture the entire subject due to its relationship with the encoded aperture pattern, and attempting to capture the entire subject can make it difficult to secure the necessary amount of light. In other words, there is a challenge in that it is difficult to satisfy both of the above conditions simultaneously.

[0008] If the efficiency of light utilization decreases, the signal-to-noise ratio (S / N) decreases. The objective of this invention is to realize an imaging device that can acquire the necessary data for more accurately calculating the distance from the lens to the pixel while securing the required imaging area and suppressing the decrease in the S / N ratio. [Means for solving the problem]

[0009] The present invention solves the above problems, and the main specific means are as follows.

[0010] (1) An imaging device comprising a lens, a light sensor for receiving light, and an aperture positioned close to the lens and between the lens and the object to be imaged, wherein the aperture comprises a first aperture that defines the imaging range and a second aperture having an encoded aperture pattern, and when capturing a first image of the object to be imaged, the center of the first aperture is shifted by a first distance in a first direction perpendicular to the optical axis of the lens, and the first aperture overlaps with a first region of the encoded aperture pattern, and when capturing a second image of the object to be imaged, the center of the first aperture is shifted by a first distance in the opposite direction to the first direction with respect to the optical axis of the lens, and the first aperture overlaps with a second region of the encoded aperture pattern, wherein the shape of the pattern in the first region of the encoded aperture pattern and the shape of the pattern in the second region are the same.

[0011] (2) The imaging apparatus according to (1), characterized in that it has a mechanism for calculating the distance between the center of the lens and the imaging position of the object to be imaged using the data of the first image and the data of the second image.

[0012] (3) When capturing a third image of the object to be imaged, the center of the first aperture is offset by a first distance in a third direction perpendicular to the optical axis of the lens and perpendicular to the first direction, and the first aperture overlaps with the third region of the encoded aperture pattern, and when capturing a fourth image of the object to be imaged, the center of the encoded aperture pattern of the second aperture is offset by a first distance in the opposite direction to the third direction with respect to the optical axis of the lens, and the first aperture overlaps with the fourth region of the encoded aperture pattern, and the shape of the pattern in the first region of the encoded aperture pattern is the same as the shape of the pattern in the second region, the shape of the pattern in the third region, and the shape of the pattern in the fourth region, as described in (1). [Brief explanation of the drawing]

[0013] [Figure 1] This is an optical model of an imaging device that uses lenses. [Figure 2] This is an example of a coded aperture pattern. [Figure 3] This is a cross-sectional view of an imaging system when there is no aperture. [Figure 4] This is a cross-sectional view of an imaging system when using an aperture with a coded aperture pattern. [Figure 5] This is an example of a coded aperture pattern. [Figure 6] This is a cross-sectional view of an imaging system when using a coded aperture pattern and expanding the shooting angle of view. [Figure 7] This is a plan view of the aperture used in FIG. 6. [Figure 8] This is a cross-sectional view of an imaging system showing the problems when using a coded aperture pattern and expanding the shooting angle of view without reducing the amount of light passing through the aperture. [Figure 9] This is an example showing the problems in the coded aperture pattern used in FIG. 8. [Figure 10] This is a cross-sectional view of an imaging system showing a part of the present invention. [Figure 11] This is an example of a plan view of the aperture of the present invention used in FIG. 10. [Figure 12] This is a cross-sectional view of an imaging system showing another part of the present invention. [Figure 13] This is an example of a plan view of the aperture of the present invention used in FIG. 12. [Figure 14] This is a plan view showing the aperture when the present invention is used for shooting a two-dimensional image. [Figure 15] This is a plan view of a liquid crystal shutter. [Figure 16] This is a plan view of a pixel of a liquid crystal shutter. [Figure 17] This is a cross-sectional view of a liquid crystal shutter. [Figure 18] This is an example of a set of coded aperture patterns. [Figure 19] This is another example of a set of coded aperture patterns. [Figure 20] This is yet another example of a set of coded aperture patterns. [Figure 21] Here is yet another example of a set of coded aperture patterns. [Modes for carrying out the invention]

[0014] A camera is a means of capturing a 3D image as a 2D image. To reconstruct a 3D image, or a fully focused image, from this captured 2D image, it is necessary to know the distance from each imaging point to the center of the lens. Figure 1 is an optical model of a camera using a lens. In Figure 1, when an object at distance u is measured by a lens with focal length f, all incident light is focused onto the plane v according to the lens law shown in (Equation 1).

[0015]

number

[0016] If the position p on the imaging plane coincides with v, a focused image is obtained. However, if it shifts forward or backward, the projected light rays are projected as a circle of size b, as shown in (Equation 2). This circle is sometimes called a circle of confusion.

[0017]

number

[0018] In equation (2), a represents the aperture size. When the size of b exceeds the size of a pixel, blurring occurs in the image. Since there is a limit to the camera's depth of field, objects at a distance from the focal point will appear blurred in the image. The magnitude of this blur depends on the distance from the camera to the object, as shown in equations (1) and (2). Therefore, by measuring the blur, it is possible to estimate the distance from the camera to the object being imaged. This method is called Depth From Defocus (DFD). As an encoded aperture pattern for effectively measuring distance using DFD, for example, Levin et al. have proposed a pattern like the one shown in Figure 2.

[0019] Figure 2 shows a typical example, and various other coding aperture patterns have been proposed. However, for the sake of clarity, in the explanation up to Figure 9, we will use a simplified coding aperture pattern, as shown in Figure 5, which has a small circular light-shielding region inside a large circle.

[0020] The object of the present invention is to realize a means for accurately acquiring data for reconstructing a 3D image or a fully focused image by calculating the distance from the lens to the subject using the DFD method. The present invention will be described in detail below with reference to examples. [Examples]

[0021] Figure 3 is a cross-sectional view when photographing a subject 40 using lens 10. In Figure 3, the subject 40 is located to the right of lens 10, and the light sensor 50 onto which the image is projected is located to the left. Hereafter, the subject 40 may also be referred to as the subject 40. However, in this case, the subject 40 refers to a broader subject that includes not only a small object but also the background surrounding this object. The light sensor 50 uses a semiconductor image sensor such as a CMOS image sensor or a CCD image sensor.

[0022] Generally, in lens 10, the thickness of the lens decreases as you move away from the center, and the angle of incidence and refraction of light increase. Also, spherical aberration increases as you move away from the center. However, since Figure 3 is a cross-sectional view for explanatory purposes, the spherical aberration of lens 10 is ignored. The same applies to Figures 4 and beyond.

[0023] In Figure 3, light emanating from the central part 40C of the subject 40, shown by the dotted line, is refracted by the lens 10 and focused at the central part 50C of the light sensor (hereinafter also simply referred to as the sensor) 50. Light emanating from the upper end 40U1 (first upper part of the subject) of the subject 40, passing through the central part 10C of the lens 10, travels in a straight line, as shown by the solid line, and forms an image at the lower end 50L1 (first lower part of the sensor) of the sensor 50. Light emanating from the upper end 40U1 (first upper part of the subject) of the subject 40, which does not pass through the center of the lens 10, is refracted by the lens 10, as shown by the dashed line, and forms an image at the lower end 50L1 (first lower part of the sensor) of the sensor 50. On the other hand, light emanating from the lower end 40L1 of the subject 40, passing through the central part 10C of the lens 10, travels in a straight line, as shown by the solid line, and forms an image at the upper end 50U1 of the sensor 50. Furthermore, light that does not pass through the central part 10C of the lens 10 and exits from the lower end 40L1 of the subject 40 is refracted in the lens 10 as shown by the dashed line, and is imaged onto the upper end 50U1 of the sensor 50.

[0024] In this way, all the light emitted from the subject 40 is refracted by the lens 10 and formed as an image on the sensor 50. In Figure 3, the angle θ1 between the line connecting the upper end 40U1 of the subject 40, the center 10C of the lens 10, and the lower end 50L1 of the sensor 50, and the line connecting the lower end 40L1 of the subject 40, the center 10C of the lens 10, and the upper end 50U1 of the sensor, is the field of view that can be captured. The larger the field of view, the wider the range of image data of the subject that can be captured.

[0025] Figure 4 is a cross-sectional view of the case where the aperture 20 is placed close to the lens 10 between the lens 10 and the subject 40, and it shows the angle of view in which an image containing information for calculating distance can be captured. The aperture 20 is formed of a frame 21 made of a light-shielding material and a pattern region 22 on which the encoded aperture pattern 30 illustrated in Figure 5 is formed. The encoded aperture pattern 30 is a pattern that enables accurate calculation of the distance between the lens 10 and each position on the subject 40 using the DFD method.

[0026] In Figure 4, light emanating from the second lower part 40L2 below the center of the subject 40, indicated by the dashed-dotted line, passes through the entire pattern region 22 of the aperture 20, is refracted by the lens 10, and forms an image on the second upper part 50U2 of the sensor 50. On the other hand, light emanating from the second upper part 40U2 above the center of the subject 40, indicated by the double-dotted-dotted line, passes through the entire pattern region 22 of the aperture, is refracted by the lens 10, and forms an image on the second lower part 50L2 below the sensor 50.

[0027] The dotted line in Figure 4 is an overhead line indicating that light emitted from the center 40C of the subject 40 passes through the entire pattern region 22, is refracted by the lens 10, and is imaged at the center 50C of the sensor 50. The same applies to the following figures. This dotted line is the same as the dotted line in Figure 3.

[0028] In Figure 4, the range defined by the intersection angle (angle of view) θ2 between the line connecting the second upper part 40U2 of the subject 40's shooting location and the center 10C of the lens 10, and the line connecting the second lower part 40L2 of the subject 40's shooting location and the center 10C of the lens 10, is the range of data captured by the sensor 50, where light emanating from the subject 40 and passing through the entire pattern area 22 is captured. Light that has passed through the entire pattern area 22 can become data from which distance can be calculated. The same image data as in Figure 3 can be obtained, but the amount of light entering the sensor 50 is reduced by the amount of light from which the aperture 20 is placed. That is, the angle of view that can be obtained as image data is θ1, and the angle of view from which image data capable of calculating distance information can be obtained is θ2, so the relationship is θ2 < θ1. Therefore, by placing the aperture 20, the range of the subject from which data containing distance information is captured by the sensor 50 from the subject 40 is restricted. In other words, imaging data containing distance information can only be obtained from a part of the subject.

[0029] Figure 5 is a plan view of the aperture 20. The aperture 20 in Embodiment 1 is composed of a liquid crystal shutter, which will be described later, and has a rectangular shape. The periphery of the aperture 20 is a frame portion 21 formed of a light-shielding material. A pattern region 22 having an encoded pattern 30 is formed inside the frame portion 21.

[0030] In pattern region 22, the shaded area is a light-shielding region. The encoded pattern consists of a circle and a small light-shielding circle formed inside, with the center of the small circle offset from the center of the large circle. In Figure 5, the shaded area is the light-shielding area, and the white area is the area through which light passes. Hereafter, this light-transmitting area may also be referred to as a pseudo-crescent shape.

[0031] It should be noted that the encoding pattern in Figure 5 is not necessarily superior as an encoding pattern for DFD. It is merely presented as an example to illustrate the problems in the comparative example.

[0032] Figure 6 shows the field of view that can be captured when using aperture 20, and the field of view that can be captured when using aperture 20 is the same as the field of view that can be captured when not using aperture 20. In other words, Figure 6 is a cross-sectional view showing an example where θ1 is the same as in Figure 3. The angle between the line connecting the upper end of the subject 40 and the center of the lens and the lower end of the sensor 50, and the line connecting the lower end of the subject 40 and the center of the lens and the upper end of the sensor, as shown in Figure 6, i.e., the field of view that can be captured, is θ1, which is the same as in Figure 1.

[0033] The angle between the optical axis and the line connecting the lower edge of the subject (shown by the dashed-dotted line) and the upper edge of the pattern area of ​​aperture 20 is larger in Figure 6 than in Figure 4. Alternatively, the angle between the optical axis and the line connecting the upper edge of the subject (shown by the double-dotted-dotted line) and the lower edge of the pattern area of ​​aperture 20 is larger in Figure 6 than in Figure 4. However, some of the light from the first lower part 40L1 is blocked by the frame 21, reducing the amount of light reaching the sensor 50. Similarly, some of the light from the first upper part 40U1 is also blocked by the frame 21, reducing the amount of light reaching the sensor 50.

[0034] Figure 7 shows a plan view of the aperture in Figure 6. In Figure 7, the outer shape of the aperture 20 is the same as in Figure 5, but the width of the light-shielding frame 21 is larger than in Figure 5, and the area of ​​the pattern region 22 is smaller than in Figure 5. Consequently, the size of the encoded aperture pattern formed in the pattern region is also smaller.

[0035] In other words, simply increasing the field of view reduces the amount of light that can pass through the aperture. Consequently, the amount of light available to the sensor decreases, and the signal-to-noise ratio (S / N ratio) declines.

[0036] Figure 8 is a cross-sectional view showing the case where the shape of the aperture 20 is kept the same as in Figure 4, and the imageable field of view is the same as in Figure 6. The problem with Figure 8 is that the dashed line connecting the upper end 40U1 of the subject and the lower end of the lens lies inside the edge of the pattern region 22 of the aperture 20. In other words, it may not be possible to read the lower part of the encoded aperture pattern 30 formed in the pattern region 22. Also, the dashed line connecting the lower end of the subject 40 and the upper end of the lens 10 lies inside the upper edge of the pattern region 22 of the aperture 20. In other words, it may not be possible to read the upper part of the encoded aperture pattern formed in the pattern region 22.

[0037] Figure 9 is a plan view of the aperture 20 in this state. In Figure 9, the shape of the frame 21, pattern area 22, and encoded aperture pattern 30 of the aperture 20 is the same as in Figure 5. The difference between Figure 9 and Figure 5 is that there is an area 31 above and below the encoded aperture pattern 30 that cannot be read by the sensor.

[0038] Unless all the data for the encoded aperture pattern 30 and the subject passing through it can be read, it is not possible to perform accurate distance calculations using the DFD method.

[0039] The present invention solves the above-mentioned problems and enables accurate distance measurement using DFD. The aperture described in Figures 4 to 9 can be considered as a combination of a first aperture that defines the imaging range and a second aperture that is an encoded aperture pattern for distance measurement. In Figures 4 to 9, the first aperture and the second aperture have been used without distinction. In the present invention described below, for convenience, the first aperture and the second aperture, i.e., the encoded aperture pattern, will be explained separately.

[0040] Figure 10 is a cross-sectional view showing the first imaging configuration of Embodiment 1. Figure 10 differs from the cross-sectional views in Figures 4, 6, and 8 in that the aperture 20 is larger. An encoded aperture pattern 30 is formed across the entire surface of this large aperture 20. This corresponds to the second aperture. In Figure 10, the area captured by the camera is defined by the first aperture, and the imaging range uses the d1 region of the second aperture. An encoded aperture pattern 30 is formed in region d2, but it is not used for imaging in Figure 10. Incidentally, although only one aperture 20 is shown in the cross-sectional views of Figures 10 and 12, this configuration includes both the first and second apertures.

[0041] Figure 11 is a plan view of the aperture 20. In Figure 11, an encoded aperture pattern 30 is formed across the entire surface of the aperture 20. In Figure 11, the area indicated by circle A is the range of the first aperture that defines the imaging range. The diameter d1 of circle A in Figure 11 corresponds to the imaging area d1 in Figure 10.

[0042] Returning to Figure 10, a portion of the image of the subject 40 is blocked by the first aperture d1 and does not reach the light sensor 50. Therefore, in Figure 10, the data captured by the light sensor 50 is incomplete for distance measurement. To compensate for this incomplete data, a second image is taken, as shown in Figures 12 and 13.

[0043] Figure 12 is a cross-sectional view showing the second imaging configuration of Embodiment 1. The structure in Figure 12 is the same as in Figure 10, but the imaging location is different from that in Figure 10. In Figure 12, the range captured by the camera is defined by the first aperture, and the imaging range uses the d1 region of the second aperture, as in Figure 10. However, in Figure 12, the d1 region uses the lower part of the second aperture. The encoded aperture pattern 30 is formed in the upper region d2, but it is not used for imaging in Figure 12.

[0044] Figure 13 is a plan view of the aperture 20. In Figure 13, an encoded aperture pattern 30 is formed across the entire surface of the aperture 20. In Figure 13, the area indicated by circle A is the range of the first aperture that defines the imaging range. Compared to Figure 11, in Figure 13, circle A is located below the aperture 20 on which the encoded aperture pattern 30 is formed. The diameter d1 of circle A in Figure 13 corresponds to the imaging area d1 in Figure 12.

[0045] Returning to Figure 12, a portion of the image of the subject 40 is blocked by the first aperture d1 and does not reach the light sensor 50. Therefore, in Figure 12, the data captured by the light sensor 50 is incomplete for distance measurement. However, the area above the aperture 20 that is not captured in Figure 12 has already been captured in the first shot shown in Figure 10.

[0046] On the other hand, the area below aperture 20, which was not captured in the first shot, was captured in the second shot shown in Figure 12. Therefore, by combining the data from the first and second shots, it is possible to obtain data that can reconstruct the distance data.

[0047] Incidentally, the image captured by a camera is an image that has various degradation elements added to it compared to a fully focused image (an ideal image with no blur across the entire screen). These degradation elements are represented as the general blur function PSF (Pont Spread Function). Here, if we represent the blur function as k, the image j captured by the camera can be represented as a convolution of the fully focused image i and the blur function k, as shown in (Equation 3).

[0048]

number

[0049] In other words, the reconstruction of the all-focus image i can be performed by deconvolving the captured image j. Since obtaining the all-focus image i requires calculating the distance to each imaging point, the reconstruction of the all-focus image is equivalent to the reconstruction of the distance from the lens center to each imaging point.

[0050] Equation (4) is the inverse Fourier transform of Equation (3).

[0051]

number

[0052] Here, the inverse function K of the PSF. -1 If the frequency is known, the frequency image I of the entire focal image can be obtained as shown in (Equation 5).

[0053]

number

[0054] Then, by inversely transforming I, the all-focus image i can be reconstructed. As mentioned earlier, reconstructing the all-focus image i is equivalent to measuring the distance from the lens center to each imaging point of the subject.

[0055] Incidentally, when capturing an image through the encoded aperture pattern 30, the blur function PSF becomes predominantly influenced by the encoded aperture pattern 30. Therefore, for example, if the same encoded aperture pattern 30 is used for both the first and second captures, data processing becomes much easier when combining the data from the first and second captured images.

[0056] As shown in Figures 11 and 13, the encoded aperture pattern 30 in this invention maintains almost the same shape even when the first aperture moves in the first direction (y-direction) or the second direction (x-direction). Therefore, the blur function PSF including the encoded aperture pattern 30 is almost the same. Consequently, data processing for measuring the distance between the lens center and each shooting point of the subject becomes easier using the first and second imaging data.

[0057] Figures 10 to 13 show a model for the one-dimensional case. Since images in the real world are two-dimensional, at least four images are required. Figure 14 is a plan view showing the relationship between the imaging ranges A1, A2, A3, and A4 by the first aperture and the encoded aperture pattern 30 when the present invention is applied to two-dimensional imaging.

[0058] In Figure 14, four circular first apertures are shown inside a large second aperture 20 having an encoded aperture pattern 30, defining the imaging range. These apertures represent the relationship between the first and second apertures when the same subject is photographed four times.

[0059] In Figure 14, when the same subject is photographed four times, the second aperture containing the encoded aperture pattern 30 is the same, and only the circular first aperture that defines the shooting range moves. In Figure 14, A1 is the position of the first aperture relative to the second aperture, including the encoded aperture pattern 30, when photographing the upper left of the subject; A2 is when photographing the upper right of the subject; A3 is when photographing the lower right of the subject; and A4 is when photographing the lower left of the subject. In this case, the shape of the encoded aperture pattern 30 captured by the four photographs is almost the same. Conversely, the encoded aperture pattern 30 has a shape that hardly changes even when its position moves within the second aperture.

[0060] For example, as shown in Figure 14, the shape of the zebra pattern captured by the first aperture remains almost the same even when its position is changed. In other words, the blur function PSF including the encoded aperture pattern 30 is almost the same. Therefore, as explained in Figures 10 to 13, data processing for measuring the distance between the lens center and each shooting point of the subject becomes easy using the first to fourth imaging data.

[0061] The zebra patterns shown in Figures 11, 13, and 14 are examples of how nearly identical patterns can be obtained even when the image is cut in a different location. This does not mean that these patterns are superior as encoding aperture patterns 30, or that they can be used as is.

[0062] In practice, the most effective coded aperture pattern 30 is designed when distance measurement is performed using the DFD method. A key feature of this invention is that this pattern shape is designed to be nearly identical even when cropped at multiple locations for each photograph. Here, nearly identical patterns mean that the contribution of the coded aperture pattern 30 that constitutes the bokeh function k in (Equation 3) is nearly identical.

[0063] Incidentally, the four images are captured in an instant. Therefore, the encoding aperture pattern 30 also needs to be switched instantaneously. For this purpose, a liquid crystal shutter using a liquid crystal panel is suitable. A liquid crystal shutter can change the pattern instantaneously by changing the signal voltage applied to each pixel. It is also easy to switch the encoding aperture pattern 30 in the liquid crystal shutter in synchronization with the timing of the shot. Furthermore, by using a liquid crystal shutter, the first aperture and the second aperture, as explained in Figures 10 to 14, can be formed simultaneously.

[0064] Figure 15 is a plan view of the liquid crystal shutter. In Figure 15, the TFT substrate 111 and the opposing substrate 115 are bonded together with a sealing material around the periphery, and liquid crystal is sealed inside. A peripheral light-shielding region 21 and a pattern region 22 are formed in the area where the TFT substrate 111 and the opposing substrate 115 overlap.

[0065] The TFT substrate 111 is formed larger than the opposing substrate 115, and the portion of the TFT substrate 111 that does not overlap with the opposing substrate 115 is the terminal region 160. Driver ICs 165 and other components that drive the liquid crystal shutter are located in the terminal region 160.

[0066] In the pattern region 22 of Figure 15, scan lines 151 extend horizontally (x-direction) and are arranged vertically (y-direction) on the TFT substrate 111. Signal lines 152 also extend vertically and are arranged horizontally. Pixels 153, including pixel electrodes, are formed in the region enclosed by the scan lines 151 and signal lines 152. The liquid crystal panel performs a shutter action against light by applying a voltage between the pixel electrodes and a common electrode formed on the opposing substrate, thereby orienting the liquid crystal molecules in the desired direction.

[0067] In the region where the TFT substrate 111 and the opposing substrate 115 overlap, a frame-shaped peripheral light-shielding region 21 is formed around the periphery of the opposing substrate 115 by a black matrix. Light-shielding regions are also formed in the pattern region 22, but the light-shielding regions formed in the pattern region 22 are formed by a liquid crystal shutter, and their position and shape are variable. On the other hand, the frame-shaped light-shielding region 21 formed on the opposing substrate 115 is formed by a black matrix, so its shape is constant.

[0068] Figure 16 is a plan view of pixel 153. In Figure 16, pixel electrode 154 is formed in the region enclosed by scan line 151 and signal line 152. A TFT (Thin Film Transformer) that is switched by the scan signal is formed between pixel electrode 154 and signal line 152. The TFT is formed by a gate electrode 210 branched from scan line 151, a semiconductor film 211, a drain electrode 212 branched from signal line 153, and a source electrode 213, with the source electrode 213 connected to pixel electrode 154 via a through-hole 214. A signal is sent to pixel electrode 154 via signal line 152 and the TFT, causing the pixel 153 to open and close, forming an encoded aperture pattern 30.

[0069] Figure 17 is a cross-sectional view of a liquid crystal shutter. In Figure 17, a liquid crystal layer 300 is sandwiched between a TFT substrate 111 on which pixel electrodes 154 are formed and a counter substrate 115 on which common electrodes 155 are formed. The TFT substrate 111 and the counter substrate 115 are bonded together by a sealing material 150. Pixels 153 of the liquid crystal shutter are formed between the pixel electrodes 154 and the common electrodes 155.

[0070] A frame-shaped light-shielding film 21 made of a black matrix is ​​formed around the periphery of the opposing substrate 115. The TFT substrate 111 is formed larger than the opposing substrate 115, and the portion of the TFT substrate 111 that does not overlap with the opposing substrate 115 is a terminal region 160, where the driver IC 165 is located.

[0071] Since liquid crystals can only control polarized light, a first polarizing plate 410 is attached to the TFT substrate 111 side, and a second polarizing plate 420 is attached to the opposing substrate 115 side. Therefore, when using a liquid crystal shutter, light loss occurs due to the use of polarizing plates.

[0072] If a liquid crystal shutter is not used, the first aperture, which is a circular aperture, is superimposed with a second aperture having an encoded aperture pattern 30, and the first aperture is shifted by a first distance in a predetermined first direction perpendicular to the optical axis to capture a first image. Then, the first aperture is shifted by a first distance in the opposite direction to the first image to capture a second image and acquire data for DFD.

[0073] Since the second aperture on which the encoded aperture pattern 30 is formed is large, it does not need to be moved when taking the first and second images. However, the shape of the part of the encoded aperture pattern 30 that overlaps with the first aperture is almost the same when taking the first image and when taking the second image.

[0074] When capturing 2D images and acquiring data for DFD, the first, second, third, and fourth images are measured. The operation of the first and second apertures in this case can be approached using the same principles as the data acquisition method for 1D images described above.

[0075] Figures 18 to 21 show examples of coded aperture pattern sets. Images are captured using the two patterns shown in Figure 18(a) and Figure 18(b). The coded aperture patterns can be switched quickly by being controlled by the liquid crystal shutter. The coded aperture pattern in Figure 18(a) has a light-transmitting region that is tilted diagonally downward to the left. The coded aperture pattern in Figure 18(b) has a light-transmitting region that is tilted diagonally downward to the right. Because the coded aperture pattern in Figure 18 is a repeating pattern, the distance can be calculated with high accuracy using the same calculation process even when the light from the subject is shifted up, down, left, or right.

[0076] The encoded aperture pattern in Figure 19(a) has rectangular blocks with a triangular light-transmitting region in the upper left and a triangular light-blocking region in the lower right. These rectangular blocks are arranged in a matrix of 13 vertically and 13 horizontally. The encoded aperture pattern in Figure 19(b) has rectangular blocks with a triangular light-transmitting region in the upper right and a triangular light-blocking region in the lower left. These rectangular blocks are arranged in a matrix of 13 vertically and 13 horizontally. Because the encoded aperture pattern in Figure 19 is a repeating pattern, the distance can be calculated with high accuracy using the same calculation process even when the light from the subject is shifted vertically or horizontally.

[0077] The encoding aperture pattern in Figure 20(a) has a circular light-transmitting region in the lower left of the circular light-transmitting region. Two identical shapes are arranged vertically and horizontally. The encoding aperture pattern in Figure 20(b) has a circular light-transmitting region and an elliptical light-shielding region in the upper right. Two identical shapes are arranged vertically and horizontally.

[0078] The encoding aperture pattern in Figure 21(a) has a circular block with a circular light-shielding region in the lower left of a circular light-transmitting region. The circular blocks are arranged in a matrix of 6 vertically and 6 horizontally. The encoding aperture pattern in Figure 21(b) has a circular block with a circular light-transmitting region and an elliptical light-shielding region in the upper right. The circular blocks are arranged in a matrix of 6 vertically and 6 horizontally. [Explanation of symbols]

[0079] 10...Lens, 20...Aperture, 21...Frame made of light-shielding film, 22...Pattern area 22, 30...Encoded aperture pattern, 40...Image target, 50...Optical sensor, 111...TFT substrate, 150...Sealing material, 151...Scanning line, 152...Signal line, 153...Pixel, 154...Pixel electrode, 155...Common electrode, 160...Terminal area, 165...Driver IC, 210...Gate electrode, 211...Semiconductor film, 212...Drain electrode, 213...Source electrode, 214...Through-hole, 300...Liquid crystal layer, 410...First polarizer, 420...Second polarizer

Claims

1. An imaging device comprising a lens, a light sensor for receiving light, and an aperture positioned close to the lens and between the lens and the object to be imaged, The aperture comprises a first aperture that defines the imaging range and a second aperture having an encoded aperture pattern. When capturing the first image of the object to be imaged, the center of the first aperture is offset by a first distance in a first direction perpendicular to the optical axis of the lens, and the first aperture overlaps with the first region of the encoded aperture pattern. When capturing a second image of the object to be imaged, the center of the first aperture is offset by a first distance in the opposite direction to the first direction with respect to the optical axis of the lens, and the first aperture has a mechanism that overlaps with the second region of the encoded aperture pattern. An imaging device characterized in that the shape of the pattern in the first region of the encoded aperture pattern and the shape of the pattern in the second region are identical.

2. The imaging apparatus according to claim 1, characterized in that it has a mechanism for calculating the distance between the center of the lens and the imaging position of the object to be imaged using the data of the first image and the data of the second image.

3. The imaging apparatus according to claim 1, characterized in that the shape of the first aperture is circular.

4. The imaging apparatus according to claim 1, characterized in that the first aperture and the second aperture are separate components.

5. The imaging apparatus according to claim 1, characterized in that the first aperture and the second aperture are formed to be the same aperture.

6. The imaging device according to claim 5, characterized in that the aperture is composed of a liquid crystal shutter.

7. When capturing a third image of the object to be imaged, the center of the first aperture is in a direction perpendicular to the optical axis of the lens and in a third direction perpendicular to the first direction, with the first distance shift, and furthermore, the first aperture overlaps with the third region of the encoded aperture pattern. When capturing a fourth image of the object to be imaged, the center of the encoded aperture pattern of the second aperture is offset by a first distance in the direction opposite to the third direction with respect to the optical axis of the lens, and the first aperture has a mechanism that overlaps with the fourth region of the encoded aperture pattern. The imaging apparatus according to claim 1, characterized in that the shape of the pattern in the first region of the coding aperture pattern, the shape of the pattern in the second region, the shape of the pattern in the third region, and the shape of the pattern in the fourth region are the same.

8. The imaging apparatus according to claim 7, characterized in that it has a mechanism for calculating the distance between the center of the lens and the imaging position of the object to be imaged using the data of the first image, the data of the second image, the data of the third image, and the data of the fourth image.

9. The imaging apparatus according to claim 7, characterized in that the shape of the first aperture is circular.

10. The imaging apparatus according to claim 7, characterized in that the first aperture and the second aperture are separate components.

11. The imaging apparatus according to claim 7, characterized in that the first aperture and the second aperture are formed to be the same aperture.

12. The imaging device according to claim 11, characterized in that the aperture is composed of a liquid crystal shutter.

Images