Lens assembly
The lens assembly with a liquid crystal diaphragm and flexible wiring board facilitates high-speed pattern switching for precise distance measurement, addressing the challenge of 3D image reconstruction and all-in-focus image reproduction in imaging devices.
Patent Information
- Application Number
- JP2024044837
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-03-21
- Publication Date
- 2025-10-03
AI Technical Summary
Existing imaging devices struggle to measure distance from the lens to the subject and obtain distance data for forming 3D images or all-in-focus images without requiring complex setups or large-sized apertures.
A lens assembly incorporating a liquid crystal diaphragm with a flexible wiring board connection allows for high-speed pattern switching and precise distance measurement, using a coded aperture pattern to calculate distances from the lens to the pixel.
Enables efficient 3D image reconstruction and all-in-focus image reproduction by accurately measuring distances through a compact and versatile liquid crystal diaphragm configuration.
Smart Images
Figure 2025144916000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to an imaging device using a coded imaging method. [Background technology]
[0002] Imaging with a camera involves capturing a two-dimensional image from a three-dimensional world. In a normal camera, the image at the focal point is clear, but as the distance from the focal point increases, the image becomes blurred.
[0003] On the other hand, there is a demand for full-screen display, which allows for clear images to be displayed on the entire screen, or for obtaining 3D images. To achieve this, information on the distance between each position of the object to be imaged and the lens is required.
[0004] Non-Patent Document 1 describes a technique for measuring and calculating distance information while taking a photograph with a camera using a specially shaped coded aperture. Non-Patent Document 2 describes a technique for using a pair of coded apertures, one for dealing with image blur and the other for acquiring distance information. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Image and Depth from a Conventional Camera with a Coded Aperture Anat Levin Rob Fergus et al. [Non-patent document 2] Coded Aperture Pairs for Depth from Defocus and Defocus Deblurring Changyin Zhou Stephen Lin Shree K. Nayar Summary of the Invention [Problem to be solved by the invention]
[0006] One imaging technique that can measure the distance from the lens to the subject and obtain distance data for forming a 3D image or an all-in-focus image simply by taking a photograph is to use a specially shaped aperture pattern (hereinafter referred to as a coded aperture pattern). In other words, by taking a photograph using this coded aperture pattern, it is possible to calculate the distance from the lens to the pixel.
[0007] If this coded aperture pattern is made of liquid crystal, the degree of freedom in pattern formation can be increased. Hereinafter, this will be referred to as a liquid crystal aperture. A liquid crystal aperture has the advantage of being extremely small in size compared to a normal liquid crystal display device. Also, color images or gray displays are not required, and only black and white displays are sufficient. Instead, a clear difference between white and black displays is required. In other words, a large contrast is required between the display and black displays.
[0008] An object of the present invention is to realize a liquid crystal diaphragm suitable for forming such a coded diaphragm pattern. [Means for solving the problem]
[0009] The present invention is intended to solve the above problems, and the main specific means are as follows.
[0010] (1) A lens assembly comprising: a front lens having a first lens disposed on a first surface side of a liquid crystal diaphragm; a rear lens having a second lens disposed on a second surface side opposite the first surface of the liquid crystal diaphragm; the rear lens being housed in a lens barrel; the rear lens having a top surface and side walls; the side walls of the rear lens being housed in the lens barrel; the liquid crystal diaphragm being disposed on the top surface of the rear lens; a flexible wiring board for supplying electrical signals and power being connected to the liquid crystal diaphragm; a notch being formed in the side wall of the rear lens at a location corresponding to the flexible wiring board; the flexible wiring board passing through the notch and being pulled out to the outside from the top of the lens barrel.
[0011] (2) The lens assembly described in (1) is characterized in that the front lens has a side wall, the side wall of the front lens is housed within the side wall of the rear lens, and the flexible wiring board is pulled out to the outside through the space between the lens barrel and the side wall of the front lens.
[0012] (3) The lens assembly described in (1), characterized in that the front lens is placed on the liquid crystal diaphragm. [Brief explanation of the drawings]
[0013] [Figure 1] This is an optical model of an imaging device that uses a lens. [Figure 2] 10 is an example of a coded aperture pattern. [Figure 3] 10 is another example of a coded aperture pattern. [Figure 4] FIG. 10 is a cross-sectional view of an imaging system using an aperture having a coded aperture pattern. [Figure 5] 10 is an example of a coded aperture pattern. [Figure 6] 6 is a cross-sectional view taken along the line AA in FIG. 5. [Figure 7] FIG. 2 is a plan view showing the shape of an upper layer electrode. [Figure 8] FIG. 2 is a plan view showing the shape of a lower layer electrode. [Figure 9] FIG. 10 is a plan view showing an example of a divided lower electrode. [Figure 10] FIG. 10 is a plan view of a lens assembly according to a comparative example. [Figure 11] 11 is a cross-sectional view of FIG. 10 taken along line B-B. [Figure 12] FIG. 1 is a plan view of a lens assembly according to Example 1 before the front lens element is placed. [Figure 13] 13 is a cross-sectional view taken along CC in FIG. 12. [Figure 14] FIG. 1 is a plan view of a lens assembly according to a first embodiment. [Figure 15] FIG. 15 is a cross-sectional view taken along the line DD in FIG. DETAILED DESCRIPTION OF THE INVENTION
[0014] A camera is a means of capturing a 3D image as a 2D image. To reconstruct a 3D image or an all-in-focus image from this captured 2D image, it is necessary to know the distance from each imaging point to the center of the lens. Figure 1 shows an optical model of a camera using a lens. In Figure 1, when an object at a distance u is measured using a lens with a focal length f, all incident light is focused on a plane v according to the lens law shown in (Equation 1).
[0015]
number
[0016] If the position p of the imaging plane coincides with v, a focused image will be obtained, but if it is shifted forward or backward, the projected light rays will be projected as a circle of size b, as shown in (Equation 2). This circle is sometimes called the circle of confusion.
[0017]
number
[0018] In (Equation 2), a represents the aperture size. If the size of b exceeds the pixel size, blur will occur in the image. Since the camera's depth of field is limited, objects at depths far from the focal position will appear blurred in the image. The magnitude of this blur depends on the distance from the camera to the object, as shown in (Equation 1) and (Equation 2). Therefore, by measuring the blur, it is possible to conversely estimate the distance from the camera to the object being imaged. This method is called Depth From Defocus (DFD). For example, Levin et al. proposed a coded aperture pattern, such as the one shown in Figure 2, to effectively measure distance using DFD.
[0019] Incidentally, images captured by a camera are images that contain various degradation factors compared to an all-in-focus image (an ideal image with no blur across the entire screen). These degradation factors are expressed as a general blur function, PSF (Pont Spread Function). Here, if the blur function is represented by k, the image j captured by the camera can be expressed as the convolution of the all-in-focus image i and the blur function k, as shown in (Equation 3).
[0020]
number
[0021] That is, the all-in-focus image i can be restored by deconvolving the captured image j. Note that since calculating the all-in-focus image i requires calculating the distance to each imaging point, restoring the all-in-focus image is equivalent to restoring the distance from the lens center to the imaging point.
[0022] (Equation 4) is the inverse Fourier transform of (Equation 3).
[0023]
number
[0024] Here, the inverse function of the PSF, K -1 If is known, the frequency image I of the all-in-focus image can be obtained as shown in (Equation 5).
[0025]
number
[0026] Then, by inversely transforming I, the all-in-focus image i can be restored. As mentioned above, restoring the all-in-focus image i is equivalent to measuring the distance from the lens center to each imaging point on the subject. When an image is captured through the coded aperture pattern 30, the influence of the coded aperture pattern 30 becomes dominant on the blur function PSF.
[0027] The blur function k suitable for reproducing a general all-in-focus image is different from the blur function k suitable for distance measurement using a DFD. The blur function is determined by the coded aperture pattern 30. In order to perform accurate distance measurement and reproduce an all-in-focus image using this, Zhou proposes using a coded aperture pattern suitable for distance measurement using a DFD and a coded aperture pattern paired with data for reproducing an all-in-focus image, as shown in Figure 3.
[0028] Imaging devices that enable distance measurement using DFD, or reproduction of all-in-focus images using distance data, and even reproduction of three-dimensional images, are required to be able to handle various coded aperture patterns, and when using multiple coded aperture patterns, to have a configuration that allows for high-speed pattern switching.
[0029] The present invention achieves a configuration that satisfies these requirements by using a liquid crystal diaphragm, and also achieves a lens assembly including a liquid crystal diaphragm that is used in such an imaging configuration. [Example]
[0030] FIG. 4 is a cross-sectional view of a case where a lens 10 is used to capture an image of a target 40. In FIG. 4, the target 40 is on the right side of the lens 10, and an optical sensor 50 onto which an image is projected is on the left side. Hereinafter, the target 40 may also be referred to as the subject 40. However, in this case, the subject 40 refers not only to a small object but also to a broad range of objects including the background surrounding the object. A semiconductor imaging element such as a CMOS image sensor or a CCD image sensor is used as the optical sensor 50.
[0031] Generally, the refractive index of lens 10 increases with increasing distance from the center. Also, spherical aberration increases with increasing distance from the center. However, since Figure 4 is a cross-sectional view for the purpose of explanation, the spherical aberration of lens 10 is ignored. This also applies to Figure 4 and subsequent figures.
[0032] In FIG. 4 , light that leaves the center of subject 40, shown by the dotted line, is refracted by lens 10 and focused at the center of optical sensor (hereinafter simply referred to as sensor) 50. Light that leaves the upper end of subject 40 and passes through the center of lens 10 travels straight and forms an image at the lower end of sensor 50, as shown by the solid line. Light that leaves the upper part of subject 40 but does not pass through the center of lens 10 is refracted by lens 10 and forms an image at the lower part of sensor 50, as shown by the two-dot chain line. On the other hand, light that leaves the lower end of subject 40 and passes through the center of lens 10 travels straight and forms an image at the upper end of sensor 50, as shown by the solid line. Light that leaves the lower part of subject 40 but does not pass through the center of lens 10 is refracted by lens 10 and forms an image at the upper part of sensor 50, as shown by the one-dot chain line.
[0033] In Figure 4, an aperture 20 is placed between the lens and the subject, close to the lens. This aperture has a coded aperture pattern 30. Also, a second aperture 21 that determines the amount of light passing through is located outside the coded aperture pattern 30. In this specification, the coded aperture pattern 30 and the second aperture are collectively referred to as the aperture. However, the second aperture 21 is not essential. It is also possible for the outer frame of the coded aperture pattern 30 to serve as the second aperture 21.
[0034] Fig. 5 is a plan view of the diaphragm 20 when it is configured as a liquid crystal diaphragm. The liquid crystal diaphragm is configured such that a TFT substrate on which electrodes etc. are formed and an opposing substrate on which a light-shielding film etc. are formed are sealed at their peripheries with a sealant, and liquid crystal is disposed inside. In Fig. 5, a light-shielding film 201 is formed in a frame shape. The light-shielding film 201 is manufactured using the same material and process as a black matrix used in liquid crystal display devices etc. The light-shielding film 201 is formed on the opposing substrate.
[0035] A common electrode is formed on the opposing substrate in a planar shape. Pixel electrodes facing the common electrode are formed on the TFT substrate. In Fig. 5, columnar spacers 210 are arranged in the frame portion, overlapping the light-shielding film 201, to define the distance between the TFT substrate and the opposing substrate.
[0036] A coded aperture pattern 30 is formed inside a frame formed by the light-shielding film 201. As will be described with reference to FIGS. 7 and 8, the coded aperture pattern 30 can be formed on either the lower-layer electrode 101 or the upper-layer electrode 103. In FIG. 5, in order to display the coded aperture pattern 30 on the TFT substrate, the lower-layer electrode 101 and the upper-layer electrode 103 are formed with an insulating film sandwiched between them. The lower-layer electrode 101 and the upper-layer electrode 103 are fixed patterns. In FIG. 5, when the lower-layer electrode 101 is ON, the upper-layer electrode 103 is OFF, and when the lower-layer electrode 101 is OFF, the upper-layer electrode 103 is ON. This makes it possible to form two types of coded aperture patterns as needed.
[0037] Fig. 6 is a cross-sectional view taken along the line AA in Fig. 5. In Fig. 6, a counter substrate 200 having a common electrode 203 formed thereon is disposed opposite a TFT substrate 100 having aperture pattern electrodes 101 and 103 formed thereon, and a liquid crystal layer 300 is sandwiched between the TFT substrate 100 and the counter substrate 200. The TFT substrate 100 and the counter substrate 220 are bonded together at their peripheries by a sealant 150. The gap between the TFT substrate 100 and the counter substrate 200 is maintained by columnar spacers 210.
[0038] 6, a lower-layer electrode 101 is formed on a TFT substrate 100, an interlayer insulating film 102 is formed covering the lower-layer electrode 101, and an upper-layer electrode 103 is formed on the interlayer insulating film 102. The diaphragm pattern 30 that is formed differs depending on whether a voltage is applied to the lower-layer electrode 101 or the upper-layer electrode 103.
[0039] Fig. 7 shows an example of the coded aperture pattern 30 formed when only the upper layer electrode 103 is turned ON. The example in Fig. 7 is a combination of three rectangular and two L-shaped regions. Fig. 7 shows an example in which all five regions are turned ON, but a different coded aperture pattern 30 can be obtained by turning OFF any one, or two to four patterns.
[0040] Fig. 8 shows an example of a coded aperture pattern 30 formed when only the lower layer electrode 101 is turned on. The example in Fig. 8 is a pattern that corresponds to the gaps in the pattern in Fig. 7. The lower layer electrode shown in Fig. 8 has such a complex pattern, but by electrically isolating it in parts, it is possible to form various patterns.
[0041] Figure 9 shows an example in which the pattern in Figure 8 is divided into left and right halves. By using either pattern, different coded aperture patterns 30 can be formed. The width of the divided part should be as small as possible as long as electrical insulation is maintained. If the width is too wide, there is a risk of light leakage.
[0042] 5 and 7 to 9 are merely examples. Various types of coded aperture patterns 30 can be formed by freely changing the shapes of the upper layer electrode 103 and the lower layer electrode 101. However, in order to prevent light leakage, it is desirable that the gap formed by the upper layer electrode 103 be covered by the lower layer electrode 101 in a plan view.
[0043] Figures 10 and 11 show a specific assembly of a lens and a liquid crystal diaphragm as a comparative example. Hereinafter, including Figures 12 to 15, such a lens assembly of a liquid crystal diaphragm and a lens will be referred to as a lens assembly. Figure 10 is a plan view of the lens assembly as seen from above, and Figure 11 corresponds to the cross-sectional view taken along the line B-B of Figure 10. Incidentally, since Figure 4 is a schematic diagram for explanatory purposes, it is represented by a single lens. However, in reality, multiple lenses are used to address aberrations, etc.
[0044] In FIG. 11, above the liquid crystal diaphragm, two lenses 501 and 502 are arranged spaced apart in a metal container called front lens element 510. These lenses are also called front lenses. Below the liquid crystal diaphragm, two lenses 503 and 504 are arranged spaced apart in a metal container called rear lens element 520. These lenses are also called rear lenses. Lenses 501, 502, 503, 504, etc. are made of glass, for example. Rear lens element 520 is housed in lens barrel 530, which is made of metal. Note that front lens element 510 is housed within the side wall of rear lens element 520.
[0045] Examples of dimensions are as follows: outer diameter d1 of lens barrel 530 is, for example, 40 mm, and height h1 of lens barrel 530 is, for example, 40 mm. Opening diameter d2 of front lens 510 is, for example, 30 mm, and height h2 from the bottom surface of lens barrel 530 to the top surface of front lens 510 is, for example, 50 mm.
[0046] As shown in Fig. 10, the overall plan view of the lens assembly is circular. However, the liquid crystal diaphragm 600 is rectangular, as shown in Fig. 14, for example. The dimensions of the liquid crystal diaphragm 600 will be described with reference to Fig. 12 and the like.
[0047] 5 to 9, the aperture pattern of the liquid crystal diaphragm 600 is determined by supplying a signal from the outside. However, in the configurations of Figures 10 and 11, it is not possible to supply an electrical signal to the liquid crystal diaphragm 600.
[0048] 12 to 15 show the lens assembly configuration of Example 1, which solves the above-mentioned problems. The general configuration of Example 1 is such that a notch is formed in a portion of the side wall of the rear lens element on which liquid crystal diaphragm 600 is mounted, and flexible wiring board 700 connected to liquid crystal diaphragm 600 is pulled out through this notch along the inside of lens barrel 530. Thereafter, front lens element 510 is inserted into rear lens element 520, as in FIGS. 10 and 11. This configuration makes it possible to maintain the basic configuration and manufacturing process of the comparative example shown in FIGS. 10 and 11 without causing light leakage.
[0049] 12 and 13 are a plan view and a cross-sectional view showing the configuration before the front lens element 510 is inserted in Example 1. Fig. 13 corresponds to the cross-sectional view taken along CC in Fig. 12. In Fig. 12, the rear lens element 520 is housed in the lens barrel 530, and the liquid crystal diaphragm 600 is placed on the flat portion of the rear lens element 520.
[0050] The liquid crystal diaphragm 600 has a rectangular outer shape, and as explained in Fig. 6, the TFT substrate 100 and the counter substrate 200 are arranged facing each other, with liquid crystal 300 arranged between them. An effective area for forming an diaphragm pattern is formed where the TFT substrate 100 and the counter substrate 200 overlap. The TFT substrate 100 is formed larger than the counter substrate 200, and a terminal area 610 is formed where the TFT substrate 100 does not overlap with the counter substrate 200. A flexible wiring substrate 700 is connected to this terminal area 610 and is drawn out to the outside, and electrical signals and power are supplied to the liquid crystal diaphragm.
[0051] Examples of dimensions related to the liquid crystal diaphragm 600 are as follows: The area where the TFT substrate 100 and the counter substrate 200 overlap, forming the effective area, is, for example, a square, with wx and wy being, for example, about 25 mm. The width wt of the terminal area 610 to which the flexible wiring substrate 700 is connected is, for example, about 2.5 mm. The width wf of the flexible wiring substrate 700 is, for example, 10 mm. In the liquid crystal diaphragm 600, the plate thicknesses of the TFT substrate 100 and the counter substrate 200 are, for example, about 0.5 mm.
[0052] As shown in Figure 12, the side wall of rear lens 530 is cut out corresponding to the portion where flexible wiring board 700 is connected, and as shown in Figure 13, flexible wiring board 700 can be pulled out to the outside along lens barrel 530.
[0053] 14 and 15 are plan and cross-sectional views showing the configuration of Example 1, showing the state in which front lens element 510 has been inserted into the configurations of FIGS. 12 and 13. FIG. 15 corresponds to the cross-sectional view taken along the line DD of FIG. 14. As shown in FIG. 14, liquid crystal diaphragm 600 is covered by front lens element 510 having lens 501. However, flexible wiring board 700 connected to liquid crystal diaphragm 600 is drawn out from the upper end of lens barrel 530.
[0054] In Figure 15, the side wall of rear lens element 520 is cut out in the portion where flexible wiring board 700 is present, but this portion is covered by the side wall of front lens element 510, providing a sufficient light-blocking effect. A feature of Figure 15 is that front lens element 510 has a side wall, and the side wall of front lens element 510 and the side wall of lens barrel 530 have an overlapping portion. This can improve the effect of preventing light leakage.
[0055] The features of the present invention are as follows: The liquid crystal diaphragm 600 can be replaced with a liquid crystal diaphragm 600 having a different pattern, as needed. According to the configuration of the present invention described above, the liquid crystal diaphragm 600 can be replaced simply by removing the front lens element 510.
[0056] 5 to 9 are merely examples of the patterns formed on the liquid crystal diaphragm 600, and various diaphragm patterns can be formed depending on the shape of the electrodes formed on the TFT substrate 100. Also, the diaphragm patterns in the examples of Figures 5 to 9 are fixed patterns formed by fixed electrodes, but it is also possible to form diaphragm patterns of any shape by, for example, forming electrodes on the TFT substrate 100 side as pixel electrodes in a matrix. [Explanation of symbols]
[0057] 10...lens, 20...diaphragm, 21...frame made of light-shielding film, 22...pattern area 22, 30...coded diaphragm pattern, 40...image pickup target, 50...optical sensor, 100...TFT substrate, 101...lower layer electrode, 102...interlayer insulating film, 103...upper layer electrode, 200...opposing substrate, 201...light-shielding film (black matrix), 203...common electrode, 210...columnar spacer, 300...liquid crystal layer, 500...diaphragm-lens assembly, 501...lens, 502...lens, 503...lens, 504...lens, 510...front lens, 520...rear lens, 530...lens barrel, 600...liquid crystal diaphragm, 610...terminal area, 700...flexible wiring board
Claims
1. a front lens element having a first lens element is disposed on a first surface side of a liquid crystal diaphragm, and a rear lens element having a second lens element is disposed on a second surface side of the liquid crystal diaphragm opposite to the first surface, the rear lens element being a lens assembly housed in a lens barrel; the rear lens element has a top surface and a side wall; a side wall of the rear lens is housed in the lens barrel; the liquid crystal diaphragm is disposed on the upper surface of the rear lens; a flexible wiring board for supplying electric signals and power is connected to the liquid crystal diaphragm; a notch is formed in the side wall of the rear lens at a position corresponding to the flexible wiring board; The lens assembly is characterized in that the flexible wiring board passes through the notch and is drawn out from the upper part of the lens barrel to the outside.
2. the front lens has a side wall; 2. The lens assembly according to claim 1, wherein the side wall of the front lens is housed within the side wall of the rear lens, and the flexible wiring board is pulled out to the outside through a gap between the lens barrel and the side wall of the front lens.
3. 2. The lens assembly according to claim 1, wherein the front lens is mounted on the liquid crystal diaphragm.
4. The lens assembly of claim 1 , wherein the first lens comprises multiple lenses.
5. The lens assembly of claim 1 , wherein the second lens comprises a plurality of lenses.
6. 2. The lens assembly according to claim 1, wherein the electrodes of the liquid crystal diaphragm have fixed electrodes for forming an iris pattern.
7. 2. The lens assembly according to claim 1, wherein the front lens element, the rear lens element, and the lens barrel have a circular planar shape, and the liquid crystal diaphragm has a rectangular planar shape.
8. 2. The lens assembly according to claim 1, wherein the front lens element, the rear lens element, and the lens barrel are made of metal, and the liquid crystal diaphragm is made of glass.