Camera module

Abstract

The camera module according to an embodiment of the present invention comprises: an imaging element; a liquid crystal panel which is provided with an incident light control area including first to fourth light transmissive areas, a liquid crystal layer, and a driver for driving the liquid crystal layer; and a lens. The sizes of the third and fourth light transmissive areas are smaller than the sizes of the first and second light transmissive areas. A first distance to a subject is calculated on the basis of a first image based on light which has passed through the first light transmissive area and the lens and is made incident to the imaging element and a second image based on light which has passed through the second light transmissive area and the lens and is made incident to the imaging element. A second distance to the subject is calculated on the basis of a third image based on light which has passed through the third light transmissive area and the lens and is made incident to the imaging element and a fourth image based on light which has passed through the fourth light transmissive area and the lens and is made incident to the imaging element.

Description

Technical Field

[0001] Embodiments of the present invention relate to a camera module.

Background Art

[0002] In recent years, electronic devices such as smartphones equipped with a liquid crystal panel and a camera (imaging device) provided on the back surface of the liquid crystal panel have been put into practical use.

[0003] By the way, there is known an encoded aperture technique for calculating the distance from the camera to a subject in the image by using the blur generated in the image generated based on the light incident on the imaging device through the aperture of the camera included in the above-described electronic device.

[0004] However, in the encoded aperture technique, there are cases where an appropriate distance cannot be calculated from the image, and it is required to improve the accuracy of the calculated distance.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Therefore, the problem to be solved by the present invention is to provide a camera module capable of improving the accuracy of the distance calculated from an image.

Means for Solving the Problems

[0007] The camera module according to the embodiment comprises an image sensor, an incident light control region having first to fourth light-transmitting regions positioned to allow light to enter the image sensor, a liquid crystal panel having a liquid crystal layer positioned to overlap with the incident light control region, and a driver that drives the liquid crystal layer to transmit light through each of the first to fourth light-transmitting regions, and a lens positioned between the image sensor and the liquid crystal panel. The sizes of the third and fourth light-transmitting regions are smaller than the sizes of the first and second light-transmitting regions. A first distance to the subject in the first and second images is calculated based on a first image based on light that has passed through the first light-transmitting region and the lens and entered the image sensor by driving the liquid crystal layer, and a second image based on light that has passed through the second light-transmitting region and the lens and entered the image sensor by driving the liquid crystal layer. Based on the third image, which is formed by driving the liquid crystal layer and passing through the third light-transmitting region and the lens and into the image sensor, and the fourth image, which is formed by driving the liquid crystal layer and passing through the fourth light-transmitting region and the lens and into the image sensor, the second distance to the subject in the third and fourth images is calculated. The driver drives the liquid crystal layer to transmit light through the first and second light-transmitting regions when the distance to the subject is greater than or equal to a predetermined value, and drives the liquid crystal layer to transmit light through the third and fourth light-transmitting regions when the distance to the subject is less than the predetermined value. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an exploded perspective view showing an example configuration of an electronic device in an embodiment. [Figure 2] Figure 2 is a cross-sectional view showing the area around the camera of an electronic device. [Figure 3] Figure 3 is a plan view showing the arrangement of the LCD panel and camera. [Figure 4] Figure 4 is a cross-sectional view showing the pixels of a liquid crystal panel. [Figure 5] Figure 5 is a cross-sectional view showing the incident light control region of a liquid crystal panel. [Figure 6] Figure 6 is a diagram illustrating the overview of the camera module used to calculate the distance to the subject. [Figure 7]Figure 7 is a diagram illustrating the overview of the camera module used to calculate the distance to the subject. [Figure 8] Figure 8 is a diagram illustrating coded aperture pairs. [Figure 9] Figure 9 is a schematic plan view showing an example of the incident light control region of a liquid crystal panel. [Figure 10] Figure 10 is a flowchart showing an example of how the camera module operates when calculating the distance to the subject. [Figure 11] Figure 11 is a schematic plan view showing another example of the incident light control region of a liquid crystal panel. [Figure 12] Figure 12 is a schematic plan view showing yet another example of the incident light control region of a liquid crystal panel. [Modes for carrying out the invention]

[0009] Embodiments will be described below with reference to the drawings. Note that the disclosure is merely an example, and modifications that can be easily conceived by those skilled in the art while maintaining the spirit of the invention are naturally included within the scope of the present invention. Furthermore, in order to clarify the explanation, the drawings may schematically represent the width, thickness, shape, etc., of each part compared to the actual embodiment, but these are merely examples and do not limit the interpretation of the present invention. Also, in this specification and in each drawing, elements similar to those described above in previously shown drawings are denoted by the same reference numerals, and detailed explanations may be omitted as appropriate.

[0010] Figure 1 is an exploded perspective view showing one example configuration of the electronic device 100 in this embodiment. As shown in Figure 1, directions X, Y, and Z are orthogonal to each other, but they may intersect at angles other than 90 degrees.

[0011] The electronic device 100 includes a liquid crystal display device DSP as a display device and a camera 1. The liquid crystal display device DSP includes a liquid crystal panel PNL as a display panel and an illumination device (backlight) IL.

[0012] The lighting device IL includes a light guide LG1, a light source EM, and a case CS. Such a lighting device IL illuminates a liquid crystal panel PNL schematically shown by a dashed line in FIG. 1, for example.

[0013] The light guide LG1 is formed in a flat plate shape parallel to the X-Y plane defined by the direction X and the direction Y. The light guide LG1 faces the liquid crystal panel PNL. The light guide LG1 has a side surface SA, a side surface SB opposite to the side surface SA, and a through hole h1 surrounding the camera 1. The side surfaces SA and SB each extend in the direction X. For example, the side surfaces SA and SB are surfaces parallel to the X-Z plane defined by the direction X and the direction Z. The through hole h1 penetrates the light guide LG1 along the direction Z. The through hole h1 is located between the side surfaces SA and SB in the direction Y and is closer to the side surface SB than the side surface SA.

[0014] The plurality of light sources EM are arranged side by side at intervals in the direction X. Each of the light sources EM is mounted on a wiring board F1 and is electrically connected to the wiring board F1. The light source EM is, for example, a light emitting diode (LED) and emits white illumination light. The illumination light emitted from the light source EM enters the light guide LG1 from the side surface SA and travels inside the light guide LG1 from the side surface SA toward the side surface SB.

[0015] The case CS houses the light guide LG1 and the light source EM. The case CS has side walls W1 to W4, a bottom plate BP, a through hole h2, and a protrusion PP. The side walls W1 and W2 extend in the direction X and face each other in the direction Y. The side walls W3 and W4 extend in the direction Y and face each other in the direction X. The through hole h2 overlaps the through hole h1 in the direction Z. The protrusion PP is fixed to the bottom plate BP. The protrusion PP protrudes from the bottom plate BP toward the liquid crystal panel PNL along the direction Z and surrounds the through hole h2.

[0016] The light guide LG1 overlaps the liquid crystal panel PNL. The camera 1 is mounted on the wiring board F2 and electrically connected to the wiring board F2. The camera 1 faces the liquid crystal panel PNL through the through-hole h2, inside the protrusion PP, and the through-hole h1.

[0017] FIG. 2 is a cross-sectional view showing the periphery of the camera 1 of the electronic device 100. As shown in FIG. 2, the lighting device IL further includes a light reflection sheet RS, a light diffusion sheet SS, and prism sheets PS1 and PS2.

[0018] The light reflection sheet RS, the light guide LG1, the light diffusion sheet SS, and the prism sheets PS1 and PS2 are arranged in order in the Z direction and housed in the case CS. The case CS includes a metal case CS1 and a resin light shielding wall CS2 as a peripheral member. The light shielding wall CS2 is adjacent to the camera 1 and forms the protrusion PP together with the case CS1. The light shielding wall CS2 is located between the camera 1 and the light guide LG1 and forms a cylindrical shape. The light shielding wall CS2 is formed of a resin that absorbs light, such as black resin. The light diffusion sheet SS, and the prism sheets PS1 and PS2 each have a through-hole overlapping the through-hole h1. The protrusion PP is located inside the through-hole h1.

[0019] [[ID=|11]] The liquid crystal panel PNL further has polarizing plates PL1 and PL2. The liquid crystal panel PNL and the cover glass CG as a cover member are arranged in the Z direction and constitute a liquid crystal element LCD having an optical switching function with respect to the light traveling in the Z direction. The liquid crystal element LCD is attached to the lighting device IL by an adhesive tape TP1. The adhesive tape TP1 is adhered to the protrusion PP, the prism sheet PS2, and the polarizing plate PL1.

[0020] The liquid crystal panel PNL may have any configuration that supports a display mode that utilizes a transverse electric field along the main surface of the substrate, a display mode that utilizes a longitudinal electric field along the normal to the main surface of the substrate, a display mode that utilizes a gradient electric field that is tilted diagonally with respect to the main surface of the substrate, or a display mode that utilizes a combination of the transverse, longitudinal, and gradient electric fields as appropriate. Here, the main surface of the substrate is the surface parallel to the XY plane.

[0021] The liquid crystal panel PNL comprises a display area DA for displaying images, a non-display area NDA outside the display area DA, and an incident light control area PCA surrounded by the display area DA and having a circular shape. In this embodiment, the incident light control area PCA is described as being circular, but the shape of the incident light control area PCA may be other than circular.

[0022] The liquid crystal panel PNL comprises a first substrate SUB1, a second substrate SUB2, a liquid crystal layer LC, and a sealing material SE.

[0023] The sealing material SE is located in the non-display area NDA and joins the first substrate SUB1 and the second substrate SUB2. The liquid crystal layer LC is positioned to overlap with the display area DA and the incident light control area PCA, and is held between the first substrate SUB1 and the second substrate SUB2. The liquid crystal layer LC is formed in the space enclosed by the first substrate SUB1, the second substrate SUB2, and the sealing material SE.

[0024] An image is displayed in the display area DA by controlling the amount of light transmitted from the lighting device IL using the liquid crystal panel PNL. The user of the electronic device 100 is positioned on the Z-direction side of the cover glass CG and views the light emitted from the liquid crystal panel PNL as an image. On the other hand, in the incident light control area PCA, the amount of light transmitted is also controlled by the liquid crystal panel PNL, but the light enters the camera 1 from the Z-direction side of the cover glass CG through the liquid crystal panel PNL.

[0025] In this embodiment, the light that travels from the lighting device IL through the liquid crystal panel PNL to the cover glass CG is referred to as the emitted light, and the light that travels from the cover glass CG through the liquid crystal panel PNL to the camera 1 is referred to as the incident light.

[0026] Here, the main parts of the first substrate SUB1 and the second substrate SUB2 will be described. The first substrate SUB1 comprises an insulating substrate 10 and an alignment film AL1. The second substrate SUB2 comprises an insulating substrate 20, a color filter CF, a light-shielding layer BM, a transparent layer OC, and an alignment film AL2.

[0027] The insulating substrates 10 and 20 are transparent substrates such as glass substrates or flexible resin substrates. The alignment films AL1 and AL2 are in contact with the liquid crystal layer LC.

[0028] The color filter CF, light-shielding layer BM, and transparent layer OC are located between the insulating substrate 20 and the liquid crystal layer LC. In the illustrated example, the color filter CF is provided on the second substrate SUB2, but it may also be provided on the first substrate SUB1. The color filter CF is located in the display area DA.

[0029] The incident light control region PCA includes at least a light-shielding region LSA located on the outermost periphery and having an annular shape, and a light-transmitting region TA surrounded by the light-shielding region LSA and in contact with the light-shielding region LSA.

[0030] The light-shielding layer BM includes a light-shielding portion located in the display area DA that demarcates pixels, and a light-shielding portion on a frame located in the non-display area NDA. Furthermore, in the incident light control area PCA, the light-shielding layer BM is located at least in the light-shielding area LSA and forms an aperture OP1 located in the light-transmitting area TA.

[0031] The transparent layer OC is in contact with the color filter CF in the display area DA, in contact with the light-shielding layer BM in the non-display area NDA, in contact with the light-shielding layer BM in the light-shielding area LSA, and in contact with the insulating substrate 20 in the light-transmitting area TA.

[0032] Alignment films AL1 and AL2 are provided across the display area DA, the incident light control area PCA, and the non-display area NDA.

[0033] Details of the color filter CF are omitted here, but the color filter CF includes, for example, a red coloring layer placed on pixels that display red, a green coloring layer placed on pixels that display green, and a blue coloring layer placed on pixels that display blue. The color filter CF may also include a transparent resin layer placed on pixels that display white. The transparent layer OC covers the color filter CF and the light-shielding layer BM. The transparent layer OC is, for example, a transparent organic insulating layer.

[0034] Camera 1 is located inside the through-hole h2 of the case CS. In direction Z, camera 1 overlaps the cover glass CG and the liquid crystal panel PNL. The liquid crystal panel PNL may further include optical sheets other than polarizers PL1 and PL2 in the incident light control region PCA. Examples of such optical sheets include phase difference plates, light scattering layers, and anti-reflective layers. In the electronic device 100 equipped with the liquid crystal panel PNL and camera 1, camera 1 is located on the back side of the liquid crystal panel PNL as seen from the user of the electronic device 100.

[0035] Camera 1 comprises, for example, an optical system 2 including at least one lens, an image sensor 3, and a case 4. The image sensor 3 includes an imaging surface 3a facing the liquid crystal panel PNL. The optical system 2 faces the incident light control region PCA of the liquid crystal panel PNL. The optical system 2 is located between the imaging surface 3a and the liquid crystal panel PNL and includes a light-receiving surface 2a facing the liquid crystal panel PNL. The light-receiving surface 2a overlaps the incident light control region PCA. The optical system 2 is positioned with a gap between it and the liquid crystal panel PNL. The case 4 houses the optical system 2 and the image sensor 3.

[0036] Although not shown in Figure 2, a light source for illuminating the subject to be photographed by camera 1 may be provided at the top of case 4.

[0037] The image sensor 3 of camera 1 receives light through the cover glass CG, liquid crystal panel PNL, and optical system 2. The image sensor 3 is configured to convert incident light that has passed through the incident light control area PCA of the liquid crystal panel PNL and the optical system 2 into an image (data). Although camera 1 (image sensor 3) is configured to convert visible light (for example, light in the wavelength range of 400 nm to 700 nm) that has passed through the cover glass CG and liquid crystal panel PNL into an image, it may also be configured to convert infrared light (for example, light in the wavelength range of 800 nm to 1500 nm) into an image.

[0038] Polarizing plate PL1 is bonded to insulating substrate 10. Polarizing plate PL2 is bonded to insulating substrate 20. Cover glass CG is attached to polarizing plate PL2 by transparent adhesive layer AD.

[0039] Furthermore, in order to prevent the influence of electric fields from the liquid crystal layer (LC), a transparent conductive layer may be provided between the polarizing plate PL2 and the insulating substrate 20. Such a transparent conductive layer is formed of a transparent conductive material such as indium tin oxide (ITO) or indium zinc oxide (IZO).

[0040] The liquid crystal panel PNL has a first surface S1 on the side that displays the image and a second surface S2 on the opposite side of the first surface S1. In this embodiment, the polarizing plate PL2 has the first surface S1, and the polarizing plate PL1 has the second surface S2.

[0041] The display area DA and the incident light control area PCA are regions that overlap with the first substrate SUB1, the second substrate SUB2, and the liquid crystal layer LC.

[0042] Figure 3 is a plan view showing the arrangement of the liquid crystal panel PNL and camera 1 as shown in Figure 2. Figure 3 also shows the equivalent circuit of a single pixel PX.

[0043] As shown in Figure 3, the display area DA is essentially a rectangular area, but its four corners may be rounded, and it may also be a polygon or circular area other than a rectangle. The display area DA is surrounded by the sealing material SE.

[0044] The liquid crystal panel PNL has a pair of short sides E11 and E12 extending along direction X, and a pair of long sides E13 and E14 extending along direction Y. The liquid crystal panel PNL has a plurality of pixels PX arranged in a matrix in directions X and Y within the display area DA. Each pixel PX in the display area DA has the same circuit configuration.

[0045] As shown in the enlarged view in Figure 3, each pixel PX is equipped with a switching element SW, a pixel electrode PE, a common electrode CE, a liquid crystal layer LC, and a capacitance CP, etc.

[0046] The switching element SW is composed of, for example, a thin-film transistor (TFT). The switching element SW is electrically connected to one corresponding scan line G out of a plurality of scan lines G, one corresponding signal line S out of a plurality of signal lines S, and the pixel electrode PE. A control signal for controlling the switching element SW is supplied to the scan line G. An image signal, such as a video signal, is supplied to the signal line S as a signal different from the control signal.

[0047] A common voltage is applied to the common electrode CE. The liquid crystal layer LC is driven by the voltage (electric field) generated between the pixel electrode PE and the common electrode CE.

[0048] Capacitance CP is formed, for example, between an electrode at the same potential as the common electrode CE and an electrode at the same potential as the pixel electrode PE.

[0049] The liquid crystal panel PNL further comprises a wiring board 5 and a driver 6 (display driver). The wiring board 5 is mounted on an extension Ex of the first substrate SUB1 and connected to the extension Ex. The driver 6 is mounted on the wiring board 5 and electrically connected to the wiring board 5. Alternatively, the driver 6 may be mounted on the extension Ex and electrically connected to the extension Ex. The driver 6 is configured to drive the liquid crystal panel PNL (liquid crystal layer LC) by outputting signals necessary for image display, for example, and is mounted as an IC chip. The wiring board 5 may be a bendable flexible printed circuit board.

[0050] In Figure 3, the electronic device 100 is equipped with a camera 1 within the display area DA, which is located in the upper center of the display area DA in the plan view shown in Figure 3. Furthermore, an incident light control area PCA is located at a position overlapping with the camera 1.

[0051] Figure 4 is a cross-sectional view showing the pixel PX of a liquid crystal panel PNL. Figure 4 shows the configuration of a liquid crystal panel PNL that corresponds to FFS (Fringe Field Switching) mode, one of the display modes that utilizes a transverse electric field.

[0052] As shown in Figure 4, the first substrate SUB1 includes an insulating layer 11, signal lines S, an insulating layer 12, a common electrode CE, a metal layer ML, an insulating layer 13, and a pixel electrode PE between the insulating substrate 10 and the alignment film AL1. A polarizing plate PL1 is also formed on the outside of the first substrate SUB1.

[0053] The insulating layer 11 is provided on the insulating substrate 10. While a detailed explanation is omitted here, the scanning line G, the gate electrode and semiconductor layer of the switching element SW, and other insulating layers are arranged between the insulating substrate 10 and the insulating layer 11. The signal line S is formed on the insulating layer 11. The insulating layer 12 is provided on the insulating layer 11 and the signal line S.

[0054] The common electrode CE is provided on the insulating layer 12. The metal layer ML is provided on the common electrode CE and is in contact with the common electrode CE. The metal layer ML is located directly above the signal line S. In the example shown in Figure 4, the first substrate SUB1 is provided with the metal layer ML, but the metal layer ML may be omitted. The insulating layer 13 is provided on the common electrode CE and the metal layer ML.

[0055] The pixel electrodes PE are formed on the insulating layer 13. Each pixel electrode PE is positioned between adjacent signal lines S and faces the common electrode CE. The pixel electrodes PE also have slits at positions facing the common electrode CE. The common electrode CE and the pixel electrodes PE are formed from transparent conductive materials such as ITO and IZO.

[0056] The insulating layer 13 is sandwiched between the common electrode CE and the pixel electrode PE. The alignment film AL1 is provided on top of the insulating layer 13 and the pixel electrode PE, and covers the pixel electrode PE and the like.

[0057] On the other hand, the second substrate SUB2 is equipped with a light-shielding layer BM, a color filter CF, a transparent layer OC, and an alignment film AL2 on the side of the insulating substrate 20 facing the first substrate SUB1.

[0058] The light-shielding layer BM is formed on the inner surface of the insulating substrate 20. The light-shielding layer BM is located directly above the signal line S and the metal layer ML. The color filter CF is formed on the inner surface of the insulating substrate 20 and partially overlaps with the light-shielding layer BM. The transparent layer OC covers the color filter CF. The alignment film AL2 covers the transparent layer OC. In addition, a polarizing plate PL2 is formed on the outside of the second substrate SUB2.

[0059] Furthermore, the liquid crystal panel PNL can be configured without forming a light-shielding layer BM in the display area DA. In this case, for example, a metal layer ML can be formed in a grid pattern in the display area DA, and the light-shielding function can be provided by the metal layer ML instead of the light-shielding layer BM.

[0060] Here, for example, if the transmission axes of polarizers PL1 and PL2 are orthogonal, and no voltage (electric field) is generated between the pixel electrode PE and the common electrode CE at pixel PX, and in the off state where no voltage is applied to the liquid crystal layer LC, the liquid crystal molecules contained in the liquid crystal layer LC are initially oriented in the direction of the transmission axis of polarizer PL1 between the alignment films AL1 and AL2. Therefore, no phase difference occurs in the liquid crystal layer LC, and because the transmission axes of polarizers PL1 and PL2 are orthogonal, pixel PX has minimum transmittance and displays black. In other words, the liquid crystal panel PNL exhibits a light-shielding function at pixel PX.

[0061] On the other hand, in the ON state, when a voltage (electric field) is applied to the liquid crystal layer LC between the pixel electrode PE and the common electrode CE in the pixel PX, the liquid crystal molecules align in a direction different from their initial orientation direction, and this orientation direction is controlled by the electric field. Therefore, a phase difference occurs in the liquid crystal layer LC, and in the pixel PX, the liquid crystal panel PNL exhibits a light transmission (light-transmitting) function. As a result, the ON state of the pixel PX displays a color corresponding to the color filter.

[0062] In this embodiment, the liquid crystal panel PNL is assumed to be a normally black type that displays black when off, but it may also be a normally white type that displays black when on (and white when off).

[0063] Of the pixel electrode PE and the common electrode CE, the electrode closer to the liquid crystal layer LC is the pixel electrode PE, and this pixel electrode PE functions as a display electrode as described above. However, the electrode closer to the liquid crystal layer LC of the pixel electrode PE and the common electrode CE may be the common electrode CE. In this case, the common electrode CE is formed to have a slit and functions as a display electrode.

[0064] Figure 5 is a cross-sectional view showing the incident light control area (PCA) of a liquid crystal panel (PNL). In Figure 5, signal lines S and scan lines G, etc., are omitted. Also, in Figure 5, the same reference numerals are used for the same parts as in Figure 4, and their detailed explanations are omitted.

[0065] As shown in Figure 5, one of the two conductors formed with the insulating layer 13 in between is provided in the same layer as one of the pixel electrodes PE and the common electrode CE, and is made of the same material as that electrode. The other conductor of the two conductors described above is provided in the same layer as the other electrode of the pixel electrode PE and the common electrode CE, and is made of the same material as that electrode.

[0066] In Figure 5, the first wiring WL1 and the first control electrode RL1 are provided on an insulating layer 12 and covered with an insulating layer 13. The first wiring WL1 and the first control electrode RL1 are provided in the same layer as the common electrode CE and are made of the same transparent conductive material as the common electrode CE.

[0067] The second wiring WL2 and the second control electrode RL2 are provided on the insulating layer 13 and covered with the alignment film AL1. The second wiring WL2 and the second control electrode RL2 are provided in the same layer as the pixel electrode PE and are made of the same transparent conductive material as the pixel electrode PE.

[0068] In the example shown in Figure 5, the insulating layer 13 is sandwiched between the first control electrode RL1 and the second control electrode RL2, but the first control electrode RL1 and the second control electrode RL2 may be formed in the same layer.

[0069] In the incident light control region PCA, the alignment film AL1 covers the second wiring WL2 and the second control electrode RL2, and is in contact with the liquid crystal layer LC.

[0070] In the second substrate SUB2, the color filter CF is not provided in the incident light control region PCA.

[0071] Voltages generated by the first control electrode RL1 and the second control electrode RL2 are applied to the liquid crystal layer LC. In this case, a first control voltage is applied to the first control electrode RL1 and a second control voltage is applied to the second control electrode RL2 via a wiring harness (not shown). The voltage level of the first control voltage may be the same as that of either the image signal or the common voltage, and the voltage level of the second control voltage may be the same as that of the other of the image signal or the common voltage.

[0072] Furthermore, the first control voltage may have a voltage level of first polarity relative to the common voltage, and the second control voltage may have a voltage level of second polarity relative to the common voltage. Note that one of the first polarity and the second polarity described above is positive polarity, and the other is negative polarity.

[0073] In this embodiment, the liquid crystal layer LC, which is positioned to overlap with the incident light control region PCA, is driven to transmit light into the light transmission region TA. This control is performed by a driver 6 provided in the liquid crystal panel PNL.

[0074] If the liquid crystal panel PNL described above is, for example, a normally black method, the driver 6 applies a voltage generated by the first control electrode RL1 and the second control electrode RL2 to the liquid crystal layer LC, which is formed in a position overlapping with the incident light control region PCA (light transmission region TA), thereby turning on the liquid crystal layer LC and allowing light to pass through the light transmission region TA. As a result, the light that has passed through the light transmission region TA is incident on the image sensor 3, and the image sensor 3 can convert the incident light into an image.

[0075] On the other hand, the driver 6 can also prevent light from passing through the light-transmitting region TA by not applying a voltage to the liquid crystal layer LC, which is formed in a position overlapping with the incident light control region PCA (light-transmitting region TA) (i.e., turning off the liquid crystal layer LC).

[0076] In this embodiment, we consider the case where the distance from the camera 1 to the subject in the image (hereinafter simply referred to as the distance to the subject) is calculated using an image based on light that has passed through the light-transmitting region TA and entered the image sensor 3 (i.e., an image of the subject captured by the camera 1).

[0077] One technique for calculating the distance to a subject from an image is coded aperture technology. While a detailed explanation is omitted here, coded aperture technology calculates the distance to a subject by analyzing the blur that occurs in the image depending on the subject's position.

[0078] In other words, by utilizing the above-described encoded aperture technology, the electronic device 100 can be used for purposes such as calculating the distance of an object based on an image and creating a depth map representing the distance of the object. The processes of calculating the distance of the object and creating the depth map can be performed by a CPU or the like provided in the electronic device 100, which operates according to a predetermined application program.

[0079] The following describes the overview of the camera module used to calculate the distance to the subject as described above, with reference to Figures 6 and 7. In this embodiment, the camera module comprises a camera 1 (optical system 2 including a lens and an image sensor 3) for photographing the subject, and a liquid crystal panel PNL that drives the liquid crystal layer LC to cause light to enter the image sensor 3 through the light transmission region TA.

[0080] Figure 6 shows the positional relationship between the camera module and the subject. First, let's consider the case where we calculate the distance to subject 200a, which is located relatively far from camera 1 (electronic device 100), in Figure 6. Generally, in camera 1, for example, by changing the distance between lens 2b included in optical system 2 and image sensor 3, subject 200a can be photographed in focus. However, as shown in Figure 6, if subject 200a is photographed when it is not in focus, a discrepancy occurs between the image formation position and the position of the imaging surface 3a of image sensor 3, resulting in blurring of the image based on the light incident on image sensor 3.

[0081] According to the encoding aperture technique described above, the distance to subject 200a is calculated based on the blurring that occurs in the image.

[0082] Next, let's consider the case where we calculate the distance from camera 1 to subject 200b, which is relatively close. When calculating the distance to subject 200b, as described above, subject 200b is photographed when it is out of focus. However, as shown in Figure 6, when the distance from camera 1 to subject 200b is short, some of the light that passes through the light transmission area TA and lens 2b does not enter the image sensor 3. In this case, even if the distance to subject 200b is calculated from the image based on the light that entered the image sensor 3, the information of the light that did not enter the image sensor 3 (blur) cannot be used in calculating the distance to subject 200b, so it is possible that an error will occur in the distance (i.e., the accuracy of the distance will be low).

[0083] In this case, for example, as shown in Figure 7, by reducing the size of the light transmission region TA, all the light that passes through the light transmission region TA and lens 2b can be incident on the image sensor 3. This improves the accuracy of the distance to the subject 200b compared to the case where the size of the light transmission region TA is large, as shown in Figure 6 above.

[0084] Furthermore, reducing the size of the light-transmitting region TA reduces the amount of light incident on the image sensor 3, so it is not always desirable to use a small-sized light-transmitting region TA.

[0085] Therefore, in this embodiment, a configuration is adopted that allows switching the size of the light transmission region TA, taking into consideration the case when the subject is far from the camera 1 and the case when it is close to the camera 1. Furthermore, as described above, when calculating the distance of the subject from the image, it is considered that preparing two light transmission regions TA (encoded aperture pairs), as shown in Figure 8, and using the image (and the resulting blur) based on the light transmitted through one light transmission region TA and the image (and the resulting blur) based on the light transmitted through the other light transmission region TA contributes to improving the accuracy of the distance.

[0086] The configuration of the camera module according to this embodiment will be described below. Figure 9 is a schematic plan view showing the incident light control area (PCA) of the liquid crystal panel (PNL) provided in the camera module of this embodiment.

[0087] As shown in Figure 9, in the camera module CM according to this embodiment, the liquid crystal panel PNL includes an incident light control region PCA having a plurality of light transmission regions TA. Specifically, in the example shown in Figure 9, the incident light control region PCA has first to fourth light transmission regions TA1 to TA4.

[0088] In this embodiment, the size and shape of the first light-transmitting region TA1 are the same as those of the second light-transmitting region TA2. The first light-transmitting region TA1 and the second light-transmitting region TA2 constitute the first coding aperture pair.

[0089] Furthermore, the size and shape of the third light-transmitting region TA3 are identical to those of the fourth light-transmitting region TA4. The third light-transmitting region TA3 and the fourth light-transmitting region TA4 constitute the second coding aperture pair.

[0090] Furthermore, in this embodiment, the sizes of the first light-transmitting region TA1 and the second light-transmitting region TA2 are smaller than the sizes of the third light-transmitting region TA3 and the fourth light-transmitting region TA4.

[0091] In other words, in this embodiment, two coded aperture pairs are prepared, and one of the two coded aperture pairs is formed to the size of a pinhole, making it possible to calculate the distance to the subject with high accuracy even when the subject is located at a close distance. To put it another way, the first coded aperture pair, consisting of a first light-transmitting region TA1 and a second light-transmitting region TA2, is suitable when the subject is located at a close distance from the camera 1, and the second coded aperture pair, consisting of a third light-transmitting region TA3 and a fourth light-transmitting region TA4, is suitable when the subject is located at a medium or long distance from the camera 1.

[0092] Furthermore, for apertures used at short distances, the size and shape of the first light-transmitting region TA1 do not have to be the same as those of the second light-transmitting region TA2. Also, for medium or long distances, the size and shape of the third light-transmitting region TA3 do not have to be the same as those of the fourth light-transmitting region TA4.

[0093] Next, referring to the flowchart in Figure 10, we will explain an example of how the camera module operates when calculating the distance to the subject.

[0094] First, in order to calculate the distance to the subject, it is necessary to photograph the subject with camera 1. In this case, the driver 6 provided in the liquid crystal panel PNL applies a voltage to the liquid crystal layer LC (hereinafter referred to as the liquid crystal layer LC of the first light-transmitting region TA1) formed in a position overlapping with the first light-transmitting region TA1, in order to transmit light through the first light-transmitting region TA1, thereby making the liquid crystal layer LC transparent (step S1). In this case, the liquid crystal layer LC (hereinafter referred to as the liquid crystal layer LC of the second light-transmitting region TA2), the liquid crystal layer LC (hereinafter referred to as the liquid crystal layer LC of the third light-transmitting region TA3), and the liquid crystal layer LC (hereinafter referred to as the liquid crystal layer LC of the fourth light-transmitting region TA4) formed in a position overlapping with the second light-transmitting region TA2, the liquid crystal layer LC (hereinafter referred to as the liquid crystal layer LC of the third light-transmitting region TA3), and the liquid crystal layer LC (hereinafter referred to as the liquid crystal layer LC of the fourth light-transmitting region TA4) formed in a position overlapping with the fourth light-transmitting region TA4 are in an opaque state.

[0095] If the process in step S1 is executed, an image based on the light (incident light) that has passed through the first light transmission region TA1 and been incident on the image sensor 3 (hereinafter referred to as the first image) is output (step S2).

[0096] Next, the driver 6 makes the liquid crystal layer LC of the second light-transmitting region TA2 transparent in order to transmit light through the second light-transmitting region TA2 (step S3). In this case, the liquid crystal layers LC of the first light-transmitting region TA1, the third light-transmitting region TA3, and the fourth light-transmitting region TA4 are opaque.

[0097] If the process in step S3 is executed, an image based on the light (incident light) that has passed through the second light transmission region TA2 and been incident on the image sensor 3 (hereinafter referred to as the second image) is output (step S4).

[0098] Here, the electronic device 100 (its CPU, etc.) performs a process to calculate the distance of the subject based on the blur that occurs in the first image output in step S2 and the second image output in step S4.

[0099] Next, the driver 6 makes the liquid crystal layer LC of the third light-transmitting region TA3 transparent in order to transmit light through the third light-transmitting region TA3 (step S5). In this case, the liquid crystal layers LC of the first light-transmitting region TA1, the second light-transmitting region TA2, and the fourth light-transmitting region TA4 are opaque.

[0100] If the process in step S5 is executed, an image based on the light (incident light) that has passed through the third light transmission region TA3 and been incident on the image sensor 3 (hereinafter referred to as the third image) is output (step S6).

[0101] Furthermore, the driver 6 makes the liquid crystal layer LC of the fourth light-transmitting region TA4 transparent in order to transmit light through the fourth light-transmitting region TA4 (step S7). In this case, the liquid crystal layers LC of the first light-transmitting region TA1, the second light-transmitting region TA2, and the third light-transmitting region TA3 are opaque.

[0102] If the process in step S7 is executed, an image based on the light (incident light) that has passed through the fourth light transmission region TA4 and been incident on the image sensor 3 (hereinafter referred to as the fourth image) is output (step S8).

[0103] Here, the electronic device 100 (its CPU, etc.) performs a process to calculate the distance of the subject based on the blurring that occurs in the third image output in step S6 and the fourth image output in step S8 described above.

[0104] As described above, the camera module CM according to this embodiment comprises an image sensor 3, a liquid crystal panel PNL, and a lens 2b. The liquid crystal panel PNL comprises an incident light control region PCA having first to fourth light transmission regions TA1 to TA4 positioned to allow light to enter the image sensor 3, a liquid crystal layer LC positioned to overlap with the incident light control region PCA, and a driver 6 that drives the liquid crystal layer LC to transmit light through each of the first to fourth light transmission regions TA1 to TA4. In this embodiment, the sizes of the third and fourth light transmission regions TA3 and TA4 are smaller than the sizes of the first and second light transmission regions TA1 and TA2.

[0105] In this embodiment, the first distance to the subject in the first and second images is calculated based on a first image based on light incident on the image sensor 3 after passing through the first light transmission region TA1 and lens 2b by driving the liquid crystal layer LC, and a second image based on light incident on the image sensor 3 after passing through the second light transmission region TA2 and lens 2b by driving the liquid crystal layer LC. In this embodiment, the second distance to the subject in the third and fourth images is calculated based on a third image based on light incident on the image sensor 3 after passing through the third light transmission region TA3 and lens 2b by driving the liquid crystal layer LC, and a fourth image based on light incident on the image sensor 3 after passing through the fourth light transmission region TA4 and lens 2b by driving the liquid crystal layer LC.

[0106] In this embodiment, with the above configuration, for example, if the subject is located relatively far from the camera 1 (electronic device 100), the first distance can be used, and if the subject is located relatively close to the camera 1 (electronic device 100), the second distance can be used. Therefore, as explained in Figure 6 above, it is possible to avoid a decrease in the accuracy of the distance of the subject calculated when the subject is at close range (i.e., to improve the accuracy of the distance calculated from the image).

[0107] While it is conceivable to use an image based on light incident through a grid pattern, for example, to improve the accuracy of the distance of the subject calculated from the image, this embodiment reduces the effort required to prepare and arrange a grid pattern compared to such a configuration.

[0108] In this embodiment, as shown in Figure 9, an example has been described in which the first light transmission region TA1 and the second light transmission region TA2 are arranged side by side in direction X, and the third light transmission region TA3 and the fourth light transmission region TA4 are arranged side by side in direction Y. However, the incident light control region PCA may also be configured such that the first light transmission region TA1 and the second light transmission region TA2 are arranged side by side in direction Y, and the third light transmission region TA3 and the fourth light transmission region TA4 are arranged side by side in direction X.

[0109] Furthermore, if the distance calculated based on, for example, a first image based on incident light transmitted through a first light-transmitting region TA1 and a second image based on incident light transmitted through a second light-transmitting region TA2 is expressed using, for example, three-dimensional coordinates, then, as shown in Figure 9, if the first light-transmitting region TA1 and the second light-transmitting region TA2 are arranged side by side in direction X, then an error in direction Y (i.e., the Y coordinate) may occur when calculating the distance to the subject. Similarly, if the distance calculated based on, for example, a third image based on incident light transmitted through a third light-transmitting region TA3 and a fourth image based on incident light transmitted through a fourth light-transmitting region TA4 is expressed using, for example, three-dimensional coordinates, then, as shown in Figure 9, if the third light-transmitting region TA3 and the fourth light-transmitting region TA4 are arranged side by side in direction Y, then an error in direction X (i.e., the X coordinate) may occur when calculating the distance to the subject.

[0110] Therefore, in the incident light control region PCA of this embodiment, the first to fourth light transmission regions TA1 to TA4 may be arranged as shown in Figure 11. In the example shown in Figure 11, the first light transmission region TA1 and the second light transmission region TA2 are arranged side by side in an oblique direction with respect to direction X (or direction Y). On the other hand, the third light transmission region TA1 and the fourth light transmission region TA4 are arranged side by side in a direction perpendicular to the direction in which the first light transmission region TA1 and the second light transmission region TA2 are arranged side by side. In other words, the incident light control region PCA shown in Figure 11 corresponds to the incident light control region PCA shown in Figure 9 rotated 22.5 degrees to the right. With such a configuration, it may be possible to mitigate the above-mentioned errors in direction X or direction Y.

[0111] Furthermore, in this embodiment, it is sufficient to have a configuration in which a first coded aperture pair consisting of a first light transmission region TA1 and a second light transmission region TA2 and a second coded aperture pair consisting of a third light transmission region TA3 and a fourth light transmission region TA4 are provided. For this reason, the incident light control region PCA in this embodiment may be configured as shown in Figure 12, for example. In the incident light control region PCA shown in Figure 12, the first to fourth light transmission regions TA1 to TA4 are configured such that the center of the first light transmission region TA1 coincides with the center of the third light transmission region TA3, and the center of the second light transmission region TA2 coincides with the center of the fourth light transmission region TA4. In this embodiment, even if the incident light control region PCA (first to fourth light transmission regions TA1 to TA4) is configured as shown in Figure 12, it is sufficient to perform the same processing as shown in Figure 10.

[0112] In the configuration shown in Figure 12, the centers of the first light transmission region TA1 and the third light transmission region TA3 coincide, and the centers of the second light transmission region TA2 and the fourth light transmission region TA4 coincide. However, these central axes may be offset.

[0113] In Figure 12, the first light-transmitting region TA1 (third light-transmitting region TA3) and the second light-transmitting region TA2 (fourth light-transmitting region TA4) are arranged side by side in direction X. However, the first light-transmitting region TA1 (third light-transmitting region TA3) and the second light-transmitting region TA2 (fourth light-transmitting region TA4) may also be arranged side by side in direction Y, or diagonally with respect to directions X and Y.

[0114] Furthermore, although this embodiment has been described as providing two coding aperture pairs with different light transmission region sizes, there may be three or more coding aperture pairs with different light transmission region sizes.

[0115] Furthermore, although this embodiment has described the shapes of the first to fourth light transmission regions TA1 to TA4 of the incident light control region PCA as being circular, the shapes of the first to fourth light transmission regions TA1 to TA4 may be other than circular (for example, rectangular).

[0116] Furthermore, in this embodiment, in order to calculate the distance of the subject in the electronic device 100, it has been explained that a first image based on incident light transmitted through the first light-transmitting region TA1, a second image based on incident light transmitted through the second light-transmitting region TA2, a third image based on incident light transmitted through the third light-transmitting region TA3, and a fourth image based on incident light transmitted through the fourth light-transmitting region TA4 are output. However, for example, if the distance from the camera 1 (electronic device 100) to the subject is greater than or equal to a predetermined value (i.e., the subject is located at a medium or long distance), the liquid crystal layer LC may be driven to transmit light through the first light-transmitting region TA1 and the second light-transmitting region TA2 (i.e., the first and second images are output). On the other hand, if the distance from the camera 1 (electronic device 100) to the subject is less than a predetermined value (i.e., the subject is located at a short distance), the liquid crystal layer LC may be driven to transmit light through the third light-transmitting region TA3 and the fourth light-transmitting region TA4 (i.e., the third and fourth images are output).

[0117] Furthermore, as described above, the distance to the subject for determining (selecting) the light-transmitting region does not need to be an exact value; it may be measured using a predetermined sensor, or it may be specified by the user of the electronic device 100.

[0118] Furthermore, in this embodiment, an image based on incident light transmitted through one of the first to fourth light transmission regions TA1 to TA4 is used to calculate the distance to the subject. However, for example, the distance to the subject may be calculated using an image based on incident light transmitted through two (or three) of the first to fourth light transmission regions TA1 to TA4.

[0119] Furthermore, although this embodiment has been described as calculating the distance to the subject using the first to fourth images output from the camera module CM, the first to fourth images may also be used, for example, to generate a fully focused image of the subject.

[0120] All camera modules that a person skilled in the art can implement by appropriately modifying the design based on the camera modules described above as embodiments of the present invention also fall within the scope of the present invention, insofar as they encompass the gist of the present invention.

[0121] Within the scope of the spirit of the present invention, a person skilled in the art can conceive of various modifications, and such modifications are also understood to fall within the scope of the present invention. For example, modifications made by a person skilled in the art to the above-described embodiments, such as adding, deleting, or changing the design of components, or adding, omitting, or changing the conditions of processes, are also included within the scope of the present invention, as long as they retain the gist of the present invention.

[0122] Furthermore, any other effects and benefits brought about by the embodiments described above that are obvious from the description herein or that can be appropriately conceived by those skilled in the art are naturally considered to be brought about by the present invention. [Explanation of symbols]

[0123] 100...Electronic device, 1...Camera, 2...Optical system, 2b...Lens, 3...Image sensor, DSP...Liquid crystal display device, PNL...Liquid crystal panel, LC...Liquid crystal layer, PCA...Incident light control area, TA, TA1~TA4...Light transmission area, LSA...Light shielding area.

Claims

1. Image sensor and A liquid crystal panel comprising: an incident light control region having first to fourth light-transmitting regions positioned to cause light to be incident on the image sensor; a liquid crystal layer positioned to overlap with the incident light control region; and a driver that drives the liquid crystal layer to transmit light through each of the first to fourth light-transmitting regions. A lens located between the image sensor and the liquid crystal panel It is equipped with, The sizes of the third and fourth light-transmitting regions are smaller than the sizes of the first and second light-transmitting regions. Based on the first image, which is formed by driving the liquid crystal layer and passing through the first light-transmitting region and the lens and incident on the image sensor, and the second image, which is formed by driving the liquid crystal layer and passing through the second light-transmitting region and the lens and incident on the image sensor, the first distance to the subject in the first and second images is calculated. Based on the third image, which is formed by the driving of the liquid crystal layer and the light that passes through the third light-transmitting region and the lens and is incident on the image sensor, and the fourth image, which is formed by the driving of the liquid crystal layer and the light that passes through the fourth light-transmitting region and the lens and is incident on the image sensor, the second distance to the subject in the third and fourth images is calculated. The driver drives the liquid crystal layer to transmit light through the first and second light-transmitting regions when the distance to the subject is greater than or equal to a predetermined value, and drives the liquid crystal layer to transmit light through the third and fourth light-transmitting regions when the distance to the subject is less than the predetermined value. Camera module.

2. The first and second light-transmitting regions are arranged side by side in the first direction. The third and fourth light-transmitting regions are arranged side by side in a second direction different from the first direction. The camera module according to claim 1.

3. The first and second light-transmitting regions are arranged side by side in the first direction. The third and fourth light-transmitting regions are arranged side by side in the first direction, The first and third light-transmitting regions are arranged such that the center of the first light-transmitting region coincides with the center of the third light-transmitting region. The second and fourth light-transmitting regions are arranged such that the center of the second light-transmitting region coincides with the center of the fourth light-transmitting region. The camera module according to claim 1.

4. The camera module according to claim 1, wherein the first to fourth light-transmitting regions are circular in shape.

Images