Camera module

Abstract

A camera module according to an embodiment of the present invention comprises: an imaging element; and a liquid crystal panel. The liquid crystal panel includes an opening disposed at a position for causing light to enter the imaging element, a liquid crystal layer disposed at a position overlapping the opening, an electrode disposed at a position overlapping the liquid crystal layer, and a driver that drives the liquid crystal layer by applying a voltage to the electrode. The driver drives the liquid crystal layer on the basis of a first control value, and drives the liquid crystal layer on the basis of a second control value. A first image captured by driving the liquid crystal layer on the basis of the first control value is used for calculating the distance to an object having a first luminance. A second image captured by driving the liquid crystal layer on the basis of the second control value is used for calculating the distance to an object having a second luminance.

Description

Technical Field

[0006] , , , , ,

[0007] , , , ,

[0001] Embodiments of the present invention relate to a camera module.

Background Art

[0002] In recent years, camera modules have been developed that include a liquid crystal panel and an image pickup device (camera) provided on the back of the liquid crystal panel.

[0003] By the way, a camera module can capture an image when light enters the image pickup device provided in the camera module. However, an encoded aperture technique is known for calculating the distance to a subject in the image by using the blur that occurs in the image captured by the camera module.

[0004] However, depending on the image captured by the camera module, it may not be possible to calculate an appropriate distance from the image.

Prior Art Documents

[0008] [Figure 1] Figure 1 is an exploded perspective view showing an example of the configuration of a camera module according to an embodiment. [Figure 2] Figure 2 is a schematic plan view showing an example of a camera module. [Figure 3] Figure 3 is a diagram illustrating the principle of calculating the distance to a subject using a camera module. [Figure 4] Figure 4 is a schematic diagram showing a cross-section of the camera module along line AA' shown in Figure 2. [Figure 5] Figure 5 is a cross-sectional view showing the light-transmitting region included in the liquid crystal panel provided in the camera module. [Figure 6] Figure 6 shows another example of a light-transmitting region formed in an aperture. [Figure 7] FIG. 7 is a diagram showing another example of the light transmission region formed in the opening. [Figure 8] FIG. 8 is a diagram showing yet another example of the light transmission region formed in the opening. [Figure 9] FIG. 9 is a diagram showing yet another example of the light transmission region formed in the opening. [Figure 10] FIG. 10 is a diagram for explaining the operation of the camera module according to the comparative example of the present embodiment. [Figure 11] FIG. 11 is a diagram showing an example of an image transferred from the image sensor. [Figure 12] FIG. 12 is a diagram for explaining the first operation example of the camera module according to the present embodiment. [Figure 13] FIG. 13 is a diagram for explaining the second operation example of the camera module according to the present embodiment. [Figure 14] FIG. 14 is a diagram for explaining the third operation example of the camera module according to the present embodiment. [Figure 15] FIG. 15 is a diagram for explaining the distance map created when medium luminance images and high luminance images are captured. [Figure 16] FIG. 16 is a diagram showing another example of an image transferred from the image sensor. [Figure 17] FIG. 17 is a diagram for explaining the fourth operation example of the camera module according to the present embodiment. [Figure 18] FIG. 18 is a diagram showing an example of the circuit of the image sensor. [Figure 19] FIG. 19 is a diagram for explaining the fifth operation example of the camera module according to the present embodiment. [Figure 20] FIG. 20 is a diagram for explaining the distance map created when medium luminance images and low luminance images are captured. [Figure 21] FIG. 21 is a diagram for explaining the distance map created when medium luminance images, high luminance images, and low luminance images are captured.

Best Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described with reference to the drawings. Note that the disclosure is merely an example, and for those that can be easily conceived by those skilled in the art with appropriate modifications while maintaining the gist of the invention, they are naturally included in the scope of the present invention. In addition, for the purpose of making the description clearer, the drawings may schematically show the width, thickness, shape, etc. of each part compared to the embodiments, but this is merely an example and does not limit the interpretation of the present invention. Also, in this specification and each drawing, the same reference numerals may be assigned to the same elements as those described above with respect to the previously shown drawings, and detailed descriptions may be omitted as appropriate.

[0010] FIG. 1 is an exploded perspective view showing an example of the configuration of a camera module according to this embodiment. FIG. 1 shows a three-dimensional space defined by a direction X, a direction Y perpendicular to the direction X, and a direction Z perpendicular to both the direction X and the direction Y. Note that the directions X, Y, and Z are orthogonal to each other, but they may intersect at an angle other than 90°. Also, in this embodiment, the direction Z is defined as up, and the direction opposite to the direction Z is defined as down. When referring to "the second member above the first member" and "the second member below the first member", the second member may be in contact with the first member or may be located away from the first member.

[0011] As shown in FIG. 1, the camera module CM includes a liquid crystal panel PNL covered with a cover glass CG as a cover member, and an imaging device IS provided on the lower side (back side) of the liquid crystal panel PNL.

[0012] The liquid crystal panel PNL comprises an array substrate SUB1 and a counter substrate SUB2. In a plan view from direction Z, when viewing the camera module CM, the array substrate SUB1 has a keyhole shape (contour) that combines a substantially circular first portion 1a and a substantially rectangular second portion 1b connected to the first portion 1a. On the other hand, the counter substrate SUB2 has a shape that, when positioned to overlap with the first portion 1a of the array substrate SUB1, exposes the second portion 1b of the array substrate SUB1 in a plan view.

[0013] Although not shown in Figure 1, the liquid crystal panel PNL further comprises a liquid crystal layer held between the array substrate SUB1 and the opposing substrate SUB2.

[0014] The image sensor IS is a photoelectric conversion element that converts light incident on the image sensor IS into a voltage signal (electrical signal), and together with an optical system including at least one lens (not shown), it constitutes a camera that captures images.

[0015] In the camera module CM according to this embodiment, the liquid crystal layer provided in the liquid crystal panel PNL is driven, causing light transmitted through the cover glass CG and the liquid crystal panel PNL (liquid crystal layer) to be incident on the image sensor IS. As a result, the camera module CM can capture an image based on the light incident on the image sensor IS.

[0016] Figure 1 is a diagram illustrating the positional relationship in direction Z of the cover glass CG, liquid crystal panel PNL (array substrate SUB1 and opposing substrate SUB2), and image sensor IS (camera). The size and shape of the cover glass CG, liquid crystal panel PNL, and image sensor IS may differ from those shown in Figure 1.

[0017] Figure 2 is a schematic plan view of the camera module CM. In Figure 2, for convenience, only the cover glass CG and array substrate SUB1 are shown, but a counter substrate SUB2 is positioned between the cover glass CG and the array substrate SUB1. The image sensor IS is positioned on the back side of the array substrate SUB1 (the direction opposite to direction Z).

[0018] The liquid crystal panel PNL includes, for example, a circular aperture OP. In this embodiment, the aperture OP is a portion (region) that overlaps with the liquid crystal layer held between the array substrate SUB1 and the opposing substrate SUB2 described above.

[0019] In this embodiment, the aperture OP has multiple regions formed therein. These multiple regions formed in the aperture OP are regions through which light can be transmitted, for example, by driving the liquid crystal (hereinafter referred to as light-transmitting regions), and in the example shown in Figure 2, they include the first to third light-transmitting regions TA1 to TA3.

[0020] The first light-transmitting region TA1 has a circular shape and is formed, for example, at a position that does not include the center of the aperture OP. Specifically, the first light-transmitting region TA1 is formed at a position that is offset from the center of the aperture OP in the direction opposite to the X direction.

[0021] The second light-transmitting region TA2 has a circular shape and is formed, for example, at a position opposite the first light-transmitting region TA1 with the center of the aperture OP in between. That is, the second light-transmitting region TA2 is formed at a position shifted towards the X direction from the center of the aperture OP.

[0022] In the example shown in Figure 2, the first light-transmitting region TA1 and the second light-transmitting region TA2 are formed to be of roughly the same size.

[0023] The third light-transmitting region TA3 corresponds to the region obtained by excluding the first and second light-transmitting regions TA1 and TA2 from the aperture OP.

[0024] Furthermore, the first to third light-transmitting regions TA1 to TA3 are assumed to be separated by light-shielding regions formed by, for example, a black matrix.

[0025] Here, as described above, in order to cause light to enter the image sensor IS, it is necessary to drive the liquid crystal layer by applying a voltage to an electrode (hereinafter referred to as a driving electrode) located at a position corresponding to the liquid crystal layer. In this embodiment, the liquid crystal panel PNL is provided with a plurality of driving electrodes located at positions corresponding to each of a plurality of light-transmitting regions.

[0026] In the example shown in Figure 2, the liquid crystal panel PNL includes a first drive electrode positioned to overlap with the first light transmission region TA1, a second drive electrode positioned to overlap with the second light transmission region TA2, and a third drive electrode positioned to overlap with the third light transmission region TA3.

[0027] With this configuration, for example, if a voltage is applied only to the first drive electrode, the liquid crystal layer can be driven to transmit light to the image sensor IS via the first light-transmitting region TA1. Similarly, if a voltage is applied only to the second drive electrode, the liquid crystal layer can be driven to transmit light to the image sensor IS via the second light-transmitting region TA2. Likewise, if a voltage is applied only to the third drive electrode, the liquid crystal layer can be driven to transmit light to the image sensor IS via the third light-transmitting region TA3. Here, it is assumed that a normally black method, in which light is transmitted when a voltage is applied to the drive electrodes (i.e., in the ON state), is employed in the liquid crystal panel PNL.

[0028] In this embodiment, the camera module CM is used to calculate the distance from the camera module CM (image sensor IS) to the subject in the image (hereinafter referred to as the subject distance) by using an image based on light that has passed through each of the first to third light transmission regions TA1 to TA3 described above and been incident on the image sensor IS (i.e., an image of the subject captured by the camera module CM).

[0029] One technique for calculating the distance to a subject from an image is coded aperture technology. While a detailed explanation is omitted here, coded aperture technology calculates the distance to a subject by analyzing the blur that occurs in the image depending on the subject's position.

[0030] In other words, by utilizing the above-described encoded aperture technology, the camera module CM can be used for applications such as calculating the distance to an object based on an image and creating a distance map (depth map) representing the distance to that object. The processes of calculating the distance to the object and creating the distance map can be implemented, for example, by a predetermined application program running on an electronic device connected to the camera module CM (the electronic device on which the camera module CM is mounted).

[0031] Here, referring to Figure 3, we will briefly explain the principle of calculating the distance to the subject using the image captured by the camera module CM described above. Figure 3 shows the positional relationship between the camera module CM and the subject. Although omitted in Figure 1 above, in the camera module CM, a lens LNS is positioned between the image sensor IS and the liquid crystal panel PNL.

[0032] Here, we assume the case of calculating the distance of the subject S shown in Figure 3. Generally, in a camera, the subject S can be photographed in focus by changing the distance between the lens LNS and the image sensor IS. However, as shown in Figure 3, if the subject S is photographed when it is out of focus, a discrepancy occurs between the focal point and the position of the imaging surface of the image sensor IS, resulting in blurring of the image based on the light incident on the image sensor IS.

[0033] According to the encoding aperture technique described above, the distance to the subject S is calculated based on the blurring that occurs in the image.

[0034] Although Figure 3 shows the case where light passes through the first light transmission region TA1, in this embodiment, as described above, three light transmission regions (first to third light transmission regions TA1 to TA3) are prepared, and the accuracy of the distance to the subject can be improved by using multiple images based on the light that has passed through each of the three light transmission regions (i.e., multiple blur patterns based on the light that has passed through different light transmission regions).

[0035] In this embodiment, as described above, light can be transmitted to the image sensor IS through the first to third light transmission regions TA1 to TA3 by applying a voltage to each of the first to third drive electrodes. However, in order to apply a voltage to the first to third drive electrodes in this way, it is necessary to electrically connect the first to third drive electrodes to a driver (not shown) configured to drive the liquid crystal panel PNL (liquid crystal layer).

[0036] In this case, for example, the first drive electrode (i.e., the drive electrode positioned in a location overlapping with the first light transmission region TA1) is electrically connected to the first pad P1 via the first wiring W1, and the first pad P1 is electrically connected to the driver via a flexible printed circuit board (FPC).

[0037] Furthermore, the second drive electrode (i.e., the drive electrode positioned in a location overlapping with the second light transmission region TA2) is electrically connected to the second pad P2 via the second wiring W2, and the second pad P2 is electrically connected to the driver via a flexible printed circuit board (FPC).

[0038] Similarly, the third drive electrode (i.e., the drive electrode positioned to overlap with the third light transmission region TA3) is electrically connected to the third pad P3 via the third wiring W3, and the third pad P3 is electrically connected to the driver via a flexible printed circuit board (FPC).

[0039] For example, OLB (Outer Lead Bonding) pads can be used as the first to third pads P1 to P3 mentioned above.

[0040] Furthermore, the liquid crystal panel PNL includes a non-opening NOP surrounding the opening OP, and the first to third pads P1 to P3 are arranged in the non-opening NOP as shown in Figure 2. In the example shown in Figure 2, the first to third pads P1 to P3 extend in direction Y and are arranged side by side in direction X. In this case, the first to third wirings W1 to W3 described above are connected to the ends of the first to third pads P1 to P3 opposite to the Y direction.

[0041] Here, Figure 4 schematically shows a cross-section of the camera module CM along the line AA' shown in Figure 2. As shown in Figure 4, the liquid crystal panel PNL includes the array substrate SUB1, the opposing substrate SUB2, and the liquid crystal layer LC held between the array substrate SUB1 and the opposing substrate SUB2, as well as a drive board DB located on the back side (opposite direction Z) of the array substrate SUB1.

[0042] As shown in Figure 4, the driver DR that drives the liquid crystal panel PNL (liquid crystal layer LC) is mounted on the drive board DB. The flexible printed circuit board (FPC) described above extends along the first to third pads P1 to P3 (i.e., in the Y direction), and the first pad P1 shown in Figure 4 is connected to the driver DR via the flexible printed circuit board (FPC) which is bent at the Y-side end of the first pad P1. The first pad P1 and the flexible printed circuit board (FPC) can be electrically connected, for example, by crimping them together via an anisotropic conductive film (ACF).

[0043] Furthermore, the liquid crystal panel PNL includes a sealing material SE located at the non-aperture NOP, and the array substrate SUB1 and the opposing substrate SUB2 are joined by this sealing material SE. As a result, the liquid crystal layer LC can be formed in the space surrounded by the array substrate SUB1, the opposing substrate SUB2, and the sealing material SE.

[0044] Although not shown in Figure 4, the image sensor IS is located, for example, between the array substrate SUB1 and the drive board DB.

[0045] Below, with reference to Figure 5, a brief explanation of an example of the configuration of the liquid crystal panel PNL provided in the camera module CM is given. Here, the light-transmitting region (i.e., aperture OP) included in the liquid crystal panel PNL will be mainly explained.

[0046] As shown in Figure 5, the array substrate SUB1 includes insulating layers 11, 12, and 13 between the insulating substrate 10 and the alignment film AL1. A polarizing plate PL1 is also formed on the outside of the array substrate SUB1.

[0047] The insulating layer 11 is provided on the insulating substrate 10. The insulating layer 12 is also provided on top of the insulating layer 11.

[0048] In Figure 5, the first drive electrode E1 is provided on the insulating layer 12 and covered by the insulating layer 13. The first drive electrode E2 is provided on the insulating layer 13 and covered by the alignment film AL1. The alignment film AL1 is in contact with the liquid crystal layer LC.

[0049] The first drive electrodes E1 and E2 are formed from transparent conductive materials such as indium tin oxide (ITO) and indium zinc oxide (IZO). In the example shown in Figure 5, the insulating layer 13 is sandwiched between the first drive electrodes E1 and E2, but the first drive electrodes E1 and E2 may also be formed in the same layer.

[0050] On the other hand, the opposing substrate SUB2 is equipped with a light-shielding layer BM, a transparent layer OC, and an alignment film AL2 on the side of the insulating substrate 20 that faces the array substrate SUB1.

[0051] The light-shielding layer BM is formed on the inner surface of the insulating substrate 20 to form a light-shielding region that demarcates the first light-transmitting region TA1, etc. The transparent layer OC covers the insulating substrate 20 and the light-shielding layer BM. The alignment film AL2 covers the transparent layer OC and is in contact with the liquid crystal layer LC.

[0052] The liquid crystal layer (LC) is driven by applying a voltage between the first drive electrodes E1 and E2. In this case, for example, a first voltage is applied to the first drive electrode E1 and a second voltage is applied to the first drive electrode E2 via a first pad P1 and a flexible printed circuit board (FPC) located in a position overlapping with a non-aperture OP. One of the first and second voltages has a positive polarity voltage level, and the other has a negative polarity voltage level.

[0053] In this embodiment, for example, the liquid crystal layer LC is driven to transmit light to the image sensor IS via the first light transmission region TA1 by applying a voltage between the first drive electrodes E1 and E2. This driving of the liquid crystal layer LC is achieved by a driver DR.

[0054] Here, for example, the transmission axes of polarizers PL1 and PL2 are orthogonal, and the liquid crystal molecules contained in the liquid crystal layer LC are initially oriented in the direction of the transmission axis of polarizer PL1 between the alignment films AL1 and AL2.

[0055] In this case, when no voltage is applied between the first drive electrodes E1 and E2 (i.e., the liquid crystal layer LC is not driven), no phase difference occurs in the liquid crystal layer LC, and therefore the light transmittance in the first light-transmitting region TA1 is minimized (i.e., light cannot pass through the first light-transmitting region TA1).

[0056] On the other hand, in the ON state, when a voltage is applied between the first drive electrodes E1 and E2 (i.e., the liquid crystal layer LC is driven), the liquid crystal molecules align in a direction different from the initial orientation direction, and a phase difference occurs in the liquid crystal layer LC, so the light transmittance in the first light-transmitting region TA1 increases (i.e., light can pass through the first light-transmitting region TA1). The light that has passed through the first light-transmitting region TA1 in this way is incident on the image sensor IS, and the camera module CM can capture an image based on the light incident on the image sensor IS.

[0057] Here, we assume that a normally black method, which does not transmit light when off, is employed in the liquid crystal panel PNL. However, in this embodiment, a normally white method, which does not transmit light when on (and transmits light when off), may also be employed.

[0058] Although Figures 4 and 5 above mainly describe the first light-transmitting region TA1, the second and third light-transmitting regions TA2 and TA3 can be configured in the same way as the first light-transmitting region TA1, except that their position, size, and shape at the aperture OP differ.

[0059] In this embodiment, it has been described that a circular opening OP has three light-transmitting regions (first to third light-transmitting regions TA1 to TA3) formed within it. However, the shape of the opening OP, and the position, size, shape, and number of light-transmitting regions formed within it, may be appropriately changed depending on, for example, the subject whose distance is calculated as described above (i.e., the environment in which the image is captured).

[0060] Specifically, the aperture OP may have four light-transmitting regions TA1 to TA4, as shown in Figure 6. Furthermore, four light-transmitting regions TA1 to TA4 (i.e., two coded aperture pairs), as shown in Figure 7, may be formed in the aperture OP. For example, if the subject is located at a medium or long distance from the camera module CM, the distance to the subject may be calculated using the light-transmitting regions TA1 and TA2 (and the images based on the light transmitted through them). If the subject is located at a short distance from the camera module CM, the distance to the subject may be calculated using the light-transmitting regions TA3 and TA4 (and the images based on the light transmitted through them). Additionally, the light-transmitting regions TA1 to TA4 shown in Figure 7 may be arranged as shown in Figure 8 to mitigate errors in the direction X and direction Y of the calculated distance to the subject. Furthermore, the light-transmitting regions TA1 to TA4 shown in Figure 7 may be arranged as shown in Figure 9.

[0061] In this embodiment, it is assumed that multiple light-transmitting regions are formed (arranged) in the aperture OP. However, if it is possible to calculate the distance to the subject, a configuration in which only one light-transmitting region is formed in the aperture OP is also acceptable.

[0062] In this embodiment, the distance to the subject is calculated using an image captured by light passing through at least one light-transmitting region and entering the image sensor IS. However, depending on the brightness of the subject included in the image, for example, the accuracy of the distance to the subject may be low.

[0063] The operation of a camera module CM (driver DR and image sensor IS) according to a comparative example of this embodiment will be briefly described below with reference to Figure 10. Figure 10 shows the time during which the image sensor IS receives light that has passed through the liquid crystal layer (hereinafter referred to as exposure time), and the timing at which the image captured by light incident on the image sensor IS during the exposure time is transferred from the image sensor IS to the outside.

[0064] In this example, we assume that the camera module CM captures multiple images (i.e., a video) and uses each of these images to calculate the distance to the subject. Furthermore, in this embodiment, the image transmitted from the image sensor IS to the outside includes a voltage signal (luminance value) corresponding to the amount of light incident on the image sensor IS (i.e., the luminance of the subject).

[0065] Figure 10 shows that the driver DR drives the liquid crystal layer LC according to a predetermined frame rate (pre-defined exposure time), causing images based on light incident on the image sensor IS to be sequentially transferred (output) from the image sensor IS.

[0066] Here, Figure 11 shows an example of an image transferred from the image sensor IS when the driver DR drives the liquid crystal layer LC to achieve the exposure time shown in Figure 10. Here, we assume that there are medium-brightness and high-brightness subjects within the imaging range of the camera module CM. In this case, the image 100 shown in Figure 11 includes a medium-brightness region 100a and a high-brightness region 100b.

[0067] The medium-luminance region 100a is a region that includes a subject with medium luminance. In this embodiment, the distance to the subject is calculated using the blur that occurs in the image, and since the blur is relatively easy to observe in the medium-luminance region 100a, a highly accurate distance can be calculated.

[0068] On the other hand, the high-luminance region 100b is a region that includes a high-luminance subject, but in such a high-luminance region 100b, the luminance saturates (overexposure occurs), making it impossible to observe blur and thus impossible to calculate the distance of the subject.

[0069] In other words, as shown in Figure 10 above, when the driver DR and image sensor IS (camera module CM) operate to repeatedly capture images with a constant exposure time, it may not be possible to properly calculate the distance to the subject (especially a high-brightness subject) from the image (i.e., distance measurement may not be possible).

[0070] Therefore, in this embodiment, the driver DR operates to drive the liquid crystal layer LC based on a control value (first control value) for capturing a medium-brightness (first brightness) subject within the imaging range, and also drives the liquid crystal layer LC based on a control value (second control value) for capturing a high-brightness (second brightness) subject within the imaging range, in order to capture an image that can calculate the appropriate distance to the subject.

[0071] Hereinafter, a first example of operation of the camera module CM according to this embodiment will be described with reference to Figure 12. Figure 12, similar to Figure 10 described above, shows the exposure time of the image sensor IS and the timing at which the image captured by light incident on the image sensor IS during the exposure time is transferred from the image sensor IS to the outside.

[0072] In the first example of operation, the driver DR drives the liquid crystal layer LC based on the exposure time (control value) for medium brightness. In this case, an image captured based on the light incident on the image sensor IS within the exposure time for medium brightness (hereinafter referred to as the medium brightness image) is transferred from the image sensor IS.

[0073] Next, the driver DR drives the liquid crystal layer LC based on the exposure time (control value) for high brightness. In this case, the image captured based on the light incident on the image sensor IS within the high brightness exposure time (hereinafter referred to as the high brightness image) is transferred from the image sensor IS.

[0074] In the first example of operation, when capturing video, as shown in Figure 12, images for medium brightness and images for high brightness are captured alternately (that is, the driving of the liquid crystal layer LC based on the exposure time for medium brightness and the driving of the liquid crystal layer LC based on the exposure time for high brightness are repeatedly performed).

[0075] Here, the exposure time for medium brightness in the first example of operation is the same as the exposure time shown in Figure 10 above, but the exposure time for high brightness is set shorter than the exposure time for medium brightness. According to this, in the high brightness image, by shortening the exposure time, the brightness of subjects whose brightness is saturated in the medium brightness image and whose distance cannot be calculated can be suppressed. In other words, the medium brightness image is an image for accurately calculating the distance of a medium brightness subject, while the high brightness image is an image for accurately calculating the distance of a high brightness subject.

[0076] According to the first example of operation described above, by adjusting the exposure time, it becomes possible to capture a medium-brightness image (first image) that can calculate the distance of a medium-brightness subject, and a high-brightness image (second image) that can calculate the brightness of a high-brightness subject.

[0077] In the first example of operation, it is assumed that the driver DR drives the liquid crystal layer LC in accordance with the exposure time of the image sensor IS. However, this exposure time may be adjusted by the liquid crystal panel PNL, or by controlling a physical shutter, for example.

[0078] In the first example of operation, the amount of light received by the image sensor IS (i.e., the brightness of the subject in the image) was explained as being adjusted by the image sensor IS providing exposure times for medium brightness and exposure times for high brightness. In a liquid crystal panel PNL, the transmittance of light in the liquid crystal layer LC (i.e., the amount of light incident on the image sensor IS) can be adjusted by the voltage value applied to the drive electrode located in a position superimposed on the liquid crystal layer LC.

[0079] The second example of operation of the camera module CM according to this embodiment will be described below with reference to Figure 13. Figure 13 shows the time (exposure time) during which the driver DR drives the liquid crystal layer LC to cause light to enter the image sensor IS, the voltage value applied to the drive electrode located in a position overlapping with the liquid crystal layer LC to drive the liquid crystal layer LC, and the timing at which the image captured by light entering the image sensor IS within the exposure time is transferred from the image sensor IS to the outside. In other words, in the second example of operation, the exposure time is controlled by the liquid crystal panel PNL. Here, it is assumed that the liquid crystal panel PNL employs a normally black method.

[0080] In the second example of operation, the driver DR drives the liquid crystal layer LC by applying a voltage to the drive electrode based on a voltage value (control value) for medium brightness. In this case, while the liquid crystal layer LC is being driven based on the voltage value for medium brightness, the medium brightness image captured based on the light incident on the image sensor IS is transferred from the image sensor.

[0081] Next, the driver DR drives the liquid crystal layer LC by applying a voltage to the drive electrode based on a high-brightness voltage value (control value). In this case, while the liquid crystal layer LC is being driven based on the high-brightness voltage value, a high-brightness image captured based on the light incident on the image sensor IS is transferred from the image sensor.

[0082] In the second operation example, when capturing video, images for medium brightness and images for high brightness are captured alternately, similar to the first operation example described above (that is, the driving of the liquid crystal layer LC based on the voltage value for medium brightness and the driving of the liquid crystal layer LC based on the voltage value for high brightness are repeatedly performed).

[0083] Here, the voltage value for medium brightness in the second operating example is the same as the voltage value applied to the drive electrode in the first operating example, for example, but the voltage value for high brightness is set lower than the voltage value for medium brightness. In a liquid crystal panel PNL employing the normally black method described above, a high voltage can be applied to the drive electrode to achieve high light transmittance in the liquid crystal layer (LC). Therefore, in the high-brightness image, by lowering the voltage applied to the drive voltage, the brightness of subjects whose brightness saturates in the medium-brightness image and whose distance cannot be calculated can be suppressed. In other words, the high-brightness image in the second operating example can be said to be an image for accurately calculating the distance of a high-brightness subject.

[0084] Note that the image used for medium brightness in the second operation example is the same as the image used for medium brightness in the first operation example, and is therefore used to accurately calculate the distance of a medium brightness subject.

[0085] Furthermore, in the second example of operation, the brightness of the subject is adjusted by the voltage value applied to the drive electrode, so the exposure time (the time it takes to drive the liquid crystal layer LC to cause light to enter the image sensor IS) may be constant.

[0086] As described above, according to the second example of operation, by adjusting the voltage value applied to the drive electrode, it becomes possible to capture a medium-brightness image capable of calculating a medium-brightness subject and a high-brightness image capable of calculating a high-brightness subject.

[0087] Furthermore, as mentioned above, when the liquid crystal panel PNL employs a normally black method, the second operating example can reduce the applied voltage (the voltage value applied to the drive electrode) when capturing high-brightness images compared to the first operating example, thereby suppressing power consumption. Moreover, the second operating example is applicable even when brightness saturates despite shortening the exposure time.

[0088] Here, for example, in the first and second operation examples described above, if the camera module CM captures multiple images in succession (i.e., captures a video), it is necessary to capture both a medium-brightness image and a high-brightness image as one frame, which reduces the frame rate. For this reason, as a modification of the first operation example described above, a configuration may be adopted in which a medium-brightness image and a high-brightness image are captured using a fixed exposure time (i.e., the high-brightness image is captured during the exposure time used to capture the medium-brightness image).

[0089] A third example of operation of the camera module CM according to this embodiment will be described below with reference to Figure 14. Figure 14 shows the time (exposure time) during which the driver DR drives the liquid crystal layer LC to cause light to enter the image sensor IS, and the timing at which the image captured by the light entering the image sensor IS during the exposure time is transferred from the image sensor IS to the outside.

[0090] In the third operating example, the driver DR drives the liquid crystal layer LC based on a constant exposure time of the image sensor IS. The exposure time in the third operating example is, for example, the same as the exposure time for medium brightness in the first operating example.

[0091] In this third operational example, the image sensor IS transfers a high-brightness image based on the amount of light incident on the image sensor IS before the exposure time during which the driver DR drives the liquid crystal layer LC ends.

[0092] Next, the image sensor IS transfers a medium-luminance image based on the amount of light that entered the image sensor IS by the time the exposure time ends.

[0093] According to this third example of operation, since both a medium-brightness image and a high-brightness image can be captured during a single exposure time for capturing a medium-brightness image in the first and second examples of operation described above, a higher frame rate can be achieved compared to the first and second examples of operation.

[0094] It should be noted that, although this explanation assumes the presence of a high-brightness subject within the imaging range of the camera module CM, capturing high-brightness images even when such a subject is not present will result in the aforementioned decrease in frame rate or an increase in the processing load of the camera module CM.

[0095] Therefore, the camera module CM according to this embodiment may be configured to switch its operating mode depending on whether or not a high-brightness subject is present within the imaging range of the camera module CM.

[0096] Specifically, the camera module CM may, for example, operate in a first operating mode that continuously captures medium-brightness images under normal circumstances, and then, if a high-brightness subject (or region containing one) is detected from the medium-brightness images by analyzing them, operate in a second operating mode that captures the aforementioned medium-brightness and high-brightness images. The process of analyzing the medium-brightness images may be performed on the electronic device on which the camera module CM is mounted, or it may be performed by a processing circuit or the like mounted on the camera module CM.

[0097] Here, the camera module CM according to this embodiment has been described as being used for purposes such as creating a distance map representing the distance to a subject. Referring to Figure 15, the distance maps created when the above-mentioned medium-brightness image and high-brightness image are captured will be briefly explained. The process of creating the distance map is performed by the electronic device on which the camera module CM is mounted, as described above.

[0098] First, the upper part of Figure 15 schematically shows a medium-luminance image 201 and a distance map 301 that includes the distance of the subject calculated from the medium-luminance image 201.

[0099] The medium-luminance image 201 includes a medium-luminance region (a region containing a medium-luminance subject) 201a and a high-luminance region (a region containing a high-luminance subject) 201b. When the distance of a subject included in such a medium-luminance image 201 is calculated, the distance of a subject included in the medium-luminance region 201a can be calculated accurately, but the distance of a subject included in the high-luminance region 201b cannot be calculated. In this case, for example, a distance map 301 is created in which the distances of subjects included in the medium-luminance region 201a are assigned to the medium-luminance region 201a in the medium-luminance image 201.

[0100] On the other hand, the lower part of Figure 15 schematically shows a high-brightness image 202 and a distance map 302 that includes the distance of the subject calculated from the high-brightness image 202.

[0101] As described above, the brightness of the subject is suppressed in the high-brightness image 202, and therefore the high-brightness image 202 includes a low-brightness region (a region containing a low-brightness subject) 202a and a medium-brightness region (a region containing a medium-brightness subject) 202b. The low-brightness region 202a corresponds to the region in which the brightness of the subject is suppressed, which is included in the medium-brightness region 201a included in the medium-brightness image 201. The medium-brightness region 202b corresponds to the region in which the brightness of the subject is suppressed, which is included in the high-brightness region 201b included in the medium-brightness image 201. When the distance of the subject included in the high-brightness image 202 is calculated using such a high-brightness image 202, the distance of the subject included in the medium-brightness region 202b can be calculated with high accuracy. For example, a distance map 302 is created in which the distance of the subject included in the medium-brightness region 202b is assigned to the medium-brightness region 202b in the high-brightness image 202.

[0102] In this embodiment, as described above, by combining the distance map 301 created from the medium-brightness image 201 and the distance map 302 created from the high-brightness image 202, a distance map 303 can be created in which the distance of the subject is assigned with high accuracy in all areas.

[0103] In this embodiment, we have described the case where a high-luminance subject is present within the imaging range of the camera module CM (i.e., the distance of the high-luminance subject cannot be calculated). However, similarly, it is difficult to calculate the distance of a low-luminance subject when it is present within the imaging range.

[0104] Here, Figure 16 shows an example of an image transferred from the image sensor IS when, for example, the driver DR drives the liquid crystal layer LC to achieve the exposure time shown in Figure 10 above. Here, it is assumed that there are medium-luminance and low-luminance subjects within the imaging range of the camera module CM. In this case, the image 400 shown in Figure 16 includes a medium-luminance region 400a and a low-luminance region 400b.

[0105] The medium-luminance region 400a is a region that includes subjects with medium luminance. As described above, if the subject is in the medium-luminance region 400a, the distance to the subject can be calculated with high accuracy.

[0106] On the other hand, the low-luminance region 400b is a region that includes low-luminance subjects, but in such a low-luminance region 400b, the signal-to-noise ratio in the image sensor IS is low, making it difficult to calculate the distance of the subject with high accuracy.

[0107] The fourth example of operation of the camera module CM according to this embodiment will be described below with reference to Figure 17. Figure 17 shows the time (exposure time) during which the driver DR drives the liquid crystal layer LC to cause light to enter the image sensor IS, and the timing at which the image captured by the light entering the image sensor IS during the exposure time is transferred from the image sensor IS to the outside.

[0108] In the fourth operation example, the driver DR drives the liquid crystal layer LC based on the exposure time (control value) for medium brightness. In this case, the image captured based on the light incident on the image sensor IS within the exposure time for medium brightness (medium brightness image) is transferred from the image sensor IS.

[0109] Next, the driver DR drives the liquid crystal layer LC based on the exposure time (control value) for low brightness. In this case, an image captured based on the light incident on the image sensor IS within the low brightness exposure time (hereinafter referred to as the low brightness image) is transferred from the image sensor IS.

[0110] In the fourth operation example, when capturing video, as shown in Figure 17, images for medium brightness and images for low brightness are captured alternately (that is, the driving of the liquid crystal layer LC based on the exposure time for medium brightness and the driving of the liquid crystal layer LC based on the exposure time for low brightness are repeatedly performed).

[0111] Here, the exposure time for medium brightness in the fourth example is the same as the exposure time shown in Figure 10 above, but the exposure time for low brightness is set to be longer than the exposure time for medium brightness. According to this, in the low brightness image, by extending the exposure time, the brightness of subjects whose brightness is low in the medium brightness image, making it difficult to calculate the distance, can be improved. In other words, the low brightness image in the fourth example can be said to be an image for accurately calculating the distance of low-brightness subjects.

[0112] According to the fourth example of operation described above, by adjusting the exposure time, it becomes possible to capture images for medium brightness, which allows for the calculation of the distance of medium brightness subjects, and images for low brightness, which allows for the calculation of the distance of low brightness subjects.

[0113] In the fourth example of operation, it is assumed that the exposure time is adjusted by the driver DR driving the liquid crystal layer LC, but this exposure time may also be adjusted by the image sensor IS, or by controlling a physical shutter, for example.

[0114] In the fourth operation example, it is necessary to capture both a medium-brightness image and a low-brightness image as a single frame, which reduces the frame rate. Therefore, for the fourth operation example, the third operation example, which is used when a high-brightness subject exists within the imaging range as described above, may be applied to capture the medium-brightness image during the exposure time for capturing the low-brightness image.

[0115] In the second example of operation described above, a high-brightness image was captured by adjusting the voltage value applied to the drive electrode. However, when a medium-brightness image is captured, the voltage value applied to the drive electrode (i.e., the light transmittance in the liquid crystal layer LC) is generally close to the maximum value, and it is not possible to capture a low-brightness image by adjusting this voltage value to improve the brightness of the subject.

[0116] Therefore, for example, a configuration may be used in which a low-light image is captured by using a gain (analog gain) that adjusts the voltage signal corresponding to the amount of light incident on the image sensor IS.

[0117] Figure 18 shows an example of a circuit diagram of an image sensor IS. The image sensor IS comprises a vertical scanning circuit VSR, a horizontal scanning circuit HSR, and a light-receiving unit, the light-receiving unit having multiple pixel cells corresponding to multiple pixels that constitute the image captured by the camera module CM.

[0118] One of these multiple pixel cells is composed of, for example, a photodiode D and a series circuit of a switch MOSFET Q1 whose gate is connected to the vertical scan line V and a switch MOSFET Q2 whose gate is connected to the horizontal scan line H.

[0119] Furthermore, the output nodes of other similar pixel cells arranged in the same row (horizontally) as the pixel cell composed of the photodiode D, switch MOSFETs Q1 and Q2 described above are connected to the horizontal signal line HS extending horizontally in Figure 18. Similar pixel cells are formed in the other rows as well.

[0120] Furthermore, the vertical scan line V is arranged parallel to the horizontal signal line HS. Each of the switch MOSFETs of the multiple pixel cells located in the same row corresponding to that vertical scan line V is connected to this vertical scan line V. The same applies to the other vertical scan lines.

[0121] In Figure 18, the horizontal scan line H extends vertically. Each switch MOSFET of multiple pixel cells located in the same column corresponding to the horizontal scan line H is connected to the horizontal scan line H. The same applies to the other horizontal scan lines.

[0122] Furthermore, the vertical scan line V is connected to the gate of a switch MOSFET Q3 that connects the horizontal signal line HS and the vertically extending output line VS. A readout load resistor R is also provided between this output line VS and the bias voltage VB. With this configuration, a current corresponding to the amount of light (optical signal) stored in the photodiode of the pixel cell flows, simultaneously performing a readout operation from the pixel cell and a reset (precharge) operation for the next readout operation. The voltage signal obtained by the load resistor R (a voltage signal corresponding to the amount of light) is amplified by a sense amplifier SA and transmitted to an output circuit (not shown).

[0123] While not particularly limited, a MOSFET Q4 is provided on the horizontal signal line HS to effectively eliminate spurious signals such as smearing and blooming. Specifically, a bias voltage VB is applied to the drain of MOSFET Q4, and a bias voltage VB' is applied to the gate of MOSFET Q4. By setting both bias voltages VB and VB' to be equal, a common potential is given to the gate and drain of MOSFET Q4, thus creating a diode configuration.

[0124] The MOSFETQ4 described above is formed with a conductance that is sufficiently smaller than that of the switch MOSFETQ3 described above. In other words, the on-resistance of MOSFETQ4 is set to a resistance that is sufficiently larger than that of switch MOSFETQ3. For example, when a vertical scan line V is in a high-level state, one switch MOSFETQ3 and each switch MOSFET (e.g., switch MOSFETQ1, etc.) of the pixel cells arranged in that row are turned on accordingly. In this case, when a horizontal scan line H is in a high-level state, each switch MOSFET (e.g., switch MOSFETQ2, etc.) of the pixel cells in the column corresponding to that horizontal scan line H are turned on, and a readout operation is performed on one pixel cell located at the intersection of the matrix.

[0125] In Figure 18, it was explained that the voltage signal corresponding to the amount of light stored in the photodiode (i.e., the amount of light incident on the image sensor IS) is amplified by the sense amplifier SA, but this amount of amplification is adjusted by the analog gain.

[0126] The fifth example of operation of the camera module CM according to this embodiment will be described below with reference to Figure 19. Figure 19 shows the analog gain described above, along with the time (exposure time) during which the driver DR drives the liquid crystal layer LC to cause light to enter the image sensor IS, and the timing at which the image captured by light entering the image sensor IS during the exposure time is transferred from the image sensor IS to the outside.

[0127] In the fifth operating example, the driver DR drives the liquid crystal layer LC based on a constant exposure time. The exposure time in the fifth operating example is, for example, the same as the exposure time for medium brightness in the fourth operating example.

[0128] In the fifth example of operation, the image sensor IS adjusts (amplifies) a voltage signal corresponding to the amount of light incident on the image sensor IS during the first exposure time, for example, using an analog gain for medium brightness. The image sensor IS then transmits an image (medium brightness image) based on this adjusted voltage value.

[0129] Next, the image sensor (IS) adjusts (amplifies) the voltage signal corresponding to the amount of light incident on the image sensor (IS) during the next second exposure time using an analog gain for low brightness. The image sensor (IS) then transmits an image (low brightness image) based on this adjusted voltage value.

[0130] Here, while the analog gain is generally constant, the analog gain for medium brightness in the fifth operating example is set to approximately the same level as this constant analog gain. On the other hand, the analog gain for low brightness (second gain) in the fifth operating example is set higher than the analog gain for medium brightness (first gain). According to this, by amplifying with a higher analog gain, it is possible to improve the brightness of subjects whose distance is difficult to calculate due to their low brightness in the medium brightness image. In other words, the low brightness image in the fifth operating example can be said to be an image for accurately calculating the distance of low-brightness subjects, similar to the low brightness image in the fourth operating example described above.

[0131] According to this fifth example of operation, by varying the analog gain, it becomes possible to capture images for medium brightness (first brightness) subjects and images for low brightness (second brightness) subjects, which are capable of being calculated.

[0132] It should be noted that, although this explanation assumes the presence of a low-luminance subject within the imaging range of the camera module CM, capturing low-luminance images even when such a subject is not present will result in the aforementioned decrease in frame rate or an increase in the processing load of the camera module CM.

[0133] Therefore, the camera module CM according to this embodiment may be configured to switch its operating mode depending on whether or not there is a low-luminance subject within the imaging range of the camera module CM.

[0134] Specifically, the camera module CM may, for example, operate in a first operating mode that continuously captures medium-luminance images under normal circumstances, and then, if a low-luminance subject (or region containing one) is detected from the medium-luminance images by analyzing them, operate in a second operating mode that captures the aforementioned medium-luminance and low-luminance images. The process of analyzing the medium-luminance images may be performed on the electronic device on which the camera module CM is mounted, or it may be performed by a processing circuit or the like mounted on the camera module CM.

[0135] Next, with reference to Figure 20, we will briefly explain the distance maps created when the medium-luminance and low-luminance images described above are captured.

[0136] First, the upper part of Figure 20 schematically shows a medium-luminance image 501 and a distance map 601 that includes the distance of the subject calculated from the medium-luminance image 501.

[0137] The medium-luminance image 501 includes a medium-luminance region (a region containing a medium-luminance subject) 501a and a low-luminance region 501b (a region containing a low-luminance subject). When calculating the distance of subjects included in such a medium-luminance image 501, the distance of subjects included in the medium-luminance region 501a can be calculated accurately, but it is difficult to calculate the distance of subjects included in the low-luminance region 501b. In this case, for example, a distance map 601 is created in which the distances of subjects included in the medium-luminance region 501a are assigned to the medium-luminance region 501a in the medium-luminance image 501.

[0138] On the other hand, the lower part of Figure 20 schematically shows a low-luminance image 502 and a distance map 602 that includes the distance of the subject calculated from the low-luminance image 502.

[0139] As described above, the brightness of the subject is increased in the low-luminance image 502, so the low-luminance image 502 includes a high-luminance region (a region containing a high-luminance subject) 502a and a medium-luminance region (a region containing a medium-luminance subject) 502b. The high-luminance region 502a corresponds to the region in which the brightness of the subject included in the medium-luminance region 501a in the medium-luminance image 501 has been increased. The medium-luminance region 502b corresponds to the region in which the brightness of the subject included in the low-luminance region 501b, which is included in the medium-luminance region 502b, has been increased. When the distance of the subject included in the low-luminance image 502 is calculated using such a low-luminance image 502, the distance of the subject included in the medium-luminance region 502b can be calculated with high accuracy. In this case, for example, a distance map 602 is created in which the distance of the subject included in the medium-luminance region 502b is assigned to the medium-luminance region 502b in the low-luminance image 502.

[0140] In this embodiment, as described above, by combining the distance map 601 created from the medium-luminance image 501 and the distance map 602 created from the low-luminance image 502, a distance map 603 can be created in which the distance of the subject is assigned with high accuracy in all areas.

[0141] In this embodiment, the first to third operation examples for cases where a high-brightness subject is present within the imaging range, and the fourth and fifth operation examples for cases where a low-brightness subject is present within the imaging range have been described. However, in cases where both high-brightness and low-brightness subjects are present within the imaging range, the camera module CM may be configured to perform an operation that combines one of the first to third operation examples with one of the fourth and fifth operation examples. With such a configuration, it is possible to capture images for medium brightness to calculate the distance of a medium-brightness subject, images for high brightness to calculate the distance of a high-brightness subject, and images for low brightness to calculate the distance of a low-brightness subject.

[0142] Furthermore, although a detailed explanation will be omitted, as shown in Figure 21, when a medium-brightness image 701, a high-brightness image 702, and a low-brightness image 703 are captured by the camera module CM, a distance map 804 can be created by combining the distance map 801 created from the medium-brightness image 701, the distance map 802 created from the high-brightness image 702, and the distance map 803 created from the low-brightness image 703.

[0143] As described above, this embodiment provides a camera module capable of capturing images that allow for the calculation of an appropriate distance.

[0144] All camera modules that a person skilled in the art can implement by appropriately modifying the design based on the camera module described above as an embodiment of the present invention also fall within the scope of the present invention, insofar as they encompass the gist of the present invention.

[0145] Within the scope of the spirit of the present invention, a person skilled in the art can conceive of various modifications, and such modifications are also understood to fall within the scope of the present invention. For example, modifications made by a person skilled in the art to the above-described embodiments, such as adding, deleting, or changing the design of components, or adding, omitting, or changing the conditions of processes, are also included within the scope of the present invention, as long as they retain the gist of the present invention.

[0146] Furthermore, any other effects and benefits brought about by the embodiments described above that are obvious from the description herein or that can be appropriately conceived by those skilled in the art are naturally considered to be brought about by the present invention. [Explanation of Symbols]

[0147] CM...Camera module, CG...Cover glass, SUB1...Array substrate, SUB2...Opposite substrate, IS...Image sensor, OP...Aperture, TA1...First light transmission region, TA2...Second light transmission region, TA3...Third light transmission region, P1...First pad, P2...Second pad, P3...Third pad, FPC,FPC1~FPC3...Flexible wiring board, W1...First wiring, W2...Second wiring, W3...Third wiring, LNS...Lens, LC...Liquid crystal layer, DB...Driver board, DR...Driver, E1,E2...Driver electrodes.

Claims

1. Image sensor and LCD panel and It is equipped with, The aforementioned liquid crystal panel is An aperture positioned at a location where light is incident on the image sensor, A liquid crystal layer positioned to overlap with the aforementioned opening, An electrode positioned to overlap with the aforementioned liquid crystal layer, A driver that drives the liquid crystal layer by applying a voltage to the electrodes. Includes, The opening has a pair of first light-transmitting regions and a pair of second light-transmitting regions that are different from the pair of first light-transmitting regions. The aforementioned driver In the first light-transmitting region, the liquid crystal layer is driven based on a first control value for imaging a subject with a first brightness within the imaging range, and the liquid crystal layer is driven based on a second control value for imaging a subject with a second brightness different from the first brightness within the imaging range. In the second light-transmitting region, the liquid crystal layer is driven based on the first control value for imaging a subject with the first brightness within the imaging range, and the liquid crystal layer is driven based on the second control value for imaging a subject with the second brightness within the imaging range. The first image captured based on the amount of light incident on the image sensor by driving the liquid crystal layer based on the first control value in the first light transmission region is amplified by a sense amplifier and transferred to an output circuit in order to calculate the distance to a subject with a first brightness. The second image, captured based on the amount of light incident on the image sensor by driving the liquid crystal layer based on the second control value in the first light transmission region, is amplified by the sense amplifier and transferred to the output circuit in order to calculate the distance to the subject of the second brightness. The third image, captured based on the amount of light incident on the image sensor by driving the liquid crystal layer based on the first control value in the second light transmission region, is amplified by the sense amplifier and transferred to the output circuit in order to calculate the distance to the subject of the first brightness. The liquid crystal layer is driven based on the second control value in the second light-transmitting region, and the fourth image captured based on the amount of light incident on the image sensor is amplified by the sense amplifier and transferred to the output circuit in order to calculate the distance to the subject of the second brightness. Camera module.

2. The first and second control values ​​include the exposure time during which light is incident on the image sensor by driving the liquid crystal layer. If the second brightness is higher than the first brightness, the exposure time included in the second control value is shorter than the exposure time included in the first control value. The camera module according to claim 1.

3. The first and second control values ​​include the voltage values ​​of the voltage applied to the electrodes. If the second brightness is higher than the first brightness, the voltage value included in the second control value is lower than the voltage value included in the first control value. The camera module according to claim 1.

4. The camera module according to any one of claims 1 to 3, wherein the driver repeatedly performs the driving of the liquid crystal layer based on the first control value and the driving of the liquid crystal layer based on the second control value.

5. The camera module according to any one of claims 1 to 3, wherein the driver drives the liquid crystal layer based on the second control value when it is determined that the image contains a subject of the second brightness by analyzing the image captured based on the amount of light incident on the image sensor as the liquid crystal layer is driven based on the first control value.

6. Image sensor and LCD panel and It is equipped with, The aforementioned liquid crystal panel is An aperture positioned at a location where light is incident on the image sensor, A liquid crystal layer positioned to overlap with the aforementioned opening, An electrode positioned to overlap with the aforementioned liquid crystal layer, A driver that drives the liquid crystal layer by applying a voltage to the electrodes. Includes, The opening has a pair of first light-transmitting regions and a pair of second light-transmitting regions that are different from the pair of first light-transmitting regions. The driver drives the liquid crystal layer so that light is incident on the image sensor for a predetermined exposure time. The first image, based on the amount of light incident on the image sensor up to the time the exposure time in the first light-transmitting region ends, is amplified by a sense amplifier and transferred to an output circuit in order to calculate the distance to the subject with a first brightness included in the first image. The second image, based on the amount of light incident on the image sensor before the exposure time in the first light-transmitting region ends, is amplified by the sense amplifier and transferred to the output circuit in order to calculate the distance to a subject with a second brightness higher than the first brightness included in the second image. The third image, based on the amount of light incident on the image sensor up to the time the exposure time in the second light-transmitting region ends, is amplified by the sense amplifier and transferred to the output circuit in order to calculate the distance to the subject with the first brightness included in the third image. The fourth image, based on the amount of light incident on the image sensor before the exposure time in the second light-transmitting region ends, is amplified by the sense amplifier and transferred to the output circuit in order to calculate the distance to the subject of the second brightness included in the fourth image. Camera module.

7. Image sensor and LCD panel and It is equipped with, The aforementioned liquid crystal panel is An aperture positioned at a location where light is incident on the image sensor, A liquid crystal layer positioned to overlap with the aforementioned opening, An electrode positioned to overlap with the aforementioned liquid crystal layer, A driver that drives the liquid crystal layer by applying a voltage to the electrodes. Includes, The opening has a pair of first light-transmitting regions and a pair of second light-transmitting regions that are different from the pair of first light-transmitting regions. The driver drives the liquid crystal layer such that light is incident on the image sensor during a predetermined first exposure time, and light is incident on the image sensor during a second exposure time following the first exposure time. The image sensor adjusts a voltage signal corresponding to the amount of light incident on the image sensor during a first exposure time using a first gain, and adjusts a voltage signal corresponding to the amount of light incident on the image sensor during a second exposure time using a second gain that is higher than the first gain. The first image, based on a voltage signal adjusted using the first gain in the first light-transmitting region, is amplified by a sense amplifier and transmitted to an output circuit in order to calculate the distance to a subject with a first brightness contained in the first image. The second image, based on the voltage signal adjusted using the second gain in the first light transmission region, is amplified by the sense amplifier and transmitted to the output circuit in order to calculate the distance to a subject with a second brightness lower than the first brightness included in the second image. The third image, based on the voltage signal adjusted using the first gain in the second light-transmitting region, is amplified by the sense amplifier and transmitted to the output circuit in order to calculate the distance to the subject of the first brightness contained in the third image. The fourth image, based on the voltage signal adjusted using the second gain in the second light transmission region, is amplified by the sense amplifier and transmitted to the output circuit in order to calculate the distance to the subject of the second brightness contained in the fourth image. Camera module.

Images