Image pickup device

Abstract

To provide an imaging element in which a backside illumination type imaging chip for improving usability and a signal processing chip are laminated, and to provide an imaging apparatus including the same.SOLUTION: An imaging element 100 includes an imaging chip 113 provided with a pixel part having a first pixel including a first photoelectric conversion part and a second pixel including a second photoelectric conversion part, a signal processing chip 111 in which a first conversion part and a second conversion part are arranged, and a bump for connecting the imaging chip 113 and the signal processing chip 111. The bump 109 is arranged between the pixel unit and the signal processing chip 111 in the stacking direction, and is controlled so that an accumulation time for accumulating the charge converted by the first photoelectric conversion unit is different from an accumulation time for accumulating the charge converted by the second photoelectric conversion unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device. [Background technology]

[0002] An electronic device has been proposed that includes an imaging element in which a back-illuminated imaging chip and a signal processing chip are stacked (hereinafter referred to as a stacked imaging element) (see Patent Document 1). In the stacked imaging element, the back-illuminated imaging chip and the signal processing chip are stacked so that they are connected via microbumps in each predetermined area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2006-49361 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional electronic devices equipped with stacked imaging elements, there have not been many proposals for dividing an image into blocks each having one or more of the above-mentioned regions and acquiring an image for each block, and the usability of electronic devices equipped with stacked imaging elements has not been sufficient. [Means for solving the problem]

[0005] An image sensor according to a first aspect of the invention includes a first semiconductor substrate on which light from an optical system is incident, the first semiconductor substrate being provided with pixel sections each having a first pixel including a first photoelectric conversion unit that converts light into an electric charge and a first output unit that outputs a first signal based on the electric charge converted by the first photoelectric conversion unit and used for focus adjustment of the optical system, and a second pixel including a second photoelectric conversion unit that converts light into an electric charge and is arranged along the row direction together with the first photoelectric conversion unit, and a second output unit that outputs a second signal based on the electric charge converted by the second photoelectric conversion unit and used for image generation, and a semiconductor substrate stacked on the first semiconductor substrate, the first conversion unit that converts the first signal into a digital signal and the second conversion unit that converts the second signal into a digital signal. a first junction having a first conductive member through which the first signal read out from the first pixel is output, the first conductive member being arranged so as to face each other in a stacking direction in which the first semiconductor substrate and the second semiconductor substrate are stacked; and a second junction having a second conductive member through which the second signal read out from the second pixel is output, the second conductive member being arranged so as to face each other in the stacking direction, wherein the first conductive member and the second conductive member are arranged between the pixel portion and the second semiconductor substrate in the stacking direction, and an accumulation time for accumulating the charges converted by the first photoelectric conversion portion is controlled to be different from an accumulation time for accumulating the charges converted by the second photoelectric conversion portion. [Effects of the Invention]

[0006] According to the present invention, it is possible to realize an imaging device that can perform a plurality of main imaging pre-processing processes with high accuracy and is easy to use. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is a cross-sectional view of a stacked imaging element. [Figure 2] 1 is a diagram illustrating a pixel array and a unit area of ​​an imaging chip. FIG. [Figure 3] FIG. 2 is a circuit diagram corresponding to a unit area of ​​the imaging chip. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of an imaging element. [Figure 5] FIG. 1 is a block diagram illustrating a configuration of an imaging device. [Figure 6] 2 is a diagram illustrating an example of the arrangement of an AF area and an AE area in an image sensor. [Figure 7] 2 is a diagram illustrating an example of the arrangement of an AF area and an AE area in an image sensor. [Figure 8] 2 is a diagram illustrating an example of the arrangement of an AF area and an AE area in an image sensor. [Figure 9] 10 is a flowchart illustrating the flow of a photographing operation executed by a control unit according to the first embodiment. [Figure 10] 10A and 10B are diagrams illustrating the relationship between an AF area and an AE area and a live view image in a second embodiment. [Figure 11] FIG. 1 is a diagram illustrating three settings for an imaging element. [Figure 12] 10 is a flowchart illustrating the flow of a photographing operation executed by a control unit according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (First embodiment) <Explanation of stacked image sensor> First, a stacked imaging element 100 mounted in an electronic device (e.g., imaging device 1) according to a first embodiment of the present invention will be described. This stacked imaging element 100 is described in Japanese Patent Application No. 2012-139026, previously filed by the applicant of the present application. FIG. 1 is a cross-sectional view of the stacked imaging element 100. The imaging element 100 includes a back-illuminated imaging chip 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes the pixel signals, and a memory chip 112 that stores the pixel signals. The imaging chip 113, signal processing chip 111, and memory chip 112 are stacked and electrically connected to each other by conductive bumps 109 made of Cu or the like.

[0009] As shown in the figure, incident light is mainly incident in the positive direction of the Z axis, as indicated by the white arrow. In this embodiment, the surface of the imaging chip 113 on which the incident light is incident is referred to as the back surface. As shown by the coordinate axes, the left direction on the paper, perpendicular to the Z axis, is the positive X axis, and the front direction on the paper, perpendicular to the Z axis and the X axis, is the positive Y axis. In the following figures, the coordinate axes are displayed so that the orientation of each figure can be understood, based on the coordinate axes in Figure 1.

[0010] An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is arranged on the back side of the wiring layer 108. The PD layer 106 has a plurality of PDs (photodiodes) 104 arranged two-dimensionally and accumulating charges according to incident light, and transistors 105 provided corresponding to the PDs 104.

[0011] A color filter 102 is provided on the incident side of the PD layer 106, on which incident light is incident, via a passivation film 103. There are multiple types of color filters 102 that transmit different wavelength ranges, and each has a specific arrangement corresponding to each PD 104. The arrangement of the color filters 102 will be described later. A set of a color filter 102, a PD 104, and a transistor 105 forms one pixel.

[0012] A microlens 101 is provided corresponding to each pixel on the incident light side of the color filter 102. The microlens 101 condenses the incident light toward the corresponding PD 104.

[0013] The wiring layer 108 has wiring 107 that transmits pixel signals from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multi-layered, and may be provided with passive elements and active elements.

[0014] A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with a plurality of bumps 109 provided on the opposing surface of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed together, whereby the aligned bumps 109 are bonded together and electrically connected.

[0015] Similarly, a plurality of bumps 109 are arranged on the opposing surfaces of the signal processing chip 111 and the memory chip 112. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressed together, whereby the aligned bumps 109 are bonded together and electrically connected.

[0016] The bonding between the bumps 109 is not limited to Cu bump bonding by solid-phase diffusion, but may also employ micro-bump bonding by solder melting. For example, it is sufficient to provide approximately one bump 109 per unit area, which will be described later. Therefore, the size of the bumps 109 may be larger than the pitch of the PDs 104. Furthermore, in a peripheral area other than the pixel area where pixels are arranged, bumps larger than the bumps 109 corresponding to the pixel area may also be provided.

[0017] The signal processing chip 111 has through-silicon vias (TSVs) 110 that connect circuits provided on the front and back surfaces of the chip to each other. The TSVs 110 are preferably provided in the peripheral region. The TSVs 110 may also be provided in the peripheral region of the imaging chip 113 and the memory chip 112.

[0018] FIG. 2 is a diagram illustrating the pixel array of the imaging chip 113 and the unit area 131. In particular, the imaging chip 113 is shown as viewed from the back side. For example, more than 20 million pixels are arranged in a matrix in the pixel area. In this embodiment, for example, 16 pixels, 4 pixels by 4 pixels adjacent to each other, form one unit area 131. The grid lines in the figure show the concept of forming the unit area 131 by grouping adjacent pixels. The number of pixels forming the unit area 131 is not limited to this, and may be around 1000, for example, 32 pixels by 64 pixels, or may be more or less than that (for example, 1 pixel).

[0019] As shown in the partially enlarged view of the pixel region, unit region 131 contains four so-called Bayer arrays, arranged vertically and horizontally, each consisting of four pixels: green pixels Gb, Gr, blue pixels B, and red pixels R. The green pixels are pixels that have a green filter as their color filter 102 and receive light in the green wavelength band of incident light. Similarly, the blue pixels are pixels that have a blue filter as their color filter 102 and receive light in the blue wavelength band, and the red pixels are pixels that have a red filter as their color filter 102 and receive light in the red wavelength band.

[0020] In this embodiment, multiple blocks are defined, each containing at least one unit area 131, and each block can control the pixels contained in the block using different control parameters. In other words, imaging signals can be acquired under different imaging conditions for a group of pixels contained in one block and a group of pixels contained in another block. Examples of control parameters include frame rate, gain, thinning rate, number of rows or columns for adding pixel signals, charge accumulation time or number of accumulations, and digitization bit rate. Furthermore, the control parameters may be parameters for image processing after image signals are acquired from the pixels.

[0021] 3 is a circuit diagram corresponding to a unit area 131 of the imaging chip 113. In FIG. 3, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. Note that at least some of the transistors described below correspond to the transistor 105 in FIG. 1.

[0022] As described above, the unit area 131 is formed of 16 pixels. The 16 PDs 104 corresponding to the respective pixels are connected to transfer transistors 302, and each gate of each transfer transistor 302 is connected to a TX wiring 307 through which a transfer pulse is supplied. In this embodiment, the TX wiring 307 is commonly connected to the 16 transfer transistors 302.

[0023] The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the amplification transistor 304. The drain of the reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied, and the gate of the reset transistor 303 is connected to a reset wiring 306 to which a reset pulse is supplied. In this embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303.

[0024] The drain of each amplification transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of a corresponding transistor 305. The gate of each selection transistor is connected to a decoder wiring 308 to which a selection pulse is supplied. In this embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The sources of each selection transistor 305 are connected to a common output wiring 309. A load current source 311 supplies a current to the output wiring 309. In other words, the output wiring 309 for the selection transistor 305 is formed by a source follower. The load current source 311 may be provided on the imaging chip 113 side or on the signal processing chip 111 side.

[0025] Here, we will explain the flow from the start of charge accumulation to pixel output after accumulation ends. When a reset pulse is applied to the reset transistor 303 via the reset wiring 306 and at the same time a transfer pulse is applied to the transfer transistor 302 via the TX wiring 307, the potentials of the PD 104 and the floating diffusion FD are reset.

[0026] When the application of the transfer pulse is stopped, the PD 104 converts the incident light it receives into electric charges and accumulates them. Then, when the transfer pulse is applied again without the reset pulse being applied, the accumulated electric charges are transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. When a selection pulse is applied to the selection transistor 305 via the decoder wiring 308, the fluctuation in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 via the amplification transistor 304 and the selection transistor 305. As a result, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

[0027] As shown in FIG. 3, in this embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels that form the unit area 131. That is, the reset pulse and the transfer pulse are each applied simultaneously to all 16 pixels. Therefore, all pixels that form the unit area 131 start and end charge accumulation at the same timing. However, pixel signals corresponding to the accumulated charges are selectively output from the output wiring 309 by sequentially applying selection pulses to the respective selection transistors 305. Furthermore, the reset wiring 306, the TX wiring 307, and the output wiring 309 are provided separately for each unit area 131.

[0028] By configuring the circuit in this way based on the unit area 131, it is possible to control the charge accumulation time for each unit area 131. In other words, it is possible to output pixel signals at different frame rates between the unit areas 131. Furthermore, while one unit area 131 is performing a single charge accumulation, the other unit area 131 is caused to repeatedly accumulate charge and output a pixel signal each time, thereby making it possible to output frames for a moving image at different frame rates between these unit areas 131.

[0029] 4 is a block diagram showing the functional configuration of the image sensor 100. An analog multiplexer 411 sequentially selects the 16 PDs 104 that form a unit area 131 and outputs the pixel signals from each of the 16 PDs 104 to the output wiring 309 provided corresponding to the unit area 131. The multiplexer 411 is formed on the image sensor chip 113 together with the PDs 104.

[0030] The pixel signals output via the multiplexer 411 undergo correlated double sampling (CDS) and analog-to-digital (A / D) conversion by a signal processing circuit 412 that performs CDS and A / D conversion and is formed in the signal processing chip 111. The A / D converted pixel signals are passed to a demultiplexer 413 and stored in pixel memories 414 corresponding to the respective pixels. The demultiplexer 413 and pixel memories 414 are formed in the memory chip 112.

[0031] The arithmetic circuit 415 processes the pixel signals stored in the pixel memory 414 and passes them to a downstream image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or in the memory chip 112. Note that while Fig. 4 shows connections for one unit area 131, in reality, there is one arithmetic circuit for each unit area 131 and they operate in parallel. However, there does not need to be a arithmetic circuit 415 for each unit area 131; for example, one arithmetic circuit 415 may perform sequential processing while referring to the values ​​of the pixel memories 414 corresponding to each unit area 131 in order.

[0032] As described above, output wiring 309 is provided corresponding to each unit area 131. Since the imaging element 100 has the imaging chip 113, signal processing chip 111, and memory chip 112 stacked on top of each other, by using bumps 109 for electrical connection between the chips for these output wiring 309, it is possible to route the wiring without increasing the size of each chip in the planar direction.

[0033] <Explanation of imaging device> 5 is a block diagram illustrating the configuration of an imaging device 1 having the above-described imaging element 100. In FIG. 5, the imaging device 1 has an imaging optical system 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, a recording unit 60, and a control unit 70.

[0034] The imaging optical system 10 is composed of multiple lenses and guides a light beam from the subject field to the imaging unit 20. The imaging optical system 10 may be configured integrally with the imaging device 1, or may be configured to be replaceable with respect to the imaging device 1. The imaging optical system 10 may also have a built-in focus lens or a built-in zoom lens.

[0035] The imaging unit 20 has the above-mentioned imaging element 100 and a driving unit 21 that drives the imaging element 100. The imaging element 100 is driven and controlled by a control signal output from the driving unit 21, thereby enabling independent accumulation control for each block as described above. The control unit 70 issues instructions to the driving unit 21 regarding the position, shape, range, etc. of the blocks.

[0036] The image processing unit 30 performs image processing on the image data captured by the imaging unit 20 in cooperation with the work memory 40. In the present embodiment, in addition to normal image processing (such as color signal processing and gamma correction), the image processing unit 30 also performs detection processing of the main subject included in the image. The detection of the main subject by the image processing unit 30 can be performed using a known face detection function. In addition to face detection, for example, as described in Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940), the human body included in the image may be detected as the main subject.

[0037] The work memory 40 temporarily stores image data before and after JPEG compression, MPEG compression, and the like. The display unit 50 is constituted by, for example, a liquid crystal display panel 51, and displays an image (still image or moving image) captured by the imaging unit 20, various information, or an operation input screen. The display unit 50 has a configuration in which a touch panel 52 is laminated on the display surface of the liquid crystal display panel 51. The touch panel 52 outputs a signal indicating the position where the user touches the liquid crystal display panel 51.

[0038] The recording unit 60 stores various data such as image data in a storage medium such as a memory card. The control unit 70 has a CPU and controls the overall operation of the imaging device 1. In the present embodiment, the control unit 70 divides the imaging surface of the imaging element 100 (imaging chip 113) into a plurality of blocks, and acquires images at different frame rates and gains between the blocks. For this purpose, the control unit 70 instructs the drive unit 21 of the position, shape, range of the block, and control parameters for each block.

[0039] In addition, the control unit 70 calculates the focus adjustment state by the imaging optical system 10 by the AF calculation unit 71. The control unit 70 further performs exposure calculation by the AE and AWB calculation unit 72 so as to obtain appropriate exposure.

[0040] <AF Region and AE Region> In the first embodiment, the concepts of AF areas and AE areas are introduced into the screen and correspond to the above-mentioned multiple blocks. Fig. 6 is a diagram illustrating the arrangement of AF areas and AE areas in the image sensor 100 (image sensor chip 113). In Fig. 6, the hatched areas represent AE areas, and the white areas represent AF areas. In the first embodiment, the AF areas and AE areas are predetermined to be arranged in a checkerboard pattern. For example, when acquiring a live view image, the control unit 70 performs focus detection processing in the AF calculation unit 71 using signals output from the AF areas of the image sensor 100.

[0041] Here, the live view image is also called a preview image before actual imaging is performed, and refers to an image for monitoring acquired by the image sensor 100 at a predetermined frame rate (e.g., 30 fps). The focus detection process is performed, for example, by a contrast detection method. Specifically, while moving the position of the focus lens of the imaging optical system 10, the position of the focus lens of the imaging optical system 10 is adjusted so as to increase the contrast of the image formed by the signal output from the AF area.

[0042] The focus detection process may be performed using a phase difference detection method. In this case, focus detection pixels are provided within the AF area of ​​the image sensor 100 (image capture chip 113). Then, a phase difference detection calculation is performed using output signals from the focus detection pixels to detect the focus adjustment state (specifically, the defocus amount) of the image capture optical system 10. The focus detection pixels and phase difference detection calculation are well known, as described in, for example, Japanese Patent Application Laid-Open No. 2009-94881, and therefore detailed description thereof will be omitted.

[0043] Furthermore, the control unit 70 performs exposure calculation processing in the AE / AWB calculation unit 72 using the signal output from the AE region when the live view image is acquired. The AE / AWB calculation unit 72 determines exposure (frame rate, gain, etc.) so as to bring the average level of the signal output from the AE region closer to a predetermined level, for example. The control unit 70 also determines a white balance adjustment value based on the signal output from the AE region. Furthermore, the control unit 70 generates an image for monitoring based on the signal output from the AE region and displays it on the display unit 50 as the live view image.

[0044] When acquiring a live view image, the control unit 70 instructs the drive unit 21 to divide the image sensor 100 (image sensor chip 113) into an AF area and an AE area and control accumulation. In the first embodiment, accumulation control for the AF area and the AE area is performed before an instruction to actually capture an image (record a still image or record a video) is issued by a release switch (not shown).

[0045] That is, until an actual image capture command is issued, different images are obtained by applying different control parameters to the AF area and the AE area. Then, focus detection processing is performed based on the image acquired from the AF area, and exposure calculation, white balance calculation, and live view image display are performed based on the image acquired from the AE area.

[0046] The control parameters for the AF area, which are the pre-processing for actual imaging, are, for example, a frame rate of 120 fps and a higher gain for the AE area. The control parameters for the AE area, which are the pre-processing for actual imaging, are, for example, a frame rate of 30 fps, which is slower than that for the AF area, and a lower gain than that for the AF area. In this way, by applying a higher frame rate to the AF area compared to the AE area to acquire images used for focus detection processing, the responsiveness of the focus detection processing can be improved. Furthermore, by acquiring images used for exposure calculation processing and determining white balance adjustment values ​​with the gain for the AE area lower than that for the AF area, the effects of noise can be avoided and calculations can be performed with high accuracy.

[0047] The arrangement of the AF areas and AE areas in the image sensor 100 (image sensor chip 113) is not limited to the arrangement shown in Fig. 6, and the individual sizes of the AF areas and AE areas may be changed as appropriate. As shown in Fig. 7, the sizes may be smaller than those in Fig. 6, or larger than those in Fig. 6.

[0048] In addition to changing the sizes of the AF area and the AE area, the area ratio between the AF area and the AE area may also be changed as appropriate, as shown in FIG. 8. FIG. 8 is a diagram illustrating an example in which the area of ​​the AE area is made larger than the area of ​​the AF area and the area ratio between the two is changed. Since the live view image described above is generated by thinning out some of the pixels of the image sensor 100, the image quality of the live view image can be prevented from deteriorating by, for example, setting the AF area to correspond to the thinned pixels and setting the AE area to correspond to the pixels of the live view image. Note that if it is desired to place the AF area only in a predetermined position, such as the center of the screen, the AF area may be of a predetermined size and placed only in a predetermined position on the screen.

[0049] <Explanation of the flowchart> Fig. 9 is a flowchart illustrating the flow of the photographing operation executed by the control unit 70 of the imaging device 1 in the first embodiment. The control unit 70 repeatedly starts the processing shown in Fig. 9 when an ON-OFF switch (not shown) is turned on and power is being supplied to each unit of the imaging device 1. In step S101 of Fig. 9, the control unit 70 determines control parameters such as frame rate and gain for the AF area and AE area, and then proceeds to step S102. For example, values ​​to be applied in steps S102, S104, and S105 (described later) are read from program data and prepared.

[0050] In step S102, the control unit 70 sends an instruction to the drive unit 21 to start capturing images using the imaging unit 20. The image acquisition that starts in step S102 is performed by, for example, setting control parameters for the AE area, with substantially the entire imaging surface of the image sensor 100 as the AE area. The control unit 70 causes the image processing unit 30 to process live view image data based on the signal output from the imaging unit 20, and then causes the display unit 50 to display the image.

[0051] In step S103, the control unit 70 determines whether or not the release button has been pressed halfway. The release button being pressed halfway is used as an instruction to the imaging device 1 to start the actual imaging pre-processing (preparation for shooting). If the release button (not shown) has been pressed halfway, the control unit 70 makes a positive decision in step S103 and proceeds to step S104, but if the release button has not been pressed halfway, the control unit 70 makes a negative decision in step S103 and repeats the determination process. To repeat the determination process, the control unit waits for the release button to be pressed halfway.

[0052] Note that, if the imaging device 1 is not provided with a release button and an instruction to capture video is determined by detecting a tap operation on a release icon displayed on the liquid crystal display panel 51, the processing of step S103 may be omitted. In this case, after turning on the power and starting the processing of Fig. 9, the control unit 70 performs processing of steps S104 and S105 (described later) from the start of imaging in step S102, and applies different control parameters (frame rate, gain) to the AF area and the AE area that have been determined in advance, as exemplified in Fig. 6. Furthermore, even if the electronic device 1 is provided with a video switch that instructs the start and end of video capture, the control unit 70 may perform processing similar to that of a tap operation on the release icon, or may perform pre-imaging processing (preparation for shooting) within a few seconds after the video switch is turned on.

[0053] In step S104, the control unit 70 sends an instruction to the drive unit 21 to apply control parameters such as the frame rate and gain for the AF area to the AF area of ​​the image sensor 100. In step S105, the control unit 70 sends an instruction to the drive unit 21 to apply control parameters such as the frame rate and gain for the AE area to the AE area of ​​the image sensor 100. As a result, the control unit 70 fine-tunes the control parameters, determines a white balance adjustment value, and displays a live view image based on the signal output from the AE area at the above frame rate.

[0054] In step S106, the control unit 70 starts AF processing. Specifically, the AF calculation unit 71 starts focus detection processing based on the signal output from the AF area, and the process proceeds to step S107. This allows focus adjustment of the imaging optical system 10 to be performed based on the signal output from the AF area at the above frame rate.

[0055] In step S107, the control unit 70 determines whether the AF process is complete. For example, if the contrast of the image obtained from the AF area is equal to or greater than a predetermined value, the control unit 70 makes a positive determination in step S107, and if the contrast of the image obtained from the AF area is less than the predetermined value, the control unit 70 makes a negative determination in step S107. If a negative determination is made, the control unit 70 repeats the determination process while continuing the focus detection process.

[0056] In step S108, the control unit 70 causes the AE / AWB calculation unit 72 to calculate the exposure conditions and white balance adjustment values ​​to be applied during shooting based on the signal output from the AE region after focus adjustment, and then the process proceeds to step S109.

[0057] In step S109, the control unit 70 determines whether the release button has been fully pressed. The full release press is used as an instruction to the imaging device 1 to perform actual imaging. If the release button (not shown) has been fully pressed, the control unit 70 makes an affirmative decision in step S109 and proceeds to step S110, whereas if the release button (not shown) has not been fully pressed, the control unit 70 makes a negative decision in step S109 and returns to step S108. If returning to step S108, the above-mentioned processing is repeated. Note that if the release button (not shown) has been fully pressed, the control unit 70 cancels imaging by the imaging unit 20 using the control parameters for the AF region and the control parameters for the AE region.

[0058] As described above, when detecting a tap operation on the release icon displayed on the liquid crystal display panel 51 to determine whether or not an instruction to capture an actual image has been given, a positive judgment can be made in step S109 if the release icon being displayed is tapped.

[0059] In step S110, the control unit 70 sends an instruction to the driving unit 21 to set the control parameters (exposure time, gain, etc.) required for the exposure conditions for shooting (main shooting) calculated in step S108, and the process proceeds to step S111.

[0060] In step S111, the control unit 70 executes a photographing process, stores the acquired image data on a memory card or the like using the recording unit 60, and ends the process shown in FIG. 9. In the photographing process, photographing control parameters set to be common (the same) across the entire area of ​​the image sensor 100 are applied to acquire and record one still image (main image capture), and also acquire multiple frames of images after acquiring the still image. Note that if a high-brightness subject is present in part of the subject scene, the control unit 70 may change the imaging conditions for pixels corresponding to the high-brightness subject. Then, based on multiple frames of images acquired within a predetermined time before and after the full-press of the shutter release button, slow-motion playback video data is generated and recorded. Slow-motion playback video data refers to moving image data played back at a frame rate (e.g., 15 fps) slower than the frame rate (e.g., 30 fps) acquired by the image sensor 100.

[0061] The control unit 70 generates slow motion video data as follows: Specifically, the slow motion video data is generated based on a plurality of frame images (the frame rate is the frame rate set for the AE region; for example, 0.6 seconds when captured at 30 fps is 18 frames) based on the AE region temporarily stored in the work memory 40 for displaying the above-described live view image from a first predetermined time (e.g., 0.6 seconds) before the time of the full release press (t1) to time t1, and a plurality of frame images (the frame rate is the same as the frame rate set for the AE region before the actual imaging instruction is issued; for example, 0.4 seconds when captured at 30 fps is 12 frames) stored in the work memory 40 from time t1 to a second predetermined time (e.g., 0.4 seconds after t1). This generates slow motion video data with a playback time of approximately two seconds based on a plurality of frame images (30 images in total) stored in the work memory 40 during the one second period surrounding time t1 (from 0.6 seconds before time t1 to 0.4 seconds after time t1). In this way, slow motion video data is obtained based on the frame images acquired before and after the release button is fully pressed. The slow motion video data is generated by the image processor 30 as MPEG data or JPEG data.

[0062] According to the first embodiment described above, the following effects can be obtained. (1) The imaging device 1 can obtain signals used for exposure calculation processing and for detecting the focus adjustment state at appropriate levels, allowing for quick and accurate exposure calculation and focus adjustment. As a result, an easy-to-use electronic device can be realized.

[0063] (2) In response to a full press of the release switch (not shown), the control unit 70 cancels the imaging of the imaging unit 20 using the control parameters for the AF area and the control parameters for the AE area, and drives the imaging unit 20 under imaging conditions suitable for the actual imaging, thereby enabling better still images and videos to be obtained.

[0064] (3) The control unit 70 can improve the responsiveness of the focus detection process by acquiring signals used in the focus detection process at a frame rate higher in the second region than in the first region.

[0065] (4) The control unit 70 performs exposure calculation processing with a lower gain setting for the first area than for the second area, so that it is possible to perform calculations with high accuracy while suppressing the influence of noise.

[0066] (5) When the control unit 70 determines the first and second areas in advance, it can determine exposure based on a predetermined position within the screen, or perform focus adjustment based on a predetermined position within the screen.

[0067] (Second embodiment) In the first embodiment described above, an example was described in which the arrangement of the AF area and the AE area in the image sensor 100 (image sensor chip 113) was determined in advance, but in the second embodiment, the arrangement of the AF area and the AE area in the image sensor 100 (image sensor chip 113) is determined through scene recognition based on a live view image. Note that the configuration of the image sensor 1 is the same as the configuration of Fig. 5 described in the first embodiment, and therefore a description of the configuration will be omitted.

[0068] FIG. 10 is a diagram illustrating the relationship between the AF area, the AE area, and a live-view image in the second embodiment. In FIG. 10, a person is included in the foreground center of the live-view image, a tree is included in the upper left corner of the live-view image, and the sun is included in the upper right corner of the live-view image. The control unit 70 sends an instruction to the image processing unit 30 to perform known scene recognition processing on the live-view image data. By performing the scene recognition processing, the image processing unit 30 extracts a high-brightness area A, a main subject area B, and a background area C from the live-view image.

[0069] For example, image processing unit 30 determines a range where the signal level of the live view image data exceeds a predetermined high brightness threshold as high brightness region A. Image processing unit 30 also determines a range including the human body detected as described above as main subject region B. Note that in addition to people, animals such as pets may be detected and the range including this animal may be determined as main subject region B. Image processing unit 30 also performs edge detection in a position excluding high brightness region A and main subject region B, and determines a range where the density of detected edges exceeds a predetermined value as background region C. Note that if there is no data exceeding the high brightness threshold in the live view image data, high brightness region A is deemed not to exist.

[0070] The control unit 70 arranges the AF and AE areas in the main subject area B, for example, in a checkerboard pattern. In FIG. 10, the hatched areas in the main subject area B represent AE areas, and the white areas represent AF areas. The arrangement of the AF and AE areas in the main subject area B is not limited to the arrangement illustrated in FIG. 10, and the individual sizes of the AF and AE areas may be changed as appropriate. Furthermore, in addition to changing the sizes of the AF and AE areas, the area ratio of the AF and AE areas may be changed as appropriate so that the area of ​​the AE area is larger than the area of ​​the AF area. Furthermore, if it is desired to arrange the AF areas only in predetermined positions, such as the center of the main subject area B, the AF areas may be arranged in predetermined positions with predetermined sizes. Furthermore, the control unit 70 may arrange the AF and AE areas so that portions of the AF and AE areas extend beyond the main subject area B.

[0071] Furthermore, the control unit 70 determines the background area C as an AE area. In Fig. 10, the hatched portion in the background area C represents the AE area. Furthermore, the control unit 70 does not determine an AF area or an AE area for the portion exceeding the high-brightness threshold (high-brightness area A in the example of Fig. 10). In Fig. 10, the portion in the high-brightness area A with vertical lines represents that it is neither an AF area nor an AE area.

[0072] In the second embodiment, the control unit 70 determines that the areas on the screen of the live view image that do not belong to any of the high brightness area A, the main subject area B, and the background area C (the white areas in Figure 10) are AE areas.

[0073] In the second embodiment, three settings are made to the image sensor 100 (image capture chip 113) when capturing a live view image. Fig. 11 is a diagram illustrating the three settings. Setting 1 indicates that control parameters (frame rate, gain, etc.) suitable for high luminance are set for a high luminance area A (i.e., neither an AF area nor an AE area) by, for example, increasing the frame rate and decreasing the gain so as not to saturate the output signal.

[0074] Setting 2 indicates that control parameters (frame rate, gain, etc.) suitable for focus detection processing are set for the AF area in the main subject area B, as in the first embodiment, and that control parameters (frame rate, gain, etc.) suitable for exposure calculation processing are set for the AE area in the main subject area B, as in the first embodiment.

[0075] Setting 3 indicates that control parameters (frame rate, gain, etc.) suitable for exposure calculation processing are set for the background region C and the parts that do not belong to either the high-brightness region A or the main subject region B (white areas in Figure 10), in the same way as for the AE region in the main subject region B.

[0076] The control unit 70 sends instructions to the drive unit 21 to cause the image sensor 100 (image sensor chip 113) to perform the above three settings. As a result, when the image sensor 100 (image sensor chip 113) acquires a live view image, accumulation control is performed using different control parameters (frame rate, gain, etc.) for the AF area, the AE area, and areas that are neither the AF area nor the AE area. This accumulation control is performed from, for example, after the scene is recognized by the image processing unit 30 until an actual image capture instruction is issued.

[0077] That is, before an actual imaging instruction is given, different images are obtained by applying different control parameters to the AF area, the AE area, and an area that is neither an AF area nor an AE area. Then, focus detection processing is performed based on the image obtained from the AF area, exposure calculation and white balance calculation are performed based on the image obtained from the AE area, and a live view image is displayed based on the image obtained in the AE area and the image obtained in the area that is neither an AF area nor an AE area.

[0078] The control parameters for the AF area are, for example, a frame rate of 120 fps, and a higher gain for the AE area. The control parameters for the AE area are, for example, a frame rate of 30 fps, which is slower than that for the AF area, and a lower gain than that for the AF area. The reason for applying a higher frame rate to the AF area compared to the AE area to acquire images used for focus detection processing is to improve the responsiveness of the focus detection processing. In addition, the reason for lowering the gain for the AE area compared to the gain for the AF area to acquire images used for exposure calculation processing and determining white balance adjustment values ​​is to avoid the effects of noise and perform calculations with high precision.

[0079] The control parameters for areas that are neither AF nor AE areas are, for example, a frame rate of 60 fps and a lower gain than for AE areas, which makes it possible to capture high-brightness subject images without overexposure, even when the subject is overexposed.

[0080] <Explanation of the flowchart> 12 is a flowchart illustrating the flow of the photographing operation executed by the control unit 70 of the imaging device 1 in the second embodiment. Since the processing in steps S104B and S105B is different from the flowchart in the first embodiment illustrated in FIG. 9, these differences will be mainly described.

[0081] As in the case of FIG. 9, image acquisition started in step S102 of FIG. 12 is performed by setting control parameters for the AE region, with substantially the entire imaging surface of the image sensor 100 as the AE region. The control unit 70 targets live-view image data based on the signal output from the imaging unit 20 according to the above settings as the scene recognition target. In step S104B, the control unit 70 sends an instruction to the image processing unit 30 to perform known scene recognition processing on the live-view image data. By performing the scene recognition processing, the image processing unit 30 divides the image into a high-brightness region A, a main subject region B, and a background region C, as described above.

[0082] In step S105B, the control unit 70 sends an instruction to the drive unit 21 to perform the three settings described above for each divided area. This allows different images to be obtained by applying different control parameters to the AF area, the AE area, and areas that are neither AF nor AE areas. The control unit 70 can fine-tune the control parameters and determine white balance adjustment values ​​based on the image acquired from the AE area. In addition, a live view image can be displayed based on the image acquired in the AE area and the image acquired in the area that is neither AF nor AE area.

[0083] According to the second embodiment described above, the following advantageous effects can be obtained. (1) The imaging device 1 can change the position on the screen that is the target of exposure calculation and the position that is the target of focus adjustment based on the results of scene analysis.

[0084] (2) The control unit 70 defines the first and second regions within the range of the image that the image processing unit 30 determines to include the main subject, so that exposure can be determined for the main subject within the screen and focus adjustment can be performed for the main subject.

[0085] (3) The control unit 70 determines the first region outside the range determined by the image processing unit 30 in the image, so that the background other than the main subject can also be included in the exposure calculation.

[0086] (4) The control unit 70 sets the first and second regions in the range of the image in areas other than the area exceeding a predetermined brightness. Therefore, for example, if the signal from the imaging unit 20 becomes saturated, these can be excluded from the targets of exposure calculation and focus adjustment.

[0087] (Variation 1) The imaging device 1 according to the first and second embodiments described above may be configured as a high-performance mobile phone or a tablet terminal. In this case, the camera unit mounted on the high-performance mobile phone (or tablet terminal) is configured using the stacked imaging element 100.

[0088] (Variation 2) In the above description, an example has been described in which the control parameters are fine-tuned and the white balance adjustment value is determined based on the image acquired from the AE area. Alternatively, an AWB area may be newly provided in addition to the AF area and the AE area. Then, when acquiring a live view image, the image sensor 100 (imaging chip 113) performs accumulation control using different control parameters (frame rate, gain, etc.) for the AF area, AE area, and AWB area.

[0089] In Modification 2, the control unit 70 performs focus detection processing based on an image acquired from the AF area. The control unit 70 determines a white balance adjustment value based on an image acquired from the AWB area. Fine adjustment of control parameters is performed based on an image acquired from the AE area. Furthermore, live view images are displayed based on an image acquired in the AE area and an image acquired in an area that is neither the AF area nor the AE area.

[0090] Although various embodiments and modifications have been described above, the present invention is not limited to these. The configurations of the embodiments and modifications may be combined as appropriate. Other aspects conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention. The order of processing in each flowchart may also be changed as appropriate. [Explanation of symbols]

[0091] 1...imaging device 10...Imaging optical system 20...imaging unit 30...Image processing unit 40...Work memory 50...Display section 51...LCD panel 52...Touch panel 60...Recording section 70...Control unit 71...AF calculation unit 72...AE, AWB calculation section 100...Image sensor 109...Bump 111...Signal processing chip 112...Memory chip 113...imaging chip 131...Unit area

[0092] The disclosures of the following priority applications are incorporated herein by reference: Japanese Patent Application No. 2013-037617 (filed February 27, 2013)

Claims

1. a first semiconductor substrate on which light from an optical system is incident, the first pixel including a first photoelectric conversion unit that converts light into electric charges and a first output unit that outputs a first signal based on the electric charges converted by the first photoelectric conversion unit and used for focus adjustment of the optical system; a second pixel including a second photoelectric conversion unit that converts light into electric charges and is arranged along the row direction together with the first photoelectric conversion unit, and a second output unit that outputs a second signal based on the electric charges converted by the second photoelectric conversion unit and used for image generation; a second semiconductor substrate stacked on the first semiconductor substrate, the second semiconductor substrate having a first conversion unit that converts the first signal into a digital signal and a second conversion unit that converts the second signal into a digital signal disposed thereon; a first bonding portion having first conductive members to which the first signal read out from the first pixel is output, the first conductive members being arranged to face each other in a stacking direction in which the first semiconductor substrate and the second semiconductor substrate are stacked; a second bonding portion including second conductive members that are arranged to face each other in the stacking direction and through which the second signal read out from the second pixel is output; Equipped with the first conductive member and the second conductive member are disposed between the pixel unit and the second semiconductor substrate in the stacking direction; An imaging element in which an accumulation time for accumulating the electric charges converted by the first photoelectric conversion unit is controlled to be different from an accumulation time for accumulating the electric charges converted by the second photoelectric conversion unit.

2. 2. The imaging device according to claim 1, An imaging element in which an accumulation time for accumulating the electric charges converted by the first photoelectric conversion unit is controlled to be shorter than an accumulation time for accumulating the electric charges converted by the second photoelectric conversion unit.

3. 3. The imaging device according to claim 1, The first signal is an image sensor used for focus adjustment of the optical system using a phase difference detection method.

4. 4. The imaging device according to claim 3, The imaging element, wherein the first conductive member and the second conductive member are made of copper.

5. 5. The imaging device according to claim 1, an imaging element comprising a wiring layer having a first signal line, which is wiring through which the first signal read out from the first pixel is output and is electrically connected to the first conductive member, and a second signal line, which is wiring through which the second signal read out from the second pixel is output and is electrically connected to the second conductive member.

6. 6. The imaging device according to claim 5, The wiring layer is disposed between the first semiconductor substrate and the second semiconductor substrate in the stacking direction.

7. 7. The imaging device according to claim 1, An imaging element comprising a third semiconductor substrate stacked on the first semiconductor substrate, and on which a first memory unit for storing a first digital signal converted from the first signal by the first conversion unit and a second memory unit for storing a second digital signal converted from the second signal by the second conversion unit are arranged.

8. The imaging device according to any one of claims 1 to 7, a first signal line for outputting the first signal read from the first pixel to the first conversion unit; a second signal line for outputting the second signal read from the second pixel to the second conversion unit; An imaging element comprising:

9. 9. The imaging device according to claim 8, The image sensor has a first current source disposed on the second semiconductor substrate for supplying a current to the first signal line and a second current source disposed on the second semiconductor substrate for supplying a current to the second signal line.

10. 10. The imaging device according to claim 1, a first control line through which a control signal is output to control an accumulation time for accumulating the electric charges converted by the first photoelectric conversion unit; a second control line through which a control signal is output to control an accumulation time for accumulating the electric charges converted by the second photoelectric conversion unit.

11. 10. The imaging device according to claim 1, a first transfer unit included in the first pixel and configured to transfer the charges converted by the first photoelectric conversion unit; a second transfer unit included in the second pixel and configured to transfer the charges converted by the second photoelectric conversion unit; a first control line to which a first control signal is output for controlling a timing at which the first transfer unit transfers the electric charges converted by the first photoelectric conversion unit; a second control line through which a second control signal is output for controlling a timing at which the second transfer unit transfers the electric charges converted by the second photoelectric conversion unit; An imaging element comprising:

12. The imaging device according to claim 11, An imaging element that is controlled so that the timing at which the first control signal is output to the first control line is different from the timing at which the second control signal is output to the second control line.

13. 13. The imaging device according to claim 11, The first transfer unit and the second transfer unit are disposed on the first semiconductor substrate of the imaging element.

14. The imaging device according to claim 13, the first photoelectric conversion unit and the second photoelectric conversion unit are disposed in a semiconductor layer, the first control line and the second control line are arranged in a wiring layer; The wiring layer is disposed between the semiconductor layer and the second semiconductor substrate in the stacking direction.

15. The imaging device according to any one of claims 1 to 14, The second photoelectric conversion unit is disposed adjacent to the first photoelectric conversion unit in the row direction.

Images