Image sensor and electronic equipment

Abstract

To generate a plurality of kinds of images for the same subject.SOLUTION: One aspect of the present invention is an image sensor. The imaging element includes a first semiconductor unit including a first pixel, a second pixel, a third pixel that is arranged next to the first pixel in a column direction and includes a third photoelectric conversion unit that converts light into an electric charge, and a fourth pixel that is arranged next to the third pixel in a row direction and next to the second pixel in the column direction and includes a fourth photoelectric conversion unit that converts light into an electric charge. The imaging element includes a second semiconductor unit that is a semiconductor unit stacked together with the first semiconductor unit and includes a drive unit that performs control such that a timing at which accumulation of the charge converted by the first photoelectric conversion unit ends, a timing at which accumulation of the charge converted by the second photoelectric conversion unit ends, a timing at which accumulation of the charge converted by the third photoelectric conversion unit ends, and a timing at which accumulation of the charge converted by the fourth photoelectric conversion unit ends are different timings.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an imaging device. [Background technology]

[0002] An electronic device has been proposed that includes an imaging element in which a back-illuminated imaging chip and a signal processing chip are stacked (hereinafter, this imaging element will be referred to as a stacked imaging element) (see, for example, Patent Document 1). In the stacked imaging element, the back-illuminated imaging chip and the signal processing chip are stacked so that they are connected via microbumps for each block unit that groups multiple pixels together. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-49361 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the case of electronic devices equipped with conventional stacked imaging elements, there have not been many proposals for capturing images in units of multiple blocks, and the usability of electronic devices equipped with stacked imaging elements has not been sufficient.

[0005] An object of the present invention is to generate a plurality of types of images for the same subject. [Means for solving the problem]

[0006] One aspect of the present invention is an image sensor. The image sensor includes a first semiconductor unit having a first pixel including a first photoelectric conversion unit that converts light into electric charges, a second pixel arranged adjacent to the first pixel in the row direction and including a second photoelectric conversion unit that converts light into electric charges, a third pixel arranged adjacent to the first pixel in the column direction and including a third photoelectric conversion unit that converts light into electric charges, and a fourth pixel arranged adjacent to the third pixel in the row direction and adjacent to the second pixel in the column direction and including a fourth photoelectric conversion unit that converts light into electric charges. The image sensor includes a second semiconductor unit stacked together with the first semiconductor unit, the second semiconductor unit having a drive unit that controls the timings of ending accumulation of the electric charges converted by the first photoelectric conversion unit, the second photoelectric conversion unit, the third photoelectric conversion unit, and the fourth photoelectric conversion unit so that these timings are different from each other. [Effects of the Invention]

[0007] According to an aspect of the present invention, a plurality of types of images can be generated for the same subject. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a cross-sectional view of a stacked imaging element. [Figure 2] FIG. 2 is a diagram illustrating a pixel array and unit groups of an imaging chip. [Figure 3] FIG. 2 is a circuit diagram corresponding to a unit group of the imaging chip. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of an imaging element. [Figure 5] 1 is a block diagram showing a configuration of an electronic device according to a first embodiment. [Figure 6] 1 is a diagram showing the appearance of a digital camera as an example of an electronic device. [Figure 7] FIG. 2 is a functional block diagram of an image processing unit and a system control unit. [Figure 8] FIG. 10 is a diagram showing an arrangement pattern in each block. [Figure 9] 10 is a flowchart illustrating a photographing operation executed by a system control unit. [Figure 10] 10 is a flowchart illustrating an arrangement pattern setting process. [Figure 11] FIG. 10 is a diagram showing an example of setting a second arrangement pattern when a second still image mode is executed. [Figure 12] 10 is a timing chart showing the timing of charge accumulation in the first still image mode and the second still image mode. [Figure 13] 10A and 10B are diagrams showing a display example when a still image is displayed on the first display section and the second display section. [Figure 14] FIG. 10 is a diagram showing an example of setting a second arrangement pattern when a second moving image mode is executed. [Figure 15] 10 is a timing chart showing the timing of charge accumulation in the second moving image mode. [Figure 16] 10 is a diagram showing a display example in which a moving image is displayed on the first display section and a still image is displayed on the second display section. FIG. [Figure 17] FIG. 10 is a diagram showing a fifth arrangement pattern in each block. [Figure 18] FIG. 10 is a diagram showing a sixth arrangement pattern in each block. [Figure 19] 10 is a timing chart showing the timing of charge accumulation in the second embodiment. [Figure 20] 10A and 10B are diagrams showing a display example in which still images are displayed on the first display section and the second display section in the second embodiment. [Figure 21] FIG. 10 is a block diagram showing the configuration of an imaging device and an electronic device according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to these. In addition, in the drawings, in order to explain the embodiments, some parts are enlarged or emphasized, and the scale is appropriately changed.

[0010] First Embodiment 1 is a cross-sectional view of a stacked imaging element. This stacked imaging element 100 is described in Japanese Patent Application No. 2012-139026, previously filed by the applicant of the present application. The imaging element 100 includes an imaging chip 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes the pixel signals, and a memory chip 112 that stores the pixel signals. The imaging chip 113, signal processing chip 111, and memory chip 112 are stacked and electrically connected to each other by conductive bumps 109 made of Cu or the like.

[0011] As shown in the figure, incident light is mainly incident in the positive direction of the Z axis, as indicated by the white arrow. In this embodiment, the surface of the imaging chip 113 on which the incident light is incident is referred to as the back surface. As shown by the coordinate axes, the left direction on the paper, which is perpendicular to the Z axis, is the positive X axis, and the front direction on the paper, which is perpendicular to the Z axis and the X axis, is the positive Y axis. In the following figures, the coordinate axes are displayed so that the orientation of each figure can be understood, based on the coordinate axes in Figure 1.

[0012] An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is arranged on the back side of the wiring layer 108. The PD layer 106 has a plurality of photodiodes (hereinafter referred to as PDs) 104 arranged two-dimensionally and accumulating charges according to incident light, and transistors 105 provided corresponding to the PDs 104.

[0013] A color filter 102 is provided on the incident light side of the PD layer 106 via a passivation film 103. The color filter 102 is a filter that transmits a specific wavelength range of visible light. There are multiple types of color filters 102 that transmit different wavelength ranges, and each has a specific arrangement corresponding to the PD 104. The arrangement of the color filters 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

[0014] A microlens 101 is provided corresponding to each pixel on the incident light side of the color filter 102. The microlens 101 condenses the incident light toward the corresponding PD 104.

[0015] The wiring layer 108 has wiring 107 that transmits pixel signals from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multi-layered and may include passive and active elements. A plurality of bumps 109 are arranged on the surface of the wiring layer 108. These plurality of bumps 109 are aligned with a plurality of bumps 109 provided on the opposing surface of the signal processing chip 111. Then, by applying pressure or the like to the imaging chip 113 and the signal processing chip 111, the aligned bumps 109 are bonded to each other and electrically connected.

[0016] Similarly, a plurality of bumps 109 are arranged on the opposing surfaces of the signal processing chip 111 and the memory chip 112. These bumps 109 are aligned with each other. Then, by applying pressure to the signal processing chip 111 and the memory chip 112, the aligned bumps 109 are bonded and electrically connected to each other.

[0017] The bonding between the bumps 109 is not limited to Cu bump bonding by solid-phase diffusion, and micro-bump bonding by solder melting may also be used. For example, it is sufficient to provide one bump 109 for each unit group described below. Therefore, the size of the bumps 109 may be larger than the pitch of the PDs 104. Furthermore, in a peripheral region other than the pixel region (pixel region 113A shown in FIG. 2) where pixels are arranged, bumps larger than the bumps 109 corresponding to the pixel region may also be provided.

[0018] The signal processing chip 111 has TSVs (Through-Silicon Vias) 110 that connect circuits provided on the front and back surfaces of the chip to each other. The TSVs 110 are provided in the peripheral region. The TSVs 110 may also be provided in the peripheral region of the imaging chip 113 or in the memory chip 112.

[0019] FIG. 2 is a diagram illustrating the pixel arrangement and unit groups of an imaging chip. FIG. 2 particularly shows the imaging chip 113 as observed from the back side. The region in which pixels are arranged in the imaging chip 113 is called pixel region 113A. More than 20 million pixels are arranged in a matrix in pixel region 113A. In the example shown in FIG. 2, 16 pixels, 4 pixels by 4 pixels adjacent to each other, form one unit group 131. The grid lines in FIG. 2 show the concept of grouping adjacent pixels to form unit group 131. The number of pixels forming unit group 131 is not limited to this and may be around 1,000, for example, 32 pixels by 64 pixels, or it may be more or less than that.

[0020] As shown in the partially enlarged view of pixel region 113A, unit group 131 includes four so-called Bayer arrays, arranged vertically and horizontally, each consisting of four pixels: green pixels Gb and Gr, a blue pixel B, and a red pixel R. The green pixels are pixels that have a green filter as the color filter 102 and receive light in the green wavelength band of incident light. Similarly, the blue pixels are pixels that have a blue filter as the color filter 102 and receive light in the blue wavelength band. The red pixels are pixels that have a red filter as the color filter 102 and receive light in the red wavelength band.

[0021] 3 is a circuit diagram corresponding to a unit group of an imaging chip. In FIG. 3, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. Note that at least some of the transistors described below correspond to transistor 105 in FIG. 1.

[0022] As described above, the unit group 131 is formed of 16 pixels. The 16 PDs 104 corresponding to the respective pixels are connected to transfer transistors 302. The gates of the transfer transistors 302 are connected to TX wiring 307 to which transfer pulses are supplied. In this embodiment, the TX wiring 307 is commonly connected to the 16 transfer transistors 302.

[0023] The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD (charge detection unit) between the drain of the transfer transistor 302 and the source of each reset transistor 303 is connected to the gate of the amplification transistor 304. The drain of each reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The gate of each reset transistor 303 is connected to a reset wiring 306 to which a reset pulse is supplied. In this embodiment, the reset wiring 306 is commonly connected to 16 reset transistors 303.

[0024] The drain of each amplification transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of a corresponding selection transistor 305. The gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In this embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The sources of each selection transistor 305 are connected to a common output wiring 309. A load current source 311 supplies a current to the output wiring 309. In other words, the output wiring 309 for the selection transistor 305 is formed by a source follower. The load current source 311 may be provided on the imaging chip 113 side or on the signal processing chip 111 side.

[0025] Here, we will explain the flow from the start of charge accumulation to pixel output after accumulation ends. A reset pulse is applied to the reset transistor 303 through the reset wiring 306. At the same time, a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307. This resets the potentials of the PD 104 and the floating diffusion FD.

[0026] When the application of the transfer pulse is stopped, the PD 104 converts the incident light it receives into electric charges and accumulates them. Then, when the transfer pulse is applied again without the reset pulse being applied, the electric charges accumulated in the PD 104 are transferred to the floating diffusion FD. As a result, the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. When a selection pulse is applied to the selection transistor 305 via the decoder wiring 308, the fluctuation in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 via the amplification transistor 304 and the selection transistor 305. Through this circuit operation, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

[0027] As shown in FIG. 3 , in this embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels that form the unit group 131. That is, the reset pulse and the transfer pulse are each applied to all 16 pixels simultaneously. Therefore, all pixels that form the unit group 131 start and end charge accumulation at the same timing. However, pixel signals corresponding to the accumulated charges are selectively output to the output wiring 309 by sequentially applying selection pulses to the respective selection transistors 305. Furthermore, the reset wiring 306, the TX wiring 307, and the output wiring 309 are provided separately for each unit group 131.

[0028] In this way, by configuring the circuit based on the unit group 131, it is possible to control the charge accumulation time for each unit group 131. In other words, it is possible to output pixel signals with different charge accumulation times between the unit groups 131. Furthermore, by causing one unit group 131 to perform a single charge accumulation while causing the other unit group 131 to repeatedly accumulate charge and output a pixel signal each time, it is also possible to output frames for a moving image at different frame rates between these unit groups 131.

[0029] 4 is a block diagram showing the functional configuration of the image sensor. An analog multiplexer 411 sequentially selects the 16 PDs 104 that form the unit group 131. The multiplexer 411 then outputs pixel signals from the 16 PDs 104 to the output wiring 309 provided corresponding to the unit group 131. The multiplexer 411 is formed on the image sensor chip 113 together with the PDs 104.

[0030] The analog pixel signals output via the multiplexer 411 are amplified by an amplifier 412 formed in the signal processing chip 111. The pixel signals amplified by the amplifier 412 are then subjected to correlated double sampling (CDS) signal processing and A / D conversion (conversion from analog signals to digital signals) by a signal processing circuit 413 formed in the signal processing chip 111 that performs correlated double sampling (CDS) and analog / digital conversion. The pixel signals are subjected to correlated double sampling signal processing in the signal processing circuit 413, thereby reducing noise in the pixel signals. The A / D converted pixel signals are passed to a demultiplexer 414 and stored in pixel memories 415 corresponding to the respective pixels. The demultiplexer 414 and pixel memories 415 are formed in the memory chip 112.

[0031] The arithmetic circuit 416 processes the pixel signals stored in the pixel memory 415 and passes them to a downstream image processing unit. The arithmetic circuit 416 may be provided in the signal processing chip 111 or in the memory chip 112. Note that while FIG. 4 shows connections for one unit group 131, in reality, these exist for each unit group 131 and operate in parallel. However, it is not necessary for there to be a arithmetic circuit 416 for each unit group 131. For example, one arithmetic circuit 416 may perform sequential processing while referring to the values ​​of the pixel memories 415 corresponding to each unit group 131 in order.

[0032] As described above, output wiring 309 is provided corresponding to each unit group 131. The imaging element 100 is formed by stacking the imaging chip 113, the signal processing chip 111, and the memory chip 112. Therefore, by using the bumps 109 for electrical connection between the chips for these output wiring 309, it is possible to route the wiring without increasing the size of each chip in the planar direction.

[0033] Next, blocks set in the pixel region 113A (see FIG. 2) of the image sensor 100 will be described. In this embodiment, the pixel region 113A of the image sensor 100 is divided into multiple blocks. Each of the multiple blocks is defined to include at least one unit group 131. The pixels included in each block are controlled using different control parameters. That is, pixel signals obtained using different control parameters are obtained for a pixel group included in one block and a pixel group included in another block. Examples of control parameters include the charge accumulation time or number of accumulations, frame rate, gain, thinning rate, number of rows or columns for adding pixel signals, and number of digitization bits. Furthermore, the control parameters may be parameters for image processing after image signals are obtained from the pixels.

[0034] Here, the charge accumulation time refers to the time from when the PD 104 starts accumulating charge until it finishes. The number of charge accumulations refers to the number of times the PD 104 accumulates charge per unit time. The frame rate refers to a value that represents the number of frames processed (displayed or recorded) in a video per unit time. The frame rate is expressed in fps (Frames Per Second). The higher the frame rate, the smoother the movement of the subject (i.e., the object being imaged) in the video.

[0035] Gain refers to the gain factor (amplification factor) of the amplifier 412. Changing this gain can change the ISO sensitivity. ISO sensitivity is a standard for photographic film established by ISO and indicates how weak light a photographic film can record. However, ISO sensitivity is also generally used to express the sensitivity of the image sensor 100. In this case, ISO sensitivity is a value that represents the image sensor 100's ability to capture light. Increasing the gain also improves the ISO sensitivity. For example, doubling the gain doubles the electrical signal (pixel signal), resulting in appropriate brightness even with half the amount of incident light. However, increasing the gain also amplifies noise contained in the electrical signal, resulting in increased noise.

[0036] The thinning rate refers to the ratio of pixels whose pixel signals are not read out to the total number of pixels in a given region. For example, a thinning rate of 0 for a given region means that pixel signals are read out from all pixels in that given region. A thinning rate of 0.5 for a given region means that pixel signals are read out from half of the pixels in that given region. Specifically, if the unit group 131 is a Bayer array, pixels whose pixel signals are read out and pixels whose pixel signals are not read out are set alternately every other Bayer array unit in the vertical direction, i.e., every two pixels (every two rows) of pixel units. Note that thinning out pixel signal readout reduces image resolution. However, since the image sensor 100 has more than 20 million pixels, even if thinning is performed at a thinning rate of 0.5, an image can be displayed with more than 10 million pixels. For this reason, the reduction in resolution is unlikely to be noticeable to the user (photographer).

[0037] The number of summed rows refers to the number of vertical pixels (rows) to be added when pixel signals of vertically adjacent pixels are added. The number of summed columns refers to the number of horizontal pixels (columns) to be added when pixel signals of horizontally adjacent pixels are added. Such addition processing is performed, for example, by the arithmetic circuit 416. By performing processing in which the arithmetic circuit 416 adds pixel signals of a predetermined number of vertically or horizontally adjacent pixels, an effect similar to processing in which pixel signals are read out after thinning at a predetermined thinning rate is achieved. Note that in the above-mentioned addition processing, the arithmetic circuit 416 may calculate an average value by dividing the sum by the number of rows or columns added.

[0038] The number of digitization bits refers to the number of bits used when an analog signal is converted into a digital signal in A / D conversion by the signal processing circuit 413. The greater the number of bits in the digital signal, the more detailed the brightness, color changes, and the like are expressed.

[0039] In this embodiment, the accumulation conditions refer to conditions related to the accumulation of charge in the image sensor 100. Specifically, the accumulation conditions refer to the charge accumulation time or number of accumulations, frame rate, and gain among the control parameters described above. Since the frame rate can change depending on the charge accumulation time or number of accumulations, the frame rate is included in the accumulation conditions. Furthermore, the amount of light for the appropriate exposure changes depending on the gain, and the charge accumulation time or number of accumulations can also change depending on the amount of light for the appropriate exposure. For this reason, the gain is included in the accumulation conditions.

[0040] Furthermore, the imaging conditions refer to conditions related to imaging of a subject. Specifically, the imaging conditions refer to control parameters including the accumulation conditions described above. The imaging conditions include control parameters for controlling the image sensor 100 (e.g., charge accumulation time or accumulation count, frame rate, gain), as well as control parameters for controlling readout of signals from the image sensor 100 (e.g., thinning rate) and control parameters for processing signals from the image sensor 100 (e.g., the number of rows or columns to be added together for adding pixel signals, the number of digitization bits, and control parameters for image processing by the image processing unit 30, described later).

[0041] FIG. 5 is a block diagram showing the configuration of an electronic device according to a first embodiment. The electronic device 1 shown in FIG. 5 is configured as, for example, a device with an imaging function, such as a digital camera, a smartphone, a mobile phone, or a personal computer. As shown in FIG. 5, the electronic device 1 includes a lens unit 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, an operation unit 55, a recording unit 60, and a system control unit 70. The lens unit 10 is an imaging optical system composed of a plurality of lens groups. The lens unit 10 guides a light beam from a subject to the imaging unit 20. The lens unit 10 may be integrated with the electronic device 1, or may be an interchangeable lens that is detachable from the electronic device 1. The lens unit 10 may also include a built-in focus lens or a built-in zoom lens.

[0042] The imaging unit 20 includes an image sensor 100 and a driver 21. The driver 21 is a control circuit that controls the driving of the image sensor 100 in accordance with instructions from the system controller 70. The driver 21 controls the timing (or timing cycle) of applying a reset pulse and a transfer pulse to the reset transistor 303 and the transfer transistor 302, respectively, thereby controlling the charge accumulation time or number of accumulations, which are control parameters. The driver 21 also controls the frame rate by controlling the timing (or timing cycle) of applying a reset pulse, a transfer pulse, and a selection pulse to the reset transistor 303, the transfer transistor 302, and the selection transistor 305, respectively. The driver 21 also controls the thinning rate by setting the pixels to which the reset pulse, the transfer pulse, and the selection pulse are applied.

[0043] The driver 21 also controls the ISO sensitivity of the image sensor 100 by controlling the gain (also referred to as the gain rate or amplification rate) of the amplifier 412. The driver 21 also sets the number of rows or columns to be added together for pixel signal addition by sending instructions to the arithmetic circuit 416. The driver 21 also sets the number of digitization bits by sending instructions to the signal processing circuit 413. The driver 21 also sets blocks in the pixel region (imaging region) 113A of the image sensor 100. In this way, the driver 21 functions as an image sensor control unit that causes the image sensor 100 to capture images under different imaging conditions for each of a plurality of blocks and output pixel signals. The system control unit 70 gives instructions to the driver 21 regarding the position, shape, range, etc. of the blocks.

[0044] The image sensor 100 passes pixel signals from the image sensor 100 to the image processing unit 30. The image processing unit 30 uses the work memory 40 as a workspace to perform various image processes on RAW data consisting of pixel signals for each pixel, generating image data. The image processing unit 30 is provided with a first image processing unit 30A and a second image processing unit 30B. When the load of image processing is high, processing is assigned to the first image processing unit 30A and the second image processing unit 30B, respectively. The first image processing unit 30A and the second image processing unit 30B then execute the assigned processing in parallel.

[0045] In this embodiment, as will be described later, the system control unit 70 (specifically, the dividing unit 71 shown in FIG. 7) divides the pixel region (imaging region) 113A of the image sensor 100 into at least a first region and a second region. Furthermore, the system control unit 70 (specifically, the drive control unit 72 shown in FIG. 7) controls and drives the image sensor 100 so that images are captured under different imaging conditions for the first region and the second region. In this case, for example, the first image processing unit 30A performs image processing on signals from the first region. Furthermore, the second image processing unit 30B performs image processing on signals from the second region. Note that the pixel region (imaging region) 113A of the image sensor 100 is not limited to being divided into two regions, the first region and the second region, but may also be divided into multiple regions, such as the first region, the second region, the third region, and so on. Then, image processing for the multiple regions is appropriately assigned to the first image processing unit 30A and the second image processing unit 30B. The allocation of image processing may be determined in advance based on the number and range of divided regions, or may be determined by the system control unit 70 based on the number and range of divided regions.

[0046] The image processing unit 30 executes various types of image processing. For example, the image processing unit 30 generates an RGB image signal by performing color signal processing (color correction) on signals obtained from the Bayer array. The image processing unit 30 also performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. The image processing unit 30 also performs compression processing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The image processing unit 30 outputs the generated image data to the recording unit 60. The image processing unit 30 also outputs the generated image data to the display unit 50.

[0047] In this embodiment, in addition to the above-described processes, the image processing unit 30 also performs a process of detecting a main subject from image data. Here, the "main subject" refers to a subject that is the object of imaging and that is noticed by the user (photographer) or that is presumed to be noticed by the user. There may be not only one main subject in the image data, but also multiple main subjects (see, for example, FIG. 14).

[0048] The control parameters (imaging conditions) also include parameters referenced when the image processing unit 30 performs image processing. For example, the control parameters include parameters such as color signal processing (color correction), white balance adjustment, gradation adjustment, and compression rate. The signal read from the image sensor 100 changes depending on the charge accumulation time, and the parameters referenced when performing image processing also change depending on the change in the signal. The image processing unit 30 sets different control parameters for each block and performs image processing such as color signal processing based on these control parameters.

[0049] The image processing unit 30 extracts frames at predetermined intervals from among the multiple frames obtained in time series from the imaging unit 20. Alternatively, the image processing unit 30 discards frames obtained in time series from the imaging unit 20 at predetermined intervals. This reduces the amount of data, thereby reducing the load on downstream processing. The image processing unit 30 also calculates one or more frames to be interpolated between the multiple frames obtained in time series from the imaging unit 20. The image processing unit 30 then adds the calculated one or more frames between the frames. This allows for smoother motion during video playback. Although the driving unit 21 is configured to control the thinning rate, this configuration is not limited thereto. For example, the driving unit 21 may read pixel signals from all pixels, but the image processing unit 30 or the arithmetic circuit 416 may control the thinning rate by discarding certain pixel signals from the read pixel signals.

[0050] The work memory 40 temporarily stores image data and the like when image processing is performed by the image processing unit 30. The display unit 50 is configured, for example, with a liquid crystal display panel. As shown in Fig. 5, the display unit 50 has a first display unit 51, a first touch panel 52, a second display unit 53, and a second touch panel 54.

[0051] The first display unit 51 displays images (still images, videos, live view images) captured by the imaging unit 20 and various types of information. The first touch panel 52 is formed on the display screen of the first display unit 51. When the user selects an image (thumbnail images, which will be described later; see FIG. 13 ), the first touch panel 52 outputs a signal indicating the position touched by the user to the system control unit 70. The second display unit 53 displays images (still images, videos, live view images) captured by the imaging unit 20 and various types of information. The second touch panel 54 is formed on the display screen of the second display unit 53. When the user selects an image, the second touch panel 54 outputs a signal indicating the position touched by the user to the system control unit 70.

[0052] The operation unit 55 includes a release switch, a video switch, and various other operation switches operated by the user. The operation unit 55 outputs signals to the system control unit 70 in response to user operations. The recording unit 60 has two card slots into which two storage media (a first recording medium 61 and a second recording medium 62) such as memory cards can be inserted. The recording unit 60 stores image data and various other data generated by the image processing unit 30 in the recording media (the first recording medium 61 and the second recording medium 62) inserted into the card slots. In this embodiment, as described above, the first image processing unit 30A and the second image processing unit 30B each perform image processing based on signals from the first area and image processing based on signals from the second area in parallel. In this case, the first recording medium 61 stores image data based on signals from the first area in response to operation of the release switch or video switch. The second recording medium 62 stores image data based on signals from the second area in response to operation of the release switch or video switch. The recording unit 60 also has an internal memory. The recording unit 60 can also record the image data and various other data generated by the image processing unit 30 in an internal memory.

[0053] The system control unit 70 controls the overall processing and operation of the electronic device 1. The system control unit 70 includes a CPU (Central Processing Unit) 70A. In this embodiment, the system control unit 70 divides the imaging surface (pixel region 113A) of the imaging element 100 (imaging chip 113) into multiple blocks and acquires images using different charge accumulation times (or charge accumulation counts), frame rates, and gains for the blocks. To this end, the system control unit 70 instructs the driver 21 on the position, shape, and range of the blocks, as well as the accumulation conditions for each block. The system control unit 70 also acquires images using different thinning rates, numbers of rows or columns for adding pixel signals, and digitization bit rates for the blocks. To this end, the system control unit 70 instructs the driver 21 on the imaging conditions for each block (thinning rate, number of rows or columns for adding pixel signals, and digitization bit rate). The image processing unit 30 also performs image processing using different imaging conditions for the blocks (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate). For this reason, the system control unit 70 instructs the image processing unit 30 on the imaging conditions for each block (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

[0054] The system control unit 70 also records the image data generated in the image processing unit 30 in the recording unit 60. The system control unit 70 also outputs the image data generated in the image processing unit 30 to the display unit 50, thereby displaying an image on the display unit 50 (either or both of the first display unit 51 and the touch panel 52). Alternatively, the system control unit 70 reads out image data recorded in the recording unit 60 and outputs it to the display unit 50, thereby displaying an image on the display unit 50 (either or both of the first display unit 51 and the touch panel 52). Images displayed on the display unit 50 include still images, videos, and live view images. Here, a live view image is an image displayed on the display unit 50 by sequentially outputting image data generated by the image processing unit 30 to the display unit 50. The live view image is used by the user to check an image of a subject being captured by the imaging unit 20. A live view image is also called a through image or a preview image.

[0055] FIG. 6 is a diagram showing the appearance of a digital camera, which is an example of an electronic device. FIG. 6 shows the appearance of the electronic device (digital camera) 1 as viewed from the back. As shown in FIG. 6, the first display unit 51 is a display panel having a rectangular display screen. The first display unit 51 is provided on the back of the electronic device 1. The first touch panel 52 is formed on the display screen of the first display unit 51.

[0056] The second display unit 53 is a display panel having a rectangular display screen. An end of the second display unit 53 is rotatably connected to the first display unit 51 by a hinge (not shown) provided on the rear surface of the electronic device 1 and below the first display unit 51. The first display unit 51 is opened and closed by the second display unit 53 as the second display unit 53 rotates around the hinge.

[0057] The top surface of the electronic device 1 is provided with a release switch 55a, a mode dial 55b, and a video switch 55c. The release switch 55a is a switch pressed by the user when capturing a still image. Note that half-pressing the release switch 55a initiates shooting preparations such as AF (Automatic Focusing) and AE (Automatic Exposure). The mode dial 55b is a dial rotated by the user when setting various scene modes such as portrait, landscape, and night scene. The video switch 55c is a switch pressed by the user when capturing a video. A multi-selector 55d is provided on the back surface of the electronic device 1 and to the side of the first display unit 51. The multi-selector 55d is a key and switch that allows the user to select a menu (a menu for setting a shooting mode) displayed on the first display unit 51 or the second display unit 53 using up / down / left / right arrow keys and an OK switch. Note that the operation unit 55 has the release switch 55a, the mode dial 55b, the video switch 55c, and the multi-selector 55d. The operation unit 55 may also have other switches and the like.

[0058] FIG. 7 is a functional block diagram of the image processing unit and system control unit shown in FIG. 5. As shown in FIG. 7, the first image processing unit 30A includes an image generation unit 31A and a detection unit 32A. The image generation unit 31A generates image data by performing various image processes on RAW data consisting of pixel signals of each pixel in the first region output from the imaging unit 20. The detection unit 32A detects a main subject from the image data generated by the image generation unit 31A. In this embodiment, the detection unit 32A compares multiple image data obtained in time series from live view images generated by the image generation unit 31A and detects a moving subject (moving subject) as the main subject. The detection unit 32A detects the main subject using a face detection function such as that described in Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940). In addition to face detection, the detection unit 32A also detects a human body included in image data as a main subject, as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940).

[0059] The second image processing unit 30B also includes an image generation unit 31B. The image generation unit 31B generates image data by performing various image processes on RAW data consisting of pixel signals from each pixel in the second region output from the imaging unit 20. The second image processing unit 30B does not include a detection unit, but may include a detection unit. Alternatively, the first image processing unit 30A may not include the detection unit 32A, and the second image processing unit 30B may include a detection unit. In this embodiment, the image generation unit 31A and the image generation unit 31B may be collectively referred to as the image generation unit 31.

[0060] The system control unit 70 also includes a dividing unit 71, a drive control unit 72, and a display control unit 73. The dividing unit 71 divides the pixel region (imaging region) 113A of the image sensor 100 into multiple regions on a block-by-block basis. The dividing unit 71 divides the pixel region 113A into multiple regions based on a predetermined arrangement pattern in each block of the pixel region 113A (see FIGS. 8A to 8D). The drive control unit 72 sets imaging conditions for the multiple regions. The drive control unit 72 also controls the drive of the image sensor 100 in response to the user's operation of the release switch 55a or the video switch 55c. The drive control unit also controls the drive of the image sensor 100 when capturing a live view image (i.e., after the start of the shooting operation after power-on). The display control unit 73 controls the display unit 50 to output image data generated by the image generation unit 31 and to display an image (a still image, a video, or a live view image) on either or both of the first display unit 51 and the second display unit 53.

[0061] In the system control unit 70, the dividing unit 71, the drive control unit 72, and the display control unit 73 are realized by the CPU 70A executing processes based on a control program.

[0062] Next, the arrangement pattern for each block set by the dividing unit 71 will be described. Fig. 8 is a diagram showing the arrangement pattern for each block. Here, Fig. 8(A) shows the first arrangement pattern for each block. Fig. 8(B) shows the second arrangement pattern for each block. Fig. 8(C) shows the third arrangement pattern for each block. Fig. 8(D) shows the fourth arrangement pattern for each block.

[0063] 8A is an arrangement pattern in which the pixel region 113A is divided into a first region and a second region. In this first arrangement pattern, in the pixel region 113A, the first region is made up of blocks of odd-numbered columns (2m-1), and the second region is made up of blocks of even-numbered columns (2m). That is, each block in the pixel region 113A is divided into odd-numbered columns and even-numbered columns. m is a positive integer (m=1, 2, 3, . . .).

[0064] 8(B) is also an arrangement pattern in which the pixel region 113A is divided into a first region and a second region. In this second arrangement pattern, in the pixel region 113A, the first region is made up of blocks of odd-numbered rows (2n-1), and the second region is made up of blocks of even-numbered rows (2n). That is, each block in the pixel region 113A is divided into odd-numbered rows and even-numbered rows. n is a positive integer (n=1, 2, 3, . . .).

[0065] The third arrangement pattern shown in FIG. 8C is also an arrangement pattern in which the pixel region 113A is divided into a first region and a second region. In this third arrangement pattern, the first region is composed of blocks in odd-numbered rows (2n-1) in odd-numbered columns (2m-1) and blocks in even-numbered rows (2n) in even-numbered columns (2m). The second region is composed of blocks in odd-numbered rows (2n-1) in even-numbered columns (2m) and blocks in even-numbered rows (2n-1) in odd-numbered columns (2m-1). That is, the blocks in the pixel region 113A are divided into a checkerboard pattern. m and n are positive integers (m=1, 2, 3, . . ., n=1, 2, 3, . . .). As shown in FIGS. 8A to 8C, in this embodiment, the first region and the second region are not necessarily composed of only contiguous blocks, but are composed of discrete blocks.

[0066] 8(D), pixel region 113A is divided into three regions: a first region, a second region, and a third region. In this fourth arrangement pattern, in pixel region 113A, the first region is made up of blocks in the column two columns before the column that is a multiple of 3 (3m-2), the second region is made up of blocks in the column one column before the column that is a multiple of 3 (3m-1), and the third region is made up of blocks in the column that is a multiple of 3 (3m). m is a positive integer (m=1, 2, 3, . . .).

[0067] In FIG. 8, a small number of blocks are set in pixel area 113A to make the arrangement of blocks in each area easier to see, but a larger number of blocks than the number of blocks shown in FIG. 8 may also be set.

[0068] Next, a shooting operation according to the first embodiment will be described. FIG. 9 is a flowchart illustrating the shooting operation executed by the system control unit. FIG. 10 is a flowchart illustrating an array pattern setting process. In the process shown in FIG. 9, when the electronic device 1 is powered on, the system control unit 70 starts the shooting operation. Although not shown in FIG. 9, when shooting starts, the display control unit 73 displays a live-view image captured by the imaging unit 20 on the first display unit 51 and a menu for setting a shooting mode on the second display unit 53. At this time, since the live-view image does not need to be an image showing smooth subject movement, the drive control unit 72 controls the driving of the image sensor 100 so as to capture the image at a low frame rate. Note that the display control unit 73 may display the live-view image on the second display unit 53 and the menu on the first display unit 51. Alternatively, the display control unit 73 may display the live-view image and the menu on the same display unit (the first display unit 51 or the second display unit 53).

[0069] The user operates multi-selector 55d to select a shooting mode by selecting from the menu displayed on second display unit 53. Splitting unit 71 confirms the shooting mode selected by the user's operation of multi-selector 55d (step S1).

[0070] Here, the shooting modes include a still image mode for shooting still images and a video mode for shooting videos. The still image modes include a first still image mode and a second still image mode. The video modes include a first video mode and a second video mode.

[0071] The first still image mode refers to a shooting mode in which the dividing unit 71 does not divide the pixel region (imaging region) 113A of the image sensor 100, and the image sensor 100 captures a still image of the subject using the pixel region 113A as a single region. This first still image mode is a commonly used normal still image shooting mode. The second still image mode refers to a shooting mode in which the dividing unit 71 divides the pixel region 113A into multiple regions, and the image sensor 100 captures a still image of the same subject in each of the multiple regions. In this second still image mode, the process of continuously capturing still images of the same subject in each of the multiple regions can be executed in parallel. Therefore, when performing continuous shooting of still images in the second still image mode, more still images can be captured per unit time than when performing continuous shooting in the first still image mode. In other words, the second still image mode allows for faster continuous shooting than the first still image mode. The second still image mode is also referred to as a high-speed continuous shooting mode or a still-still-image mixed mode.

[0072] The first moving image mode refers to a shooting mode in which the dividing unit 71 does not divide the pixel region (imaging region) 113A of the image sensor 100, and the image sensor 100 shoots a moving image of the subject using the pixel region 113A as a single region. This first moving image mode is a commonly used normal moving image shooting mode. The second moving image mode refers to a shooting mode in which the dividing unit 71 divides the pixel region 113A into multiple regions, and the image sensor 100 shoots a still image of the subject in one of the multiple regions and shoots a moving image of the same subject in another of the multiple regions. The second moving image mode is also called a still image-moving image simultaneous shooting mode or a still image-moving image mixed mode.

[0073] When the user selects a shooting mode, the user may touch a position on the second touch panel 54 corresponding to a menu, instead of operating the multi-selector 55d.

[0074] The division unit 71 determines whether the shooting mode selected by the user is the still image mode (step S2). If the division unit 71 determines that the shooting mode is the still image mode, it determines whether the still image mode is the first still image mode (step S3). If the division unit 71 determines that the still image mode is the first still image mode, it performs a process of setting the shooting mode to the first still image mode (step S4). On the other hand, if the division unit 71 determines that the still image mode is not the first still image mode, that is, if it determines that the still image mode is the second still image mode, it performs a process of setting the shooting mode to the second still image mode (step S5).

[0075] In the processing of step S4 or step S5, the division unit 71 executes the array pattern setting processing shown in Fig. 10. In the processing shown in Fig. 10, the division unit 71 instructs the image processing unit 30 (first image processing unit 30A) to detect a main subject (step S21). The detection unit 32A compares multiple image data obtained in time series from live view images to detect moving subjects and non-moving subjects (subjects that are not moving). The detection unit 32A then outputs the detection result together with the image data to the system control unit 70. The division unit 71 confirms the presence or absence of a main subject based on the detection result of the detection unit 32A. The division unit 71 then sets an area in the pixel area 113A according to the main subject and the shooting mode (step S22).

[0076] Specifically, when the shooting mode is the first still image mode, the dividing unit 71 does not divide the pixel region 113A into multiple regions. That is, the dividing unit 71 sets all the pixel regions 113A as one region. At this time, the dividing unit 71 outputs an instruction signal to the driving unit 21 to instruct the driving unit 21 to set all the pixel regions 113A as one region.

[0077] On the other hand, when the shooting mode is the second still image mode, the division unit 71 selects one of the arrangement patterns shown in FIGS. 8A to 8D. Then, the division unit 71 checks whether the main subject is a moving subject based on the detection result of the detection unit 32A. If the main subject is not a moving subject but a stationary subject, the division unit 71 sets the first region and the second region according to the third arrangement pattern shown in FIG. 8C. If the main subject is a moving subject, the division unit 71 checks the direction of movement of the moving subject. If the moving subject moves mainly up and down, for example, if the main subject is a child sliding down a slide or a waterfall, the division unit 71 sets the first region and the second region according to the first arrangement pattern shown in FIG. 8A. If the moving subject moves mainly left and right, for example, if the main subject is a running person or a panning shot, the division unit 71 sets the first region and the second region according to the second arrangement pattern shown in FIG. 8B. Furthermore, if the moving subject moves at a high speed in the vertical direction, the dividing unit 71 sets the first, second, and third regions according to the fourth arrangement pattern shown in Fig. 8(D). In step S22, the dividing unit 71 outputs an instruction signal to the driving unit 21 to instruct the position of the blocks in each region (the first and second regions, and the first to third regions).

[0078] FIG. 11 is a diagram showing an example of setting the second arrangement pattern when the second still image mode is executed. Note that in FIG. 11, each block is depicted enlarged to make the block arrangement easier to see. However, in reality, blocks smaller than the size of the blocks shown in FIG. 11 are set in pixel region 113A. In the example shown in FIG. 11, detection unit 32A detects people O1 and O2 playing soccer and a soccer ball O3 as main subjects (moving subjects). Based on the detection result of detection unit 32A, division unit 71 determines that main subjects O1 to O3 are moving subjects and that the moving subjects move mainly in the left-right direction. As a result, division unit 71 sets first and second regions according to the second arrangement pattern shown in FIG. 8(B). At this time, the division unit divides pixel region 113A so that the first and second regions include main subjects O1 to O3.

[0079] Returning to the description of FIG. 10, the drive control unit 72 sets the imaging conditions for the areas set in step S22 (the first area and the second area in the example shown in FIG. 11) based on the detection result of the detection unit 32A (step S23). Specifically, the drive control unit 72 outputs an instruction signal to the drive unit 21 indicating imaging conditions (charge accumulation time, gain, etc.) according to the main subject. The drive control unit 72 also outputs an instruction signal to the image processing unit 30 indicating imaging conditions (parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate) according to the main subject. For example, when a moving subject is detected by the detection unit 32A, the drive control unit 72 increases the gain (ISO sensitivity) and increases the charge accumulation time (i.e., exposure time and shutter speed). When a moving subject is not detected by the detection unit 32A, the drive control unit 72 decreases the gain and decreases the charge accumulation time.

[0080] 9, the drive control unit 72 determines whether the user has operated the release switch 55a (a half-press followed by a full press) (step S6). If the drive control unit 72 determines that the release switch 55a has been operated, it causes the imaging unit 20 to capture an image in a still image mode (first still image mode or second still image mode) (step S7).

[0081] FIG. 12 is a timing chart showing the timing of charge accumulation in the first still image mode and the second still image mode. In the first still image mode (normal continuous shooting mode) shown in FIG. 12(A), all pixel regions 113A are set as a single region, as described above. In the first still image mode, the drive control unit 72 outputs an instruction signal to the drive unit 21 to repeatedly capture still images for all pixel regions 113A while the release switch 55a is being operated (fully pressed). In FIG. 12(A), the drive unit 21 starts charge accumulation for the pixel region 113A at time t1 and ends charge accumulation for the pixel region 113A at time t3. The drive unit 21 reads pixel signals from each pixel in the pixel region 113A and resets the charge accumulated in each pixel. Thereafter, the drive unit 21 starts charge accumulation for the pixel region 113A at time t4 and ends charge accumulation for the pixel region 113A at time t7. The driving unit 21 repeatedly executes such driving control of the image pickup device 100 while the release switch 55a is being operated.

[0082] In the example shown in FIG. 12A, the driving unit 21 captures images four times in succession between times t1 and t15. The periods from times t1 to t3, t4 to t7, t8 to t11, and t12 to t15 are charge accumulation periods (exposure periods). These charge accumulation periods (exposure periods) are set in the imaging condition setting process of step S23. Furthermore, the pixel signals of each pixel read from the image sensor 100 are amplified by the amplifier 412 with a gain specified by the division unit 71, and then output to the image processing unit 30. The image generating unit 31 (e.g., image generating unit 31A) checks parameters used in image processing such as color signal processing based on the instruction signal specifying the imaging conditions output from the division unit 71. The image generating unit 31 then generates image data by performing various image processing operations based on the parameters on the RAW data consisting of the pixel signals of each pixel.

[0083] In the second still image mode (high-speed continuous shooting mode) shown in FIG. 12B, the dividing unit 71 sets, for example, a first region and a second region. In the second still image mode, the drive control unit 72 outputs an instruction signal to the drive unit 21 to repeatedly capture still images for the first region and repeatedly capture still images for the second region while the release switch 55a is being operated (fully pressed). In FIG. 12B, the drive unit 21 starts charge accumulation for the first region at time t1 and ends charge accumulation for the first region at time t3. The drive unit 21 reads pixel signals from each pixel in the first region and resets the charge accumulated in each pixel. Thereafter, the drive unit 21 starts charge accumulation for the first region at time t4 and ends charge accumulation for the first region at time t7. The drive unit 21 repeatedly performs this drive control of the image sensor 100 while the release switch 55a is being operated.

[0084] 12(B), the driver 21 starts charge accumulation in the second region at time t2 and ends charge accumulation in the second region at time t5. The driver 21 reads pixel signals from each pixel in the second region and resets the charge accumulated in each pixel. Thereafter, the driver 21 starts charge accumulation in the second region at time t6 and ends charge accumulation in the second region at time t9. The driver 21 repeatedly performs this drive control of the image sensor 100 while the release switch 55a is being operated.

[0085] 12(B), the driving unit 21 performs four consecutive imaging operations in the first region between times t1 and t15. In parallel with the four imaging operations in the first region, the driving unit 21 performs four consecutive imaging operations in the second region between times t2 and t16. Therefore, the driving unit 21 performs eight consecutive imaging operations in the first and second regions between times t1 and t16.

[0086] The time from time t1 to t3, the time from time t4 to t7, the time from time t8 to t11, and the time from time t12 to t15 are charge accumulation times (exposure times) in the first region. These charge accumulation times (exposure times) are set in the imaging condition setting process of step S23. Furthermore, the time from time t2 to t5, the time from time t6 to t9, the time from time t10 to t13, and the time from time t14 to t16 are charge accumulation times (exposure times) in the second region. These charge accumulation times (exposure times) are also set in the imaging condition setting process of step S23.

[0087] Furthermore, the pixel signals of each pixel read out from the first region of the image sensor 100 are amplified in the amplifier 412 by the gain specified by the division unit 71, and then output to the image processing unit 30. The image generation unit 31A checks parameters used in image processing such as color signal processing, based on the instruction signal specifying the imaging conditions for the first region output from the division unit 71. The image generation unit 31A then generates image data for the first region by performing various image processes on the RAW data made up of the pixel signals of each pixel in the first region, based on the parameters.

[0088] Furthermore, the pixel signals of each pixel read from the second region of the image sensor 100 are amplified in the amplifier 412 by a gain specified by the division unit 71, and then output to the image processing unit 30. The image generation unit 31B checks parameters used in image processing such as color signal processing based on the instruction signal specifying the imaging conditions for the second region output from the division unit 71. The image generation unit 31B then generates image data for the second region by performing various image processes on RAW data consisting of pixel signals of each pixel in the second region based on the parameters. Furthermore, the image generation unit 31 (image synthesis unit 31A or 31B) synthesizes the image data of the first region and the image data of the second region.

[0089] In the second still image mode shown in FIG. 12(B), the drive control unit 72 controls the drive unit 21 to drive the image sensor 100 by differentiating (shifting) the timing at which imaging of the first region begins from the timing at which imaging of the second region begins. Therefore, if, for example, 30 still images can be captured per second in the first still image mode shown in FIG. 12(A), then in the second still image mode shown in FIG. 12(B), nearly 60 still images can be captured per second, almost double the rate of the first still image mode. In this case, because the pixel region 113A is divided into the first region and the second region, the number of pixels in the still image is halved. However, even if half of 20 million pixels is used, a 10-megapixel still image can still be captured. Therefore, it is believed that sufficient image quality is ensured for the user.

[0090] In the second still image mode, when the dividing unit 71 divides the pixel area 113A into the first to third areas according to the fourth arrangement pattern shown in FIG. 8(D), higher-speed continuous shooting can be achieved than when the pixel area 113A is divided into the first and second areas. For example, if 30 still images can be captured per second in the first still image mode, when three areas are divided into the pixel area 113A in the second still image mode, nearly 90 frames can be captured per second, which is almost three times faster than in the first still image mode. It is expected that the image sensor 100 will continue to have higher pixel count. For this reason, even if the pixel area 113A is divided into three areas and the number of pixels per still image is reduced to one-third, users place a higher priority on high-speed continuous shooting than on a decrease in image quality.

[0091] Returning to the explanation of FIG. 9 , the display control unit 73 outputs the image data generated by the image processing unit 30 to the display unit 50, thereby displaying a still image on the first display unit 51 or the second display unit 53 (step S8). When the drive control unit 72 captures a still image in the first still image mode, the display control unit 73 displays the still image on the first display unit 51. On the other hand, when the drive control unit 72 captures a still image in the second still image mode, the display control unit 73 displays multiple still images captured multiple times in rapid succession as thumbnail images on the first display unit 51. In addition, the display control unit 73 enlarges and displays a thumbnail image selected by the user on the second display unit 53.

[0092] Fig. 13 is a diagram showing a display example in which still images are displayed on the first display unit and the second display unit. As shown in Fig. 13, the display control unit 73 displays eight thumbnail images (still images) 511 to 518 side by side on the first display unit 51. For example, the thumbnail image 511 is the still image captured the first time in Fig. 12(B). Similarly, the thumbnail images 512 to 518 are still images captured the second to eighth times in Fig. 12(B), respectively.

[0093] 13, eight touch areas 511a to 518a are formed on the first touch panel 52 so as to overlap with the eight thumbnail images 511 to 518, respectively. When each of the touch areas 511a to 518a detects that it has been pressed (touched) by the user, it outputs a detection signal indicating the pressed position (which touch area) to the system control unit 70.

[0094] The display control unit 73 enlarges and displays the thumbnail image corresponding to the touch area pressed by the user on the second display unit 53. In the example shown in Fig. 13, the user presses the touch area 513a, and the thumbnail image 513 corresponding to the touch area 513a is enlarged and displayed on the second display unit 53.

[0095] 9, if the dividing unit 71 determines in step S2 that the shooting mode is not the still image mode, that is, if the dividing unit 71 determines that the shooting mode is the moving image mode, it determines whether the moving image mode is the first moving image mode (step S10). If the dividing unit 71 determines that the moving image mode is the first moving image mode, it performs a process of setting the shooting mode to the first moving image mode (step S11). On the other hand, if the dividing unit 71 determines that the moving image mode is not the first moving image mode, that is, if the dividing unit 71 determines that the moving image mode is the second moving image mode, it performs a process of setting the shooting mode to the second moving image mode (step S12).

[0096] In the process of step S11 or step S12, the division unit 71 executes the array pattern setting process shown in FIG. 10. In the process shown in FIG. 10, the division unit 71 instructs the image processing unit 30 (first image processing unit 30A) to detect a main subject (step S21). The detection unit 32A compares multiple image data obtained in time series from live view images to detect moving and stationary subjects. In this embodiment, the detection unit 32A recognizes faces in the image data based on the eyes, mouth, skin color, etc., and detects the faces as main subjects. In this embodiment, in addition to face detection, the detection unit 32A also detects human bodies (people) included in the image data as main subjects. The detection unit 32A then outputs the detection result together with the image data to the system control unit 70. The division unit 71 checks whether a main subject is present based on the detection result of the detection unit 32A. The division unit 71 then sets an area in the pixel area 113A according to the main subject and the shooting mode (step S22).

[0097] Specifically, when the shooting mode is the first moving image mode, the dividing unit 71 does not divide the pixel region 113A into multiple regions. That is, the dividing unit 71 sets all the pixel regions 113A as one region. At this time, the dividing unit 71 outputs an instruction signal to the driving unit 21 to instruct the driving unit 21 to set all the pixel regions 113A as one region.

[0098] On the other hand, when the shooting mode is the second video mode, the division unit 71 selects one of the arrangement patterns shown in FIGS. 8(A) to 8(D). Then, the division unit 71 checks whether the main subject is a moving subject or not based on the detection result of the detection unit 32A. When the main subject is not a moving subject but a stationary subject, the division unit 71 sets the first region and the second region according to the third arrangement pattern shown in FIG. 8(C). When the main subject is a moving subject, the division unit 71 checks the movement direction of the moving subject. When the movement direction of the moving subject is mainly up and down, the division unit 71 sets the first region and the second region according to the first arrangement pattern shown in FIG. 8(A). When the movement direction of the moving subject is mainly left and right, the division unit 71 sets the first region and the second region according to the second arrangement pattern shown in FIG. 8(B). Furthermore, if the moving subject moves at a high speed in the vertical direction, the dividing unit 71 sets the first, second, and third regions according to the fourth arrangement pattern shown in Fig. 8(D). In step S22, the dividing unit 71 outputs an instruction signal to the driving unit 21 to instruct the position of the blocks in each region (the first and second regions, and the first to third regions).

[0099] FIG. 14 is a diagram showing an example of setting the second arrangement pattern when the second video mode is executed. Note that in FIG. 14, each block is enlarged to make the arrangement of the blocks more visible. However, in reality, blocks smaller than the size of the blocks shown in FIG. 14 are set in the pixel region 113A. In the example shown in FIG. 14, the detection unit 32A detects people O1 and O2 playing soccer and a soccer ball O3 as main subjects (moving subjects). The detection unit 32A also detects the people O1 and O2 included in the image data as main subjects. Based on the detection result of the detection unit 32A, the division unit 71 determines that the main subjects O1 to O3 are moving subjects and that the moving subjects move mainly in the left-right direction. As a result, the division unit 71 sets the first region and the second region according to the second arrangement pattern shown in FIG. 8(B). The division unit 71 also determines that the main subjects O1 and O2 are people. As a result, the dividing unit 71 sets the areas 200 and 201 surrounding the main subjects O1 and O2 as second area A, and the area other than the areas 200 and 201 surrounding the main subjects O1 and O2 as second area B.

[0100] Returning to the explanation of FIG. 10, the drive control unit 72 sets the imaging conditions for the areas set in step S22 (first area, second area A, and second area B in the example shown in FIG. 14) based on the detection result of the detection unit 32A (step S23). Specifically, the drive control unit 72 outputs an instruction signal to the drive unit 21 instructing the imaging conditions (frame rate, gain, etc.) according to the main subject. The drive control unit 72 also outputs an instruction signal to the image processing unit 30 instructing the imaging conditions (parameters such as color signal processing, white balance adjustment, gradation adjustment, compression rate, etc.) according to the main subject.

[0101] For example, when a moving subject is detected by the detector 32A, the drive control unit 72 increases the gain (ISO sensitivity) of the first region and shortens the charge accumulation time of the first region. On the other hand, when a moving subject is not detected by the detector 32A, the drive control unit 72 decreases the gain of the first region and shortens the charge accumulation time of the first region. On the other hand, when a moving subject is detected by the detector 32A, the drive control unit 72 increases the frame rate of the region of the moving subject (regions 200 and 201 surrounding the main subjects O1 and O2, i.e., second region A). On the other hand, the drive control unit 72 decreases the frame rate of the region of the non-moving subject (regions other than the regions 200 and 201 surrounding the main subjects O1 and O2, i.e., second region B) compared to second region A.

[0102] 9, drive control unit 72 determines whether or not the user has operated video switch 55c (step S13). If drive control unit 72 determines that video switch 55c has been operated, drive control unit 72 causes imaging unit 20 to capture an image in a video mode (first video mode or second video mode) (step S14). Note that, as capturing an image in the first video mode is the same as normal video shooting, a detailed description thereof will be omitted.

[0103] FIG. 15 is a timing chart showing the timing of charge accumulation in the second moving image mode. In the second moving image mode (still image-moving image mixed mode) shown in FIG. 15, for example, a first region, a second region A, and a second region B are set. In the second moving image mode, while moving image switch 55c is being operated, drive control unit 72 outputs an instruction signal to drive unit 21 to repeatedly capture still images in the first region and capture moving images in second region A and second region B. In FIG. 15, while moving image switch 55c is being operated, drive unit 21 causes each pixel in the first region to capture a still image for a charge accumulation time T1. Furthermore, while moving image switch 55c is being operated, drive unit 21 causes each pixel in second region A to capture a moving image for a charge accumulation time T2A. Furthermore, while moving image switch 55c is being operated, drive unit 21 causes each pixel in second region B to capture a moving image for a charge accumulation time T2B longer than the charge accumulation time T2A. The frame rate changes depending on the charge accumulation time. Therefore, the frame rate of the moving image differs between when imaging is performed over charge accumulation time T2A and when imaging is performed over charge accumulation time T2B. For example, the frame rate corresponding to charge accumulation time T2A in the second region A is 60 fps, and the frame rate corresponding to charge accumulation time T2B in the second region B is 30 fps. The charge accumulation time and frame rate are set in the imaging condition setting process in step S23.

[0104] Furthermore, the pixel signals of each pixel read out from the first region of the image sensor 100 are amplified in the amplifier 412 by the gain specified by the division unit 71, and then output to the image processing unit 30. The image generation unit 31A checks parameters used in image processing such as color signal processing, based on the instruction signal specifying the imaging conditions for the first region output from the division unit 71. The image generation unit 31A then generates image data for the first region by performing various image processes on the RAW data made up of the pixel signals of each pixel in the first region, based on the parameters.

[0105] Furthermore, the pixel signals of each pixel read out from the second region A of the image sensor 100 are amplified in the amplifier 412 by the gain specified by the division unit 71, and then output to the image processing unit 30. The image generation unit 31B checks parameters used in image processing such as color signal processing, based on the instruction signal specifying the imaging conditions for the second region A output from the division unit 71. The image generation unit 31B then generates image data for the second region A by performing various image processing operations on the RAW data consisting of the pixel signals of each pixel in the second region A, based on the parameters.

[0106] Furthermore, the pixel signals of each pixel read out from the second region B of the image sensor 100 are amplified by the amplifier 412 with a gain specified by the division unit 71, and then output to the image processing unit 30. The image generation unit 31B checks parameters used in image processing such as color signal processing based on the instruction signal specifying the imaging conditions for the second region B output from the division unit 71. The image generation unit 31B then generates image data for the second region B by performing various image processes on RAW data consisting of the pixel signals of each pixel in the second region B based on the parameters. The image generation unit 31 (image generation unit 31A or 31B) also combines the image data of the second region A with the image data of the second region B. The image generation unit 31 (image generation unit 31A or 31B) also combines the image data of the first region, the image data of the second region A, and the image data of the second region B.

[0107] In this way, since the frame rate of the second area A is set higher than the frame rate of the second area B, the movements of the main human subjects O1 and O2 in the video become smoother.

[0108] 9, the display control unit 73 outputs image data of the moving image generated by the image processing unit 30 to the display unit 50, thereby displaying the moving image on the first display unit 51 (step S8). The display control unit 73 also outputs image data of the still image generated by the image processing unit 30 to the display unit 50, thereby displaying the still image on the second display unit 53 (step S8).

[0109] 16 is a diagram showing a display example in which a moving image is displayed on the first display unit and a still image is displayed on the second display unit. As shown in Fig. 16, the display control unit 73 displays a moving image (a moving image of two people playing soccer) that is a composite of the second area A and the second area B generated by the image generation unit 31 on the first display unit 51. The display control unit 73 also displays the still image of the first area generated by the image generation unit 31.

[0110] As described above, the first embodiment is configured to include a drive control unit 72 that controls the driving of the image sensor 100, a division unit 71 that divides the imaging area 113A of the image sensor 100 into at least a first area and a second area, and an image generation unit 31 that generates a first image by imaging the first area and a second image by imaging the second area of ​​the same subject. This configuration makes it possible to generate multiple types of images (multiple still images, a still image and a video, etc.) of the same subject. Therefore, it is possible to generate multiple types of images according to the subject and the shooting conditions, improving the usability of the electronic device 1 that includes the image sensor 100.

[0111] Furthermore, the drive control unit 72 controls the drive of the image sensor 100 by differentiating the timing at which imaging of the first area begins from the timing at which imaging of the second area begins. This configuration allows multiple types of images to be generated for the same subject at various times. It also allows a large number of images to be generated per unit time. This allows the user to capture images without missing a good opportunity to capture an image.

[0112] Furthermore, since the drive control unit 72 captures an image of the second area while capturing an image of the first area, the first and second areas can be captured in parallel, making it possible to capture images of the same subject with partially overlapping exposure times. This makes it possible to capture images of the same subject at a timing that was previously not possible. Furthermore, the drive control unit 72 differentiates at least one of the frame rate, gain, and exposure time as the imaging conditions for the first and second areas of the image sensor 100. This configuration allows the user to obtain multiple types of images captured under different imaging conditions.

[0113] Furthermore, since the image generation unit 31 generates a still image based on at least one of the imaging of the first area and the imaging of the second area, it is possible to generate multiple types of still images for the same subject. Furthermore, since the image generation unit 31 generates a moving image based on either the imaging of the first area or the imaging of the second area, it is possible to generate both a still image and a moving image for the same subject. Furthermore, since the image generation unit 31 performs correction on the first image and the second image by making at least one of white balance, gradation, and color tone correction different, the user can obtain multiple types of images that have been image-processed based on different parameters.

[0114] Furthermore, since the dividing unit 71 forms the first region from a plurality of discrete regions (a plurality of discrete blocks), the resolution of the image does not decrease partially. Furthermore, since the dividing unit 71 variably divides the first region and the second region, it is possible to divide the image into regions according to various conditions such as the shooting mode and the type of subject.

[0115] The image sensor 100 also includes a detection unit 32A that detects a main subject from an image generated by the image generation unit 31, and a division unit 71 that divides the image into a first region and a second region such that the main subject is included therein, thereby generating an image of the first region and an image of the second region for the main subject. The image sensor 100 also includes a display control unit 73 that displays the image generated by the image generation unit 31 on the display unit 50, allowing the user to check the image displayed on the display unit 50. Furthermore, the image sensor 100 has a structure in which a back-illuminated imaging chip and a signal processing chip are stacked, thereby reducing the volume required to accommodate the image sensor 100. Furthermore, the image sensor 100 can be driven and controlled based on instructions from the system control unit 70, thereby reducing the burden on the system control unit 70 and facilitating the installation of the image sensor 100 in the electronic device 1.

[0116] In the electronic device 1 according to the first embodiment shown in FIG. 5, the display unit 50 may be configured to be provided outside the electronic device. In this case, the system control unit 70 and the display unit 50 are each provided with a communication unit that transmits and receives signals (image data, control signals, etc.) via wire or wirelessly. The image processing unit 30 and the system control unit 70 may be configured as an integrated unit. In this case, the system control unit having one or more CPUs performs the functions of the image processing unit 30 and the system control unit 70 by executing processing based on a control program. In addition, although the image processing unit 30 includes two image processing units 30A and 30B, it may also be configured to include only one image processing unit.

[0117] Next, a modified example of the first embodiment will be described. In Fig. 8(A) to (C), the arrangement pattern of each block is set so that the area of ​​the first region and the area of ​​the second region are the same. That is, the arrangement pattern of each block is set so that the number of pixels in the first region and the number of pixels in the second region are the same. Also, in Fig. 8(D), the arrangement pattern of each block is set so that the area of ​​the first region, the area of ​​the second region, and the area of ​​the third region are the same. That is, the arrangement pattern of each block is set so that the number of pixels in the first region, the number of pixels in the second region, and the number of pixels in the third region are the same. However, the arrangement pattern of each block may be set so that the area (number of pixels) of each region is different.

[0118] FIG. 17 is a diagram showing a fifth arrangement pattern for each block. The fifth arrangement pattern shown in FIG. 17 is an arrangement pattern in which pixel region 113A is divided into a first region and a second region. In this fifth arrangement pattern, in pixel region 113A, the second region is made up of blocks of columns that are multiples of three (3m), and the first region is made up of blocks other than the second region, i.e., blocks of columns other than the columns that are multiples of three (3m-2, 3m-1). m is a positive integer (m=1, 2, 3, . . .). In this arrangement pattern, the ratio of the area of ​​the first region to the area of ​​the second region is 2:1.

[0119] FIG. 18 is a diagram showing a sixth arrangement pattern for each block. The sixth arrangement pattern shown in FIG. 18 is also an arrangement pattern in which the pixel region 113A is divided into a first region and a second region. In this sixth arrangement pattern, in the pixel region 113A, the second region is composed of a block in the row (4n-1) that is one row before the row that is a multiple of 4 in the odd-numbered column (2m-1) and a block in the row (4n-3) that is three rows before the row that is a multiple of 4 in the even-numbered column (2m), and the first region is composed of blocks other than the second region. m and n are positive integers (m=1, 2, 3, . . . , n=1, 2, 3, . . .). In this arrangement pattern, the ratio of the area of ​​the first region to the area of ​​the second region is 3:1.

[0120] In step S22 of Fig. 10, if dividing unit 71 determines that the shooting mode selected by the user is the second video mode, it sets the first region and the second region according to the fifth arrangement pattern shown in Fig. 17 or the sixth arrangement pattern shown in Fig. 18. Furthermore, if dividing unit 71 determines that the main subject is a stationary subject based on the detection result of detection unit 32A, it sets the first region and the second region according to the sixth arrangement pattern shown in Fig. 18. Furthermore, if dividing unit 71 determines that the main subject is a moving subject and that the moving subject moves mainly in the up-down direction based on the detection result of detection unit 32A, it sets the first region and the second region according to the fifth arrangement pattern shown in Fig. 17.

[0121] Then, in step S14, the drive control unit 72 outputs an instruction signal to the drive unit 21, causing the drive unit 21 to capture a still image in the first region and a moving image in the second region. When the first region and the second region are set according to the fifth arrangement pattern, the still image has twice the number of pixels of the moving image. That is, the still image has twice the resolution of the moving image. When the first region and the second region are set according to the sixth arrangement pattern, the still image has three times the number of pixels of the moving image. That is, the still image has three times the resolution of the moving image. This is because still images require higher image quality than moving images. Furthermore, since the subject is often moving in moving images, degradation in image quality is less noticeable than in still images. For these reasons, a larger area is allocated to still images than to moving images. Note that if the number of pixels in the pixel region 113A is 20 million, 6.66 million pixels or 5 million pixels are secured even if the number of pixels in the moving image (the number of pixels in the second region) is reduced to one-third or one-quarter. Such a pixel count is comparable to that of commercially available video cameras.

[0122] Second Embodiment In the first embodiment described above, the timing at which imaging of the first region starts is made different from the timing at which imaging of the second region starts in the second still image mode, as shown in Fig. 12(B). In contrast, in the second embodiment, the timing at which imaging of the first region starts is made the same as the timing at which imaging of the second region starts, and the exposure time (i.e., charge accumulation time) of the first region is made different from the exposure time of the second region in the second still image mode.

[0123] FIG. 19 is a timing chart showing the timing of charge accumulation in the second embodiment. In the second still image mode shown in FIG. 19, a first region and a second region are set. As shown in FIG. 19, the drive unit 21 repeatedly captures still images for each pixel in the first region for a charge accumulation time (exposure time) T11 while the release switch 55a is being operated. The drive unit 21 also repeatedly captures still images for each pixel in the second region for a charge accumulation time (exposure time) T12 while the release switch 55a is being operated. Here, the timing at which imaging of the first region starts is the same as the timing at which imaging of the second region starts. Meanwhile, the charge accumulation time T11 of the first region is different from the charge accumulation time T12 of the second region. That is, the charge accumulation time T12 is set to be longer than the charge accumulation time T11. The charge accumulation time is set in the imaging condition setting process of step S23.

[0124] The pixel signals of each pixel read out from the first region of the image sensor 100 are amplified in the amplifier 412 by the gain specified by the division unit 71, and then output to the image processing unit 30. The image generation unit 31A checks parameters used in image processing such as color signal processing based on the instruction signal specifying the imaging conditions for the first region output from the division unit 71. The image generation unit 31A then generates image data for the first region by performing various image processes based on the parameters on RAW data consisting of the pixel signals of each pixel in the first region.

[0125] Furthermore, the pixel signals of each pixel read out from the second region of the image sensor 100 are amplified in the amplifier 412 by a gain specified by the division unit 71, and then output to the image processing unit 30. The image generation unit 31B checks parameters used in image processing such as color signal processing based on the instruction signal specifying the imaging conditions for the second region output from the division unit 71. The image generation unit 31B then generates image data for the second region by performing various image processes on RAW data consisting of pixel signals of each pixel in the second region based on the parameters. The image generation unit 31 (image synthesis unit 31A or 31B) also synthesizes the image data of the first region and the image data of the second region.

[0126] Fig. 20 is a diagram showing a display example in which still images are displayed on the first display unit and the second display unit in the second embodiment. In Fig. 20, the display control unit 73 displays a still image of the first region (an image of a person captured at night) in the left region 53L of the display screen of the second display unit 53. The display control unit 73 also displays a still image of the second region in the right region 53R of the display screen of the second display unit 53. The display control unit 73 also displays a still image in the center region 51G of the first display unit 51, which is a composite of the still image of the first region and the still image of the second region.

[0127] Generally, HDR (High Dynamic Range) is an image synthesis technology for recording and displaying images with a wide dynamic range. HDR imaging is widely known. In HDR imaging, multiple images are captured while varying imaging conditions (e.g., exposure) and then combined to generate an image with minimal blown-out highlights and crushed shadows. However, in conventional HDR, for example, two images captured under different imaging conditions are captured at different times, which can result in the subject moving or the user (photographer) moving the electronic device 1. In this case, the multiple images are not of the same subject, making image composition difficult. In contrast, in the second embodiment, two images captured under different imaging conditions can be captured at the same time (or approximately the same time). Therefore, the configuration of the second embodiment can solve the problems of conventional HDR imaging. The HDR mode is selectable by the user's operation of the multi-selector 55d, for example.

[0128] <Third embodiment> In the third embodiment, the electronic device 1 in the first embodiment is separated into an imaging device 1A and an electronic device 1B.

[0129] Fig. 21 is a block diagram showing the configuration of an imaging device and electronic device according to the third embodiment. In the configuration shown in Fig. 21, imaging device 1A is a device that captures an image of a subject. This imaging device 1A includes a lens unit 10, an imaging unit 20, an image processing unit 30, a work memory 40, an operation unit 55, a recording unit 60, and a first system control unit 75. Note that, of the imaging device 1A, the configurations of 10, the imaging unit 20, the image processing unit 30, the work memory 40, the operation unit 55, and the recording unit 60 are the same as those shown in Fig. 5. Therefore, the same components are denoted by the same reference numerals, and redundant explanations will be omitted.

[0130] The electronic device 1B is a device that displays images (still images, videos, and live view images). The electronic device 1B includes a display unit 50 and a second system control unit (control unit) 70B. The display unit 50 of the electronic device 1B has the same configuration as that shown in FIG. 5. Therefore, the same components are denoted by the same reference numerals, and redundant explanations will be omitted.

[0131] The first system control unit 75 has a first communication unit 75A. The second system control unit 76 has a second communication unit 76B. The first communication unit 75A and the second communication unit 76B transmit and receive signals to and from each other via wire or wirelessly. The first system control unit 75 has components corresponding to, for example, the splitter 71 and the drive control unit 72 of the components shown in FIG. 7. The second system control unit 76 has only a component corresponding to, for example, the display control unit 73 of the components shown in FIG. 7.

[0132] 7 (the division unit 71, the drive control unit 72, and the display control unit 73) may be provided in either the first system control unit 75 or the second system control unit 76. All of the components shown in FIG. 7 may be provided in the first system control unit 75 or the second system control unit 76, or a part of the components shown in FIG. 7 may be provided in the first system control unit 75, and the components other than the part of the components shown in FIG. 7 may be provided in the second system control unit 76.

[0133] The imaging device 1A may be, for example, a digital camera, smartphone, mobile phone, or personal computer equipped with imaging and communication functions, and the electronic device 1B may be, for example, a mobile terminal equipped with communication functions, such as a smartphone, mobile phone, or portable personal computer.

[0134] The first system control unit 75 shown in Fig. 21 is realized by a CPU (not shown) executing processing based on a control program, and the second system control unit 76 shown in Fig. 21 is realized by a CPU (not shown) executing processing based on a control program.

[0135] 21, the image processing unit 30 and the first system control unit 75 may be integrated into one unit. In this case, a system control unit having one or more CPUs performs processing based on a control program, thereby fulfilling the functions of the image processing unit 30 and the first system control unit 75.

[0136] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications or improvements can be made to the above embodiments without departing from the spirit of the present invention. Furthermore, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omitted forms are also included in the technical scope of the present invention. Furthermore, it is also possible to apply appropriate combinations of the configurations of the above embodiments and modified examples.

[0137] For example, in the first and second embodiments described above, the electronic device 1 does not need to include the lens unit 10, the recording unit 60, etc., as long as it includes the imaging unit 20, the image processing unit 30 including the image generation unit 31, and the system control unit 70 including the division unit 71 and the drive control unit 72. In other words, these components may be configured separately from the electronic device 1. Also, in the third embodiment described above, the lens unit 10, the recording unit 60, etc. may also be configured separately from the imaging device 1A.

[0138] Furthermore, in each of the above-described embodiments, the color filters 102 are arranged in a Bayer array, but other arrays are also possible. Furthermore, the number of pixels forming the unit group 131 only needs to include at least one pixel. Furthermore, each block only needs to include at least one pixel. Therefore, it is possible to perform imaging under different imaging conditions for each pixel.

[0139] In each of the above-described embodiments, the drive unit 21 may have a part or all of its components mounted on the imaging chip 113, or a part or all of its components mounted on the signal processing chip 111. Also, a part of the configuration of the image processing unit 30 may be mounted on the imaging chip 113 or the signal processing chip 111. Also, a part of the configuration of the system control unit 70 may be mounted on the imaging chip 113 or the signal processing chip 111.

[0140] In each of the above-described embodiments, the imaging conditions, that is, the gain, the charge accumulation time (exposure time, shutter speed), and the frame rate, are all changeable, but it is sufficient if at least one of these is changeable. Also, while only the case where the imaging conditions are automatically set has been described, the imaging conditions may be set in response to the user's operation of the operation unit 55 or the like.

[0141] In the above-described embodiments, the block arrangement patterns are exemplified in FIGS. 8(A) to 8(D), 17, and 18, but are not limited to these arrangement patterns. The user may select the block arrangement pattern by operating the operation unit 55 or the like. The location for displaying the still images and videos captured by the imaging unit 20 may be the first display unit 51 or the second display unit 53. In the above-described embodiments, the size of the block area is set in advance, but the size of the block area may be set by the user.

[0142] In the first embodiment described above, the dividing unit 71 recognizes the subject and sets the area based on a live view image. However, the dividing unit 71 may recognize the subject and set the area based on the image when the release switch 55a or the movie switch 55c is half-pressed.

[0143] Furthermore, the first embodiment described above may be provided with a shooting mode for performing panning. Panning is a shooting method in which the moving subject remains stationary while the background (non-moving subject) is blurred, thereby expressing the sense of speed of the moving subject. In the panning shooting mode, a panning image with a long charge accumulation time (exposure time) and a moving background is captured in the first region, and a normal panning image with a shorter charge accumulation time than the first region is captured in the second region. Then, the image generation unit 31 (or the user) appropriately combines the panning image of the first region with the panning image of the second region. [Explanation of symbols]

[0144] DESCRIPTION OF SYMBOLS 1, 1B...electronic device, 1A...imaging device, 20...imaging section, 30...image processing section, 31, 31A, 31B...image generation section, 32A...detection section, 50...display section, 51...first display section, 52...first touch panel, 53...second display section, 54...second touch panel, 70...system control section, 70A...first system control section, 70B...second system control section (control section), 71...division section, 72...drive control section, 73...display control section, 100...imaging element

Claims

1. A first semiconductor unit having a plurality of pixels, A semiconductor portion stacked together with the first semiconductor portion, the second semiconductor portion having: a first amplifier portion that amplifies a first signal read from a first pixel among the plurality of pixels; a second amplifier portion that amplifies a second signal read from a second pixel arranged alongside the first pixel in a first direction among the plurality of pixels; a third amplifier portion that amplifies a third signal read from a third pixel arranged alongside the first pixel in a second direction intersecting the first direction among the plurality of pixels; and a drive unit that controls the gains of at least two of the amplifier portions, the gain of the first amplifier portion, the gain of the second amplifier portion, and the gain of the third amplifier portion, so that they have different gains. An image sensor equipped with the following features.

2. In the image sensor according to Claim 1, The drive unit controls the gain of the first amplifier section and the gain of the second amplifier section to be different gains. Image sensor.

3. In the image sensor according to Claim 2, The drive unit controls the gain of the second amplifier section and the gain of the third amplifier section to be different gains. Image sensor.

4. In the image sensor according to claim 1, The drive unit controls the gain of the first amplifier section and the gain of the third amplifier section to be different gains. Image sensor.

5. In the image sensor according to any one of claims 1 to 4, The second semiconductor unit includes a first conversion unit that converts the first signal into a digital signal, a second conversion unit that converts the second signal into a digital signal, and a third conversion unit that converts the third signal into a digital signal. Image sensor.

6. In the image sensor according to claim 5, The first conversion unit converts the first signal amplified by the first amplifier unit into a digital signal. The second conversion unit converts the second signal amplified by the second amplifier unit into a digital signal. The third conversion unit converts the third signal amplified by the third amplifier unit into a digital signal. Image sensor.

7. In the image sensor according to claim 5 or claim 6, The drive unit controls the number of bits of at least two of the conversion units, including the number of bits of the first conversion unit, the number of bits of the second conversion unit, and the number of bits of the third conversion unit, so that they are different numbers of bits. Image sensor.

8. In the image sensor according to claim 7, The drive unit controls the number of bits in the first conversion unit and the number of bits in the second conversion unit to be different. Image sensor.

9. In the image sensor according to claim 8, The drive unit controls the number of bits in the second conversion unit and the number of bits in the third conversion unit to be different. Image sensor.

10. In the image sensor according to claim 7, The drive unit controls the number of bits in the first conversion unit and the number of bits in the third conversion unit to be different. Image sensor.

11. In the image sensor according to any one of claims 1 to 10, The wiring layer includes a first signal line for outputting the first signal read from the first pixel to the first amplifier section, a second signal line for outputting the second signal read from the second pixel to the second amplifier section, and a third signal line for outputting the third signal read from the third pixel to the third amplifier section. The wiring layer is arranged between the first semiconductor portion and the second semiconductor portion in the stacking direction in which the first semiconductor portion and the second semiconductor portion are stacked. Image sensor.

12. In the image sensor according to claim 11, The wiring layer has a first control line on which a first control signal for controlling the first pixel is output, a second control line on which a second control signal for controlling the second pixel is output, and a third control line on which a third control signal for controlling the third pixel is output. Image sensor.

13. In the image sensor according to claim 12, The drive unit controls the timing of outputting at least two of the following control signals: the timing of outputting the first control signal on the first control line, the timing of outputting the second control signal on the second control line, and the timing of outputting the third control signal on the third control line, so that they are at different timings. Image sensor.

14. In the image sensor according to claim 13, The drive unit controls the timing at which it outputs the first control signal to the first control line and the timing at which it outputs the second control signal to the second control line to be different. Image sensor.

15. In the image sensor according to claim 14, The drive unit controls the timing at which it outputs the second control signal to the second control line and the timing at which it outputs the third control signal to the third control line to be different. Image sensor.

16. In the image sensor according to claim 13, The drive unit controls the timing at which it outputs the first control signal to the first control line and the timing at which it outputs the third control signal to the third control line to be different. Image sensor.

17. In the image sensor according to any one of claims 1 to 16, The second pixel is positioned next to the first pixel in the first direction. Image sensor.

18. In the image sensor according to claim 17, The third pixel is positioned next to the first pixel in the second direction. Image sensor.

19. In the image sensor according to any one of claims 1 to 16, The third pixel is positioned next to the first pixel in the second direction. Image sensor.

20. An electronic device comprising an image sensor according to any one of claims 1 to 19.

21. In the electronic device according to claim 20, An electronic device comprising an image processing unit electrically connected to the aforementioned image sensor.

22. In the electronic device according to claim 20 or claim 21, An electronic device comprising an optical system for emitting light to the aforementioned image sensor.

Images