Imaging device, control method, and control program

Abstract

Provided are an imaging device, a control method, and a control program capable of improving reading efficiency.　An imaging element (5) has: a first imaging mode for generating first image data by performing photoelectric conversion at a first conversion efficiency; and a second imaging mode for generating second image data by performing photoelectric conversion at a second conversion efficiency different from the first conversion efficiency. A system control unit (11) sets a first pixel region and a second pixel region, and, for imaging in the first pixel region, acquires the first image data and the second image data by performing imaging in the first imaging mode and the second imaging mode. For imaging in the second pixel region, the system control unit (11) acquires the second image data by performing imaging in the second imaging mode among the first imaging mode and the second imaging mode.

Description

Imaging device, control method, and control program

[0001] The present invention relates to an imaging device, a control method, and a control program.

[0002] Patent document 1 describes an image processing device that processes first and second image data from an imaging unit that is capable of amplifying a signal obtained by photoelectric conversion with different gains and outputting first and second image data, and that has an acquisition unit that acquires an exposure difference between the first and second image data, a correction unit that determines a correction amount to suppress a noise difference between the first and second image data based on the exposure difference acquired by the acquisition unit and corrects at least one of the first and second image data based on the determined correction amount, and a generation unit that generates a composite image by combining the first and second image data after correction by the correction unit.

[0003] Patent document 2 describes an analog-to-digital converter including an analog storage that stores a reset pixel value output by a dual conversion gain pixel at a first gain, a circuit that receives as a first input and a second input the reset pixel value output by the dual conversion gain pixel at a second gain and the signal pixel value output at the second gain, and outputs as a first input and a second input to a converter stage the reset pixel value at the first gain stored in the analog storage and the signal pixel value output at the first gain, and a converter stage that outputs digital values ​​that indicate the difference between the reset pixel value at the second gain and the signal pixel value at the second gain, and the difference between the reset pixel value at the first gain and the signal pixel value at the first gain, which are input as the first input and the second input.

[0004] Patent document 3 describes an imaging device that includes a drive control unit that controls the driving of an imaging element, a division unit that divides the imaging element into a first region and a second region, and a blinking detection unit that detects blinking of a light source based on a signal read out from the second region by the drive control unit.

[0005] Japanese Patent Publication No. 2023-176537 Japanese Patent Publication No. 2023-070125 Japanese Patent Publication No. 2015-092660

[0006] One embodiment of the technique of the present disclosure provides an imaging device, a control method, and a control program that can improve readout efficiency.

[0007] (1) An imaging device having a processor and a first imaging mode in which first image data is generated by photoelectric conversion at a first conversion efficiency, and a second imaging mode in which second image data is generated by photoelectric conversion at a second conversion efficiency different from the first conversion efficiency, wherein the processor sets a first pixel region and a second pixel region, and when imaging the first pixel region, performs imaging in the first imaging mode and the second imaging mode to obtain the first image data and the second image data, and when imaging the second pixel region, performs imaging in the second imaging mode of the first imaging mode and the second imaging mode to obtain the second image data.

[0008] (2) The imaging device according to (1), wherein the first pixel region is a region of pixels for a recording image, and the second pixel region is a region of pixels for phase difference detection.

[0009] (3) The imaging device according to (1), wherein the first pixel region is a region corresponding to a specific subject, and the second pixel region is a region different from the first pixel region.

[0010] (4) An imaging device according to any one of (1) to (3), wherein the processor, when imaging the first pixel region, repeatedly performs imaging in the first imaging mode and imaging in the second imaging mode, and, when imaging the second pixel region, repeatedly performs non-imaging and imaging in the second imaging mode.

[0011] (5) The imaging device according to (4), wherein the processor turns off a power supply to a conversion circuit that converts an analog imaging signal into a digital signal during the non-imaging period.

[0012] (6) The imaging device according to (4) or (5), wherein the processor matches the timing of imaging in the second imaging mode in imaging the first pixel region with the timing of imaging in the second imaging mode in imaging the second pixel region.

[0013] (7) An imaging device according to any one of (1) to (3), wherein the processor, when imaging the first pixel region, repeatedly performs imaging in the first imaging mode and imaging in the second imaging mode, and, when imaging the second pixel region, repeatedly performs imaging in the second imaging mode.

[0014] (8) The imaging device according to (7), wherein the processor causes the timing of correlated double sampling for imaging in the second imaging mode in imaging the second pixel region to correspond to the timing of correlated double sampling for imaging in the second imaging mode in imaging the first pixel region.

[0015] (9) The imaging device according to (7), wherein the processor sets an interval of correlated double sampling for imaging in the first imaging mode and the second imaging mode to correspond to a period that is an integer fraction of a period of periodic disturbance noise.

[0016] (10) The imaging device according to any one of (1) to (9), wherein the processor sets the period of the horizontal synchronization signal in the imaging of the second pixel region to N / M times the period of the horizontal synchronization signal in the imaging of the first pixel region, where N and M are integers.

[0017] (11) An imaging device according to any one of (1) to (10), wherein the processor generates a portion of the captured image corresponding to the first pixel region based on the first image data and the second image data, and generates a portion of the captured image corresponding to the second pixel region based on the second image data.

[0018] (12) A control method for an imaging device having a processor and a first imaging mode in which first image data is generated by photoelectric conversion at a first conversion efficiency, and a second imaging mode in which second image data is generated by photoelectric conversion at a second conversion efficiency different from the first conversion efficiency, wherein the processor sets a first pixel region and a second pixel region, and when imaging the first pixel region, performs imaging in the first imaging mode and the second imaging mode to obtain the first image data and the second image data, and when imaging the second pixel region, performs imaging in the second imaging mode of the first imaging mode and the second imaging mode to obtain the second image data.

[0019] (13) A control program for an imaging device having a processor and a first imaging mode in which first image data is generated by photoelectric conversion at a first conversion efficiency, and a second imaging mode in which second image data is generated by photoelectric conversion at a second conversion efficiency different from the first conversion efficiency, the control program causing the processor to execute the following processes: set a first pixel region and a second pixel region; when imaging the first pixel region, imaging is performed in the first imaging mode and the second imaging mode to obtain the first image data and the second image data; and when imaging the second pixel region, imaging is performed in the second imaging mode of the first imaging mode and the second imaging mode to obtain the second image data.

[0020] According to the present invention, it is possible to provide an imaging device, a control method, and a control program that can improve readout efficiency.

[0021] 1 is a diagram showing an example of the configuration of a digital camera 100 to which the imaging device of the present invention is applied. FIG. 1 is a schematic plan view showing a general configuration of the image sensor 5 shown in FIG. 1. FIG. 2 is a diagram showing an example of readout using Dual Gain. FIG. 3 is a diagram showing a first control example of turning Dual Gain on / off according to the region. FIG. 4 is a diagram showing a second control example of turning Dual Gain on / off according to the region. FIG. 5 is a diagram showing a third control example of turning Dual Gain on / off according to the region. FIG. 6 is a diagram showing an example of an enlarged image. FIG. 7 is a diagram showing an example of controlling the power on / off of the AD conversion unit 65b when Dual Gain is on. FIG. 8 is a diagram showing an example of controlling the power on / off of the AD conversion unit 65b when Dual Gain is off. FIG. 9 is a diagram showing a first readout example when Dual Gain is on / off. FIG. 10 is a diagram showing a second readout example when Dual Gain is on / off. FIG. 11 is a diagram showing a third readout example when Dual Gain is on / off. FIG. 12 is a diagram showing a fourth readout example when Dual Gain is on / off. FIG. 10 is a diagram showing a fifth readout example when DualGain is on / off. FIG. 11 is a diagram showing a sixth readout example when DualGain is on / off. FIG. 12 is a diagram showing a seventh readout example when DualGain is on / off. FIG. 13 is a diagram showing an example of reducing the influence of disturbance noise by adjusting the interval of CDS processing. FIG. 14 is a diagram showing an example of readout when the period of the horizontal synchronization signal is not adjusted. FIG. 15 is a diagram showing an example of readout when the period of the horizontal synchronization signal is adjusted. FIG. 16 is a block diagram showing the appearance of the smartphone 200. FIG. 17 is a block diagram showing the configuration of the smartphone 200.

[0022] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0023] <Example of the Configuration of a Digital Camera 100 to which an Imaging Apparatus of the Present Invention is Applied> FIG. 1 is a diagram showing an example of the configuration of a digital camera 100 to which an imaging apparatus of the present invention is applied.

[0024] 1 is a digital camera including a lens device 40 having an imaging lens 1, an aperture 2, a lens control unit 4, a lens drive unit 8, and an aperture drive unit 9, and a main body 100A. The main body 100A includes an imaging unit 50, a system control unit 11, an operation unit 14, a display device 22, a memory 16 including a RAM (Random Access Memory) and a ROM (Read Only Memory), a memory control unit 15 that controls data recording to and data reading from the memory 16, a digital signal processing unit 17, and an external memory control unit 20 that controls data recording to and data reading from a recording medium 21.

[0025] The lens device 40 may be detachable from the main body 100A, or may be integrated with the main body 100A. The imaging lens 1 includes a focus lens that is movable in the optical axis direction. This focus lens is a lens for adjusting the focus of the imaging optical system that includes the imaging lens 1 and the diaphragm 2, and is composed of a single lens or multiple lenses. When the focus lens moves in the optical axis direction, the position of the principal point of the focus lens changes along the optical axis, changing the focal position on the subject side. Note that the focus lens may be a liquid lens whose principal point in the optical axis direction can be changed by electrical control.

[0026] The lens control unit 4 of the lens device 40 is configured to be able to communicate with the system control unit 11 of the main body 100A via a wired or wireless connection. In accordance with commands from the system control unit 11, the lens control unit 4 controls the focus lens included in the imaging lens 1 via the lens drive unit 8 to change the position of the principal point of the focus lens, and controls the aperture value of the aperture 2 via the aperture drive unit 9.

[0027] The imaging unit 50 includes an imaging element 5 that captures an image of a subject through an imaging optical system including an imaging lens 1 and an aperture 2 , and an imaging element driving unit 10 that drives the imaging element 5 .

[0028] The imaging element 5 has an imaging surface 60 (see FIG. 2) on which a plurality of pixels 61 are arranged two-dimensionally, and converts a subject image formed on the imaging surface 60 by an imaging optical system into pixel signals by the plurality of pixels 61 and outputs the pixel signals. A CMOS (complementary metal-oxide-semiconductor) image sensor, for example, is preferably used as the imaging element 5. The following description will be given assuming that the imaging element 5 is a CMOS image sensor.

[0029] The system control unit 11, which controls the entire electrical control system of the digital camera 100, drives the image sensor 5 via the image sensor drive unit 10, and outputs the subject image captured through the imaging optical system of the lens device 40 as an image signal.

[0030] The image sensor driver 10 generates a drive signal based on a command from the system controller 11 and supplies the drive signal to the image sensor 5, thereby driving the image sensor 5. The hardware configuration of the image sensor driver 10 is an electric circuit configured by combining circuit elements such as semiconductor elements.

[0031] An instruction signal from the user is input to the system control unit 11 through the operation unit 14. The operation unit 14 includes a touch panel integrated with a display surface 22b (to be described later), various buttons, and the like.

[0032] The system control unit 11 controls the entire digital camera 100, and its hardware configuration consists of various processors that execute programs, including an imaging control program, to perform processing. The programs executed by the system control unit 11 are stored in the ROM of the memory 16.

[0033] The various types of processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes programs to perform various processes, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), or a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically to perform specific processing. More specifically, the structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0034] The system control unit 11 may be configured with one of various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0035] The display device 22 includes a display surface 22b configured by an organic EL (Electro Luminescence) panel, a liquid crystal panel, or the like, and a display controller 22a that controls the display on the display surface 22b.

[0036] The memory control unit 15 , digital signal processing unit 17 , external memory control unit 20 and display controller 22 a are interconnected by a control bus 24 and a data bus 25 , and are controlled by commands from the system control unit 11 .

[0037] The imaging device of the present invention is configured, for example, by an imaging unit 50 and a system control unit 11. In this case, the system control unit 11 is an example of the "processor" of the present invention. Alternatively, the imaging device of the present invention may be configured by the imaging unit 50. In this case, the "processor" of the present invention is a processor provided in the imaging unit 50.

[0038] <Schematic Configuration of Image Sensor 5 Shown in Fig. 1> Fig. 2 is a plan view schematic diagram showing the schematic configuration of the image sensor 5 shown in Fig. 1. The image sensor 5 has an imaging surface 60 on which a plurality of pixels 61 are two-dimensionally arranged in a row direction X and a column direction Y perpendicular to the row direction X. The plurality of pixels 61 include a ranging pixel 61b that receives one of a pair of light beams that have passed through two different portions of a pupil region of the imaging optical system aligned in the row direction X and detects a signal corresponding to the amount of received light, a ranging pixel 61c that receives the other of the pair of light beams and detects a signal corresponding to the amount of received light, and a normal pixel 61a that receives both of the pair of light beams and detects a signal corresponding to the amount of received light.

[0039] 2 , pixel lines 62 each including a plurality of normal pixels 61 a arranged in the row direction X and pixel lines 63 each including ranging pixels 61 b ​​and ranging pixels 61 c arranged alternately in the row direction X are arranged alternately in the column direction Y on the imaging surface 60. It is sufficient that the pixel lines 63 include a plurality of pairs of ranging pixels 61 b ​​and ranging pixels 61 c, and they may also include normal pixels 61 a in addition to these pairs. Hereinafter, when there is no need to distinguish between the pixel lines 62 and 63, they will also be simply referred to as pixel lines. The imaging element 5 further includes a drive circuit 64 that drives the pixels 61 arranged on the imaging surface 60 and a signal processing circuit 65 that processes pixel signals read out to signal lines from each pixel 61 in each pixel line arranged on the imaging surface 60.

[0040] In the following description, one end of the imaging surface 60 in the column direction Y (upper side in the figure) in FIG. 2 will be referred to as the upper end, and the other end of the imaging surface 60 in the column direction Y (lower side in the figure) will be referred to as the lower end.

[0041] The drive circuit 64 independently drives each pixel line based on a signal from the image sensor drive unit 10, resets each pixel 61 included in each pixel line (discharges the charge accumulated in the photoelectric conversion element), and reads out pixel signals corresponding to the charge accumulated in the photoelectric conversion element of each pixel 61 to a signal line.

[0042] The signal processing circuit 65 performs CDS (Correlated Double Sampling) processing on pixel signals read out from each pixel 61 in the pixel line to a signal line, converts the pixel signals after CDS processing into digital signals, and outputs the digital signals to the data bus 25 (see FIG. 1 ). The signal processing circuit 65 is controlled by the image sensor drive unit 10. For example, the signal processing circuit 65 includes a CDS processing unit 65a that performs CDS processing, and an AD conversion unit 65b that converts the pixel signals after CDS processing into digital signals. The AD conversion unit 65b is an example of a "conversion circuit" that converts analog image signals into digital signals.

[0043] The digital signal processing unit 17 performs signal processing such as demosaic processing and gamma correction processing on the pixel signals output from the image sensor 5 to the data bus 25 to generate captured image data.

[0044] The digital camera 100 is equipped with a continuous shooting mode in which multiple captured image data are generated in succession in response to a single image capture instruction and recorded on the recording medium 21 .

[0045] In continuous shooting mode, the system control unit 11 drives the image sensor 5 by the image sensor drive unit 10 using the rolling shutter method to capture an image of a subject. Rolling shutter driving includes rolling reset driving and rolling readout driving. Rolling reset driving is a driving method that resets each pixel 61 in a pixel line and starts exposure of each pixel 61, sequentially changing the pixel line. Rolling readout driving is a driving method that reads signals from each pixel 61 in an exposed pixel line and ends exposure of that pixel line, sequentially changing the pixel line.

[0046] In the continuous shooting mode, when the system control unit 11 receives an image capture instruction, it continuously performs recording imaging control, which records captured image data, displays a live view image on the display surface 22b, and causes the image sensor 5 to output pixel signals to be used for distance measurement. In addition, between each of the multiple times of recording imaging control, the system control unit 11 performs display imaging control at least once, which causes the image sensor 5 to display a live view image on the display surface 22b and output pixel signals to be used for distance measurement.

[0047] The ranging is, for example, a phase difference ranging used in phase difference AF (Auto Focus). For example, the ranging is a process of performing a correlation calculation between a first pixel signal group output from each ranging pixel 61 b ​​included in the same pixel line 63 and a second pixel signal group output from each ranging pixel 61 c, and deriving a drive amount of the focus lens required to focus on a target subject based on the result of the correlation calculation.

[0048] The correlation calculation is a process of calculating the area S[d] enclosed by two data waveforms when the data waveform consisting of the first pixel signal group and the data waveform consisting of the second pixel signal group are shifted by a shift amount d, by changing the shift amount d among multiple values.

[0049] <Readout Using Dual Gain> Fig. 3 is a diagram showing an example of readout using Dual Gain. Readout control in the imaging unit 50 is performed by, for example, the system control unit 11. The system control unit 11 can switch Dual Gain on and off when reading out from the image sensor 5. When Dual Gain is off, the imaging unit 50 reads out the same pixel at one gain. When Dual Gain is on, the imaging unit 50 reads out the same pixel twice at two different gains. Gain is, for example, the amplification factor during photoelectric conversion. Gain control will be described later.

[0050] The horizontal axis of Fig. 3 corresponds to the input during photoelectric conversion, i.e., the charge accumulated in the photoelectric conversion element of the pixel 61 of the image sensor 5. The vertical axis of Fig. 3 corresponds to the output during photoelectric conversion, i.e., the value of the electronic signal. The high gain 31 and the low gain 32 represent the relationship between the input and output during photoelectric conversion. The high gain 31 has a higher amplification factor than the low gain 32. The high gain 31 is an example of the "first conversion efficiency" of the present invention. The low gain 32 is an example of the "second conversion efficiency" of the present invention.

[0051] The imaging unit 50 is capable of high-gain sampling, which generates high-gain image data by photoelectric conversion at a high gain 31, and low-gain sampling, which generates low-gain image data by photoelectric conversion at a low gain 32. The high-gain image data is an example of the "first image data" of the present invention. The low-gain image data is an example of the "second image data" of the present invention. The high-gain sampling is an example of the "first imaging mode" of the present invention, and the low-gain sampling is an example of the "second imaging mode" of the present invention.

[0052] When DualGain is on, the system control unit 11 controls the imaging unit 50 to perform imaging using high-gain sampling and low-gain sampling to obtain high-gain image data and low-gain image data. That is, in the case of still image imaging, for example, the system control unit 11 controls the imaging unit 50 to perform imaging using high-gain sampling and low-gain sampling for the same pixels and generate one still image based on the obtained high-gain image data and low-gain image data. In addition, in the case of moving image imaging, the system control unit 11 controls the imaging unit 50 to perform imaging using high-gain sampling and low-gain sampling for the same pixels and generate one frame of image based on the obtained high-gain image data and low-gain image data.

[0053] Furthermore, when DualGain is off (single gain), the system control unit 11 controls the imaging unit 50 to perform only low-gain sampling out of high-gain sampling and low-gain sampling to acquire low-gain image data.

[0054] High-gain image data obtained by high-gain sampling has less noise, while low-gain image data obtained by low-gain sampling has a wide dynamic range. When DualGain is on, the system control unit 11 controls the imaging unit 50 to generate image data using the high-gain image data and low-gain image data. For example, the imaging unit 50 generates an image by mainly using low-gain image data with a wide dynamic range for bright parts of the captured image and mainly using high-gain image data with less noise for dark parts of the captured image.

[0055] <Gain Control> The gain can be controlled, for example, by providing multiple (e.g., two) floating diffusions with different capacitances in each pixel 61 and switching between the multiple floating diffusions. Alternatively, the gain may be controlled by providing multiple (e.g., two) floating diffusions in each pixel 61 and adjusting the total capacitance of the floating diffusions in the pixel 61 by switching between coupling and decoupling the multiple floating diffusions using a DCG (Dual Conversion Gain) transistor or the like.

[0056] As an example, a floating diffusion with a capacitance C1 and a floating diffusion with a capacitance C2 are provided in pixel 61, and by switching between coupling and non-coupling of these floating diffusions, the capacitance of the floating diffusion in pixel 61 can be switched between C1 and C1+C2, where C1>0 and C2>0.

[0057] In this case, the system control unit 11 switches the capacitance of the floating diffusion in the pixel 61 to C1 during high-gain sampling, and switches the capacitance of the floating diffusion in the pixel 61 to C1+C2 during low-gain sampling. As a result, if the amount of photoelectrically converted signal is Q, the voltage output from the floating diffusion in the pixel 61 is Q / C1 during high-gain sampling, and Q / (C1+C2) during low-gain sampling. Since C1<C1+C2, Q / C1>Q / (C1+C2). In other words, in the case of high-gain sampling, a larger voltage is output for the same amount of signal than in the case of low-gain sampling. The amount of gain fluctuation caused by switching from low-gain sampling to high-gain sampling is (C1+C2) / C1.

[0058] <Example of Controlling Dual Gain On / Off According to Region> Figure 4 is a diagram showing a first example of controlling Dual Gain on / off according to region. In Figure 4, within the imaging surface 60, the recording image pixel region 41 is an area of ​​recording image pixels used to generate an ornamental image (e.g., the normal pixels 61a in Figure 2), and the phase difference detection pixel region 42 is an area including phase difference detection pixels used for phase difference AF (e.g., the ranging pixels 61b and 61c in Figure 2). In this example, the phase difference detection pixel region 42 is arranged in line units on the imaging surface 60. The subject 60a is a subject (a person in this example) whose image is formed on the imaging surface 60.

[0059] For example, the system control unit 11 sets the recording image pixel area 41 as the first pixel area, and sets the phase difference detection pixel area 42 as the second pixel area. The first pixel area is an area where readout is performed with DualGain turned on, and the second pixel area is an area where readout is performed with DualGain turned off.

[0060] The system control unit 11 controls the imaging unit 50 to turn on DualGain and acquire high-gain image data and low-gain image data when imaging the recording image pixel area 41 set as the first pixel area, and also controls the imaging unit 50 to turn off DualGain and acquire only low-gain image data when imaging the phase difference detection pixel area 42 set as the second pixel area.

[0061] The first pixel region and the second pixel region are two regions set on the imaging surface 60 of one imaging element 5. However, each of the first pixel region and the second pixel region is not limited to being a single closed region of continuous pixels.

[0062] 4 , the system control unit 11 turns on DualGain for the recording image pixel area 41 that does not include pixels for phase difference detection used in phase difference AF, and turns off DualGain for the phase difference detection pixel area 42 that includes pixels for phase difference detection used in phase difference AF. Since the recording image pixel area 41 is used for displaying, for example, a through image, turning on DualGain can improve the signal-to-noise ratio in dark areas and thereby enhance the display image quality.

[0063] On the other hand, since the phase difference detection pixel region 42 is not used for display or is used for display at a low rate, turning off DualGain can reduce power consumption and improve readout speed. For example, when DualGain is turned off, the number of readouts is halved compared to when DualGain is turned on, so power consumption can be reduced by increasing blanking. Furthermore, when DualGain is turned off, the number of readouts is halved compared to when DualGain is turned on, so readouts are performed without intervals, so readout speed can be improved. Furthermore, improving the readout speed of the phase difference detection pixel region 42 allows phase difference calculation to start earlier, so AF performance (e.g., focusing speed) can be improved.

[0064] 5 is a diagram showing a second example of region-dependent on / off control of DualGain. In the example of Fig. 5, the system control unit 11 sets the phase difference detection pixel region 42 as the first pixel region and the recording image pixel region 41 as the second pixel region.

[0065] In this case, the system control unit 11 controls the imaging unit 50 to turn on DualGain and acquire high-gain image data and low-gain image data when imaging the phase difference detection pixel region 42 set as the first pixel region. Also, the system control unit 11 controls the imaging unit 50 to turn off DualGain and acquire only low-gain image data when imaging the recording image pixel region 41 set as the second pixel region.

[0066] As shown in the example of FIG. 5 , the system control unit 11 turns off DualGain for the recording image pixel area 41, which does not include phase difference detection pixels used for phase difference AF, and turns on DualGain for the phase difference detection pixel area 42, which includes phase difference detection pixels used for phase difference AF. For example, when the phase difference output is low and calculation accuracy is poor, or when the subject fluctuates significantly and phase difference calculation accuracy is poor, DualGain for the phase difference detection pixel area 42 may be turned on in this manner. This improves the accuracy of phase difference AF. In this case, the system control unit 11 also turns off the recording image pixel area 41. This improves the overall readout speed.

[0067] FIG. 6 is a diagram showing a third example of controlling the on / off of DualGain depending on the region. FIG. 7 is a diagram showing an example of an enlarged image. The digital camera 100 may have an enlargement display function for enlarging and displaying an image captured by a portion of the imaging surface 60. For example, the digital camera 100 displays an enlarged image 70 shown in FIG. 7 on the display surface 22b. The enlarged image 70 is an enlarged image of an image captured by a portion of the imaging surface 60 (e.g., the face portion of the subject 60a). In the example of FIG. 7, a reduced overall image 71 captured by the entire imaging surface 60 is superimposed and displayed at the lower right of the enlarged image 70.

[0068] 6 , on the imaging surface 60, an enlarged area 51 is an area of ​​pixels used to generate an enlarged image 70, and a non-enlarged area 52 is an area of ​​pixels not used to generate the enlarged image 70. In this example, on the imaging surface 60, a line area in which the face of the subject 60a is detected is set as the enlarged area 51.

[0069] For example, the system control unit 11 sets the enlarged region 51 as the first pixel region and the non-enlarged region 52 as the second pixel region. In this case, the system control unit 11 controls the imaging unit 50 to turn on DualGain and acquire high-gain image data and low-gain image data when imaging the enlarged region 51 set as the first pixel region. Furthermore, the system control unit 11 controls the imaging unit 50 to turn off DualGain and acquire only low-gain image data when imaging the non-enlarged region 52 set as the second pixel region.

[0070] 6 , when the enlarged area 51 and the non-enlarged area 52 are read out separately, DualGain may be turned on for the enlarged area 51 and turned off for the non-enlarged area 52. This improves the signal-to-noise ratio in the enlarged and displayed non-enlarged area 52 by using DualGain, making it possible to make noise less noticeable in the enlarged image 70. Furthermore, for the non-enlarged area 52 where noise is less noticeable, DualGain may be turned off to reduce power consumption and improve readout speed.

[0071] <Power Supply Control of AD Converter 65b in Accordance with On / Off of Dual Gain> Figure 8 is a diagram showing an example of power supply control of AD converter 65b when Dual Gain is on. In Figure 8, the horizontal axis represents the passage of time. As shown in Figure 8, when Dual Gain is on, the system control unit 11 controls the imaging unit 50 to alternately and repeatedly perform high gain sampling 81 and low gain sampling 82. In this case, the system control unit 11 controls the imaging unit 50 so that the power supply of the AD converter 65b is on (AD converter on) during both the high gain sampling 81 and the low gain sampling 82, for example.

[0072] 9 is a diagram illustrating an example of power control of the AD conversion unit 65b when DualGain is off. As shown in FIG. 9 , when DualGain is off, the system control unit 11 repeatedly performs, for example, only low-gain sampling 82, and controls the imaging unit 50 to enter a non-imaging state 83 in which no imaging (readout) is performed during the period in which high-gain sampling 81 is performed when DualGain is on. In this case, the system control unit 11 controls the imaging unit 50 to turn off the power to the AD conversion unit 65b (AD conversion unit off) during the non-imaging period 83. This reduces power consumption in the AD conversion unit 65b.

[0073] 10 is a diagram showing a first readout example when DualGain is on / off. As described in FIG. 8, when DualGain is on, the system control unit 11 controls the imaging unit 50 to alternately perform high-gain sampling and low-gain sampling.

[0074] 10 shows the readout of two images taken repeatedly during video or continuous imaging when DualGain is on. That is, a first AD conversion is performed based on the results of the first high-gain sampling and low-gain sampling, and a second AD conversion is performed based on the results of the second high-gain sampling and low-gain sampling.

[0075] Furthermore, when DualGain is off, the system control unit 11 controls the imaging unit 50 to repeatedly perform low-gain sampling, for example. The example in Fig. 10 shows the readout of four images captured repeatedly during video or continuous imaging when DualGain is off. That is, one AD conversion is performed based on each result of four low-gain samplings.

[0076] The CDS processing by the CDS processing unit 65a will be described. "Reset" and "Read" indicate the timing of the CDS processing by the CDS processing unit 65a. That is, the first sampling is performed at the timing of "Reset" before light hits the light receiving unit of the pixel 61, and the second sampling is performed at the timing of "Read" after light hits the light receiving unit of the pixel 61. Then, a signal from which noise has been removed is obtained by subtracting the first sampling result (reset noise) from the second sampling result (signal + noise). As shown in FIG. 10 , two samplings ("Reset" and "Read") are performed by CDS processing in both high-gain sampling and low-gain sampling.

[0077] The length of the high-gain sampling period may differ from the length of the low-gain sampling period. In the example of Fig. 10, the length of the high-gain sampling period is longer than the length of the low-gain sampling period. However, the length of the high-gain sampling period may also be shorter than the length of the low-gain sampling period.

[0078] 11 is a diagram showing a second readout example when DualGain is on / off. As shown in FIG. 11, the system control unit 11 may control the imaging unit 50 to repeatedly perform high-gain sampling when DualGain is off. The example in FIG. 11 shows the readout of four images captured repeatedly by video capture or continuous capture when DualGain is off. That is, one AD conversion is performed based on each result of four high-gain samplings.

[0079] As in this example, the high gain 31 may be an example of the "second conversion efficiency," and the low gain 32 may be an example of the "first conversion efficiency." In this case, the high gain image data is an example of the "second image data" of the present invention, and the low gain image data is an example of the "first image data" of the present invention. Also, in this case, the low gain sampling is an example of the "first imaging mode" of the present invention, and the high gain sampling is an example of the "second imaging mode" of the present invention.

[0080] 12 is a diagram showing a third readout example when DualGain is on / off. As shown in FIG. 12, the system control unit 11 may control the imaging unit 50 so that when DualGain is off, low-gain sampling is performed in accordance with the timing of high-gain sampling when DualGain is on. That is, in the example of FIG. 12, the timing of low-gain sampling when DualGain is off is different from the timing of low-gain sampling when DualGain is on.

[0081] Also, in this example, the timing of the CDS processing in low gain sampling when DualGain is off is made to match the timing of the CDS processing in high gain sampling when DualGain is on.

[0082] 13 is a diagram showing a fourth readout example when DualGain is on / off. As shown in FIG. 13, the system control unit 11 may control the imaging unit 50 so that, when DualGain is off, low-gain sampling is performed in accordance with the timing of low-gain sampling when DualGain is on. That is, in the example of FIG. 13, the timing of low-gain sampling when DualGain is off coincides with the timing of low-gain sampling when DualGain is on.

[0083] 14 is a diagram showing a fifth readout example when DualGain is on / off. As shown in FIG. 14, the system control unit 11 may control the imaging unit 50 to perform high-gain sampling when DualGain is off in accordance with the timing of high-gain sampling when DualGain is on. That is, in the example of FIG. 14, the timing of high-gain sampling when DualGain is off coincides with the timing of high-gain sampling when DualGain is on. In this way, the system control unit 11 may control the imaging unit 50 to perform only high-gain sampling, not low-gain sampling, when DualGain is off.

[0084] 15 is a diagram showing a sixth read example when DualGain is on / off. As shown in FIG. 15, when DualGain is on, a "Reset" and a "Read" of high-gain sampling may be performed between a "Reset" and a "Read" of low-gain sampling.

[0085] 15 , when DualGain is off, the system control unit 11 controls the imaging unit 50 to perform low-gain sampling at the same timing as low-gain sampling when DualGain is on. In this case, the system control unit 11 matches the timing of "Reset" and "Read" in low-gain sampling with the timing of "Reset" and "Read" in low-gain sampling when DualGain is on.

[0086] FIG. 16 is a diagram showing a seventh readout example when DualGain is on / off. In the example of FIG. 16, readout when DualGain is on is the same as the example of FIG. 15. As shown in FIG. 16, when DualGain is off, the system control unit 11 may control the imaging unit 50 to perform high-gain sampling in accordance with the timing of high-gain sampling when DualGain is on. In this case, the system control unit 11, for example, matches the timing of "Reset" and "Read" in high-gain sampling with the timing of "Reset" and "Read" in high-gain sampling when DualGain is on.

[0087] In the examples of Figures 15 and 16, a configuration has been described in which a "Reset" and "Read" of high-gain sampling are performed between a "Reset" and "Read" of low-gain sampling when DualGain is on, but a configuration in which a "Reset" and "Read" of low-gain sampling are performed between a "Reset" and "Read" of high-gain sampling when DualGain is on is also possible.

[0088] 13 to 16, the system control unit 11 may control the imaging unit 50 so that the timing of imaging in the second imaging mode for imaging the first pixel region (when DualGain is on) and the timing of imaging in the second imaging mode for imaging the second pixel region (when DualGain is off) correspond (substantially coincide). This makes it possible to suppress the effects of periodic disturbance noise (e.g., stripes appearing in the captured image). Note that the timing here refers to, for example, timing based on the horizontal synchronization signal for readout.

[0089] 10 and 11, the system control unit 11 may control the imaging unit 50 to repeatedly perform imaging in the first imaging mode and imaging in the second imaging mode when imaging the first pixel region (when DualGain is on), and to repeatedly perform imaging in the second imaging mode when imaging the second pixel region (when DualGain is off). This can improve the readout speed when imaging the second pixel region.

[0090] 12 to 16, the system control unit 11 may control the imaging unit 50 so that the timing of CDS processing (the timing of "Reset" and "Read" based on the synchronization signal) for imaging in the second imaging mode when imaging the second pixel region (when DualGain is off) corresponds to the timing of CDS processing for imaging in the second imaging mode when imaging the first pixel region (when DualGain is on). This makes it possible to reduce the effects of periodic disturbance noise.

[0091] <Reducing the Effects of Disturbance Noise> The effects of disturbance noise can be reduced by matching the timing of the CDS process (the timing of "Reset" and "Read") with the timing of disturbance noise (magnetic noise). For example, the system control unit 11 adjusts the drive timing of a component that applies disturbance noise to the image sensor 5 (such as a voice coil motor included in the image stabilization mechanism) to match the generation cycle of the disturbance noise with the cycle of the CDS process. As a result, in the examples shown in Figures 12 to 16, for example, the phase of the disturbance noise at the timing of the CDS process can be made constant, thereby reducing the effects of the disturbance noise.

[0092] 10, there is no high-gain sampling period when DualGain is off, so the timing at which disturbance noise occurs differs between the first pixel region where DualGain is on and the second pixel region where DualGain is off, which can increase the impact of disturbance noise.

[0093] In response to this, for example, as in the examples of Figures 12 and 13, by matching the timing of the CDS processing (the timing of "Reset" and "Read") with the timing of the disturbance noise, the influence of the disturbance noise can be reduced.

[0094] In the example of Figure 12, the timing of CDS processing matches when DualGain is on and when DualGain is off, so the level of disturbance noise is close, but the occurrence period of disturbance noise differs due to differences in the timing of AD conversion, so it may not be possible to sufficiently reduce the influence of disturbance noise.

[0095] In contrast, in the examples of Figures 13 to 16, not only the timing of CDS processing (the timing of "Reset" and "Read") but also the timing of AD conversion are matched, so the influence of disturbance noise can be further reduced.

[0096] The influence of disturbance noise is particularly large in a configuration in which a "Reset" and "Read" of high-gain sampling are performed between a "Reset" and "Read" of low-gain sampling when DualGain is on, as in the examples of Figures 15 and 16, or in a configuration in which a "Reset" and "Read" of low-gain sampling are performed between a "Reset" and "Read" of high-gain sampling when DualGain is on. In contrast, the influence of disturbance noise can be suppressed by synchronizing the timing of CDS processing when DualGain is off, as shown in Figures 15 and 16.

[0097] 17 is a diagram showing an example of reducing the influence of disturbance noise by adjusting the interval of CDS processing. The system control unit 11 may adjust the CDS period (the interval between "Reset" and "Read") for imaging with high gain sampling and low gain sampling so that the period corresponds to (substantially matches) an integer fraction of the period of periodic disturbance noise 171.

[0098] In the example of Fig. 17, the system control unit 11 controls the imaging unit 50 to repeatedly perform low-gain sampling when DualGain is off, as in the example of Fig. 10. Also, in the example of Fig. 17, the system control unit 11 sets the CDS period in high-gain sampling when DualGain is on, the CDS period in low-gain sampling when DualGain is on, and the CDS period in low-gain sampling when DualGain is off to a period that is an integer fraction of the period of the periodic disturbance noise 171.

[0099] For example, suppose the frequency of the disturbance noise 171 is 400 [kHz] (with a period of 2.5 [μs]). In this case, the system control unit 11 sets the frequency of the CDS processing in high-gain sampling when DualGain is on to 80 [kHz], i.e., the CDS period to 12.5 [μs] (five times the period of the disturbance noise 171). Furthermore, the system control unit 11 sets the frequency of the CDS processing in low-gain sampling when DualGain is on to 100 [kHz], i.e., the CDS period to 10 [μs] (four times the period of the disturbance noise 171). Furthermore, the system control unit 11 sets the frequency of the CDS processing in low-gain sampling when DualGain is off to 100 [kHz], i.e., the CDS period to 10 [μs] (four times the period of the disturbance noise 171).

[0100] This makes it possible to keep the phase of the periodic disturbance noise 171 constant at the timing of the CDS processing (the timing of "Reset" and "Read"). In the example of Fig. 17, the phase at which the intensity of the disturbance noise 171 is smallest is set at the timing of the CDS processing. However, the phase of the periodic disturbance noise 171 is not limited to the timing at which the intensity of the disturbance noise 171 is smallest at the timing of the CDS processing, and as long as the phase of the periodic disturbance noise 171 is constant at the timing of the CDS processing, that is, the intensity of the disturbance noise 171 is constant at the timing of the CDS processing, the influence of the disturbance noise 171 can be suppressed.

[0101] 17 , by setting the CDS period (the interval between "Reset" and "Read") to a period that is an integral fraction of the period of the periodic disturbance noise 171, the influence of disturbance noise can be suppressed in the same way as in a configuration in which the timing of CDS processing (the timing of "Reset" and "Read") is matched when DualGain is on and when it is off. This makes it possible to flexibly set the timing of CDS processing (the timing of "Reset" and "Read").

[0102] 18 is a diagram showing an example of readout when the period of the horizontal synchronization signal is not adjusted. The horizontal synchronization signal 181 is a signal that the drive circuit 64 generates for each pixel line on the imaging surface 60 in order to control the timing at which each pixel 61 on each pixel line on the imaging surface 60 is reset.

[0103] "PD reset" is the timing at which each pixel 61 in one pixel line is reset (the photodiode signal is discharged) based on the horizontal synchronization signal 181, i.e., the start timing of the exposure time. "Transfer completion" is the timing at which the transfer of the signal from the photodiode to the floating diffusion amplifier in the pixel 61 is completed, i.e., the end timing of the exposure time.

[0104] 18 , the frequency of the horizontal synchronization signal 181 when DualGain is on is twice as high as the frequency of the horizontal synchronization signal 181 when DualGain is off (+α). As a result, the exposure time differs between when DualGain is on and when DualGain is off. Therefore, the difference in exposure time between the first pixel region and the second pixel region can cause a phenomenon in which the brightness differs between the first pixel region and the second pixel region.

[0105] 19 is a diagram showing an example of readout when the period of the horizontal synchronization signal is adjusted. The system control unit 11 may control the period of the horizontal synchronization signal 181 when imaging the second pixel region (when DualGain is off) to correspond to (substantially match) N / M times (N and M are integers) the period of the horizontal synchronization signal 181 when imaging the first pixel region (when DualGain is on).

[0106] 19, the system control unit 11 sets the period of the horizontal synchronization signal 181 when DualGain is off to ⅓ of the period of the horizontal synchronization signal 181 when DualGain is on. In this case, during the period in which the horizontal synchronization signal 181 is generated once when DualGain is on, the horizontal synchronization signal 181 when DualGain is on is generated three times.

[0107] 19 , by setting the period of the horizontal synchronization signal 181 when DualGain is off to N / M times the period of the horizontal synchronization signal 181 when DualGain is on, it is possible to make the exposure time the same when DualGain is on and when DualGain is off. This makes it possible to suppress the phenomenon in which the brightness of an image differs between a first pixel region where DualGain is on and a second pixel region where DualGain is off.

[0108] (Variation 1) Although the configuration in which readout control in the imaging unit 50 is performed by the system control unit 11 has been described, readout control in the imaging unit 50 may be performed by a processor different from the system control unit 11. For example, readout control in the imaging unit 50 may be performed by a processor provided in the imaging unit 50.

[0109] (Variation 2) In the above embodiment, the case where the phase difference method is used as the distance measurement method (AF method) has been described, but the contrast method used in contrast AF may be used as the distance measurement method. Also, the distance measurement method may be a hybrid method that combines the phase difference method and the contrast method.

[0110] (Variation 3) The imaging device of the present invention is not limited to the digital camera 100 whose main purpose is to capture images, but can also be applied to various information terminals having an imaging function, such as smartphones, tablet terminals, notebook personal computers, etc. Next, the configuration of a smartphone 200, which is another embodiment of the imaging device of the present invention, will be described.

[0111] <Appearance of Smartphone 200> Fig. 20 shows the appearance of smartphone 200. Smartphone 200 shown in Fig. 20 has a flat housing 201, and is provided with a display input unit 204 on one surface of housing 201, in which a display panel 202 as a display unit and an operation panel 203 as an input unit are integrated.

[0112] The housing 201 also includes a speaker 205, a microphone 206, an operation unit 207, and a camera unit 208. The configuration of the housing 201 is not limited to this, and for example, it is also possible to adopt a configuration in which the display unit and the input unit are independent, or a configuration having a folding structure or a sliding mechanism.

[0113] <Configuration of Smartphone 200> FIG. 21 is a block diagram showing the configuration of the smartphone 200.

[0114] As shown in FIG. 21 , the main components of the smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a memory unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiving unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0115] The smartphone 200 also has, as its main function, a wireless communication function for performing mobile wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0116] The wireless communication unit 210 performs wireless communication with a base station device BS included in the mobile communication network NW in accordance with instructions from the main control unit 220. Using this wireless communication, various file data such as audio data and image data, e-mail data, etc. are sent and received, and web data, streaming data, etc. are received.

[0117] The display input unit 204 is a so-called touch panel that, under the control of the main control unit 220, displays images (still images and moving images) or text information, etc. to visually convey information to the user and detects user operations on the displayed information, and is equipped with a display panel 202 and an operation panel 203.

[0118] The display panel 202 uses a liquid crystal display (LCD), an organic electroluminescence display (OELD), or the like as a display device.

[0119] The operation panel 203 is a device placed so that an image displayed on the display surface of the display panel 202 can be seen, and detects one or more coordinates operated by a user's finger or a stylus. When this device is operated by the user's finger or a stylus, a detection signal generated by the operation is output to the main control unit 220. Next, the main control unit 220 detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0120] As shown in Figure 21, the display panel 202 and operation panel 203 of a smartphone 200, which is an example of one embodiment of the imaging device of the present invention, are integrated to form a display input unit 204, and the operation panel 203 is positioned so that it completely covers the display panel 202.

[0121] When such an arrangement is adopted, operation panel 203 may also have a function for detecting user operations in areas outside display panel 202. In other words, operation panel 203 may have a detection area for the overlapping portion that overlaps display panel 202 (hereinafter referred to as a display area), and a detection area for the remaining outer edge portion that does not overlap display panel 202 (hereinafter referred to as a non-display area).

[0122] The size of the display area and the size of the display panel 202 may be exactly the same, but they do not necessarily have to be the same. The operation panel 203 may also have two sensitive areas: an outer edge portion and an inner portion. The width of the outer edge portion is designed appropriately depending on the size of the housing 201, etc.

[0123] Furthermore, the position detection method used in the operation panel 203 may be a matrix switch method, a resistive film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, a capacitance method, or the like, and any method may be used.

[0124] The call unit 211 is equipped with a speaker 205 or a microphone 206, and converts the user's voice input through the microphone 206 into voice data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes voice data received by the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0125] Also, as shown in FIG. 20, for example, the speaker 205 can be mounted on the same surface as the display input unit 204, and the microphone 206 can be mounted on the side of the housing 201.

[0126] The operation unit 207 is a hardware key using a key switch or the like, and receives instructions from a user. For example, as shown in Fig. 20 , the operation unit 207 is a push-button switch mounted on the side surface of the housing 201 of the smartphone 200, which turns on when pressed with a finger or the like, and turns off when the finger is released due to the restoring force of a spring or the like.

[0127] The storage unit 212 stores the control program and control data of the main control unit 220, application software, address data associated with names or telephone numbers of communication partners, data of sent and received e-mails, web data downloaded by web browsing, downloaded content data, and also temporarily stores streaming data, etc. The storage unit 212 is composed of an internal storage unit 217 built into the smartphone and an external storage unit 218 having a removable external memory slot.

[0128] The internal memory unit 217 and the external memory unit 218 constituting the memory unit 212 are realized using storage media such as a flash memory type, a hard disk type, a multimedia card micro type, a card-type memory (e.g., MicroSD (registered trademark) memory, etc.), a RAM (Random Access Memory), a ROM (Read Only Memory), etc.

[0129] The external input / output unit 213 serves as an interface with all external devices connected to the smartphone 200, and is used to directly or indirectly connect to other external devices via communication (e.g., Universal Serial Bus (USB), Institute of Electrical and Electronics Engineers (IEEE) 1394, Bluetooth (registered trademark), RFID (Radio Frequency Identification), Infrared Data Association (IrDA) (registered trademark), UWB (Ultra Wideband) (registered trademark), ZigBee (registered trademark), etc.) or a network (e.g., Ethernet (registered trademark), wireless LAN (Local Area Network), etc.).

[0130] Examples of external devices connected to the smartphone 200 include a wired / wireless headset, a wired / wireless external charger, a wired / wireless data port, a memory card connected via a card socket, a SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) card, an external audio / video device connected via an audio / video I / O (Input / Output) terminal, a wirelessly connected external audio / video device, a wired / wirelessly connected smartphone, a wired / wirelessly connected personal computer, a wired / wirelessly connected personal computer, and earphones.

[0131] The external input / output unit 213 can transmit data received from such external devices to each component inside the smartphone 200, or transmit data inside the smartphone 200 to external devices.

[0132] The GNSS receiving unit 214 receives GNSS signals transmitted from the GNSS satellites ST1 to STn in accordance with instructions from the main control unit 220, executes positioning calculation processing based on the received multiple GNSS signals, and detects a position consisting of the latitude, longitude, and altitude of the smartphone 200. When the GNSS receiving unit 214 can acquire position information from the wireless communication unit 210 or the external input / output unit 213 (for example, a wireless LAN), it can also detect the position using the position information.

[0133] The motion sensor unit 215 includes, for example, a three-axis acceleration sensor, and detects the physical movement of the smartphone 200 in accordance with instructions from the main control unit 220. By detecting the physical movement of the smartphone 200, the direction of movement or acceleration of the smartphone 200 is detected. The detection result is output to the main control unit 220.

[0134] The power supply unit 216 supplies power stored in a battery (not shown) to each unit of the smartphone 200 in accordance with instructions from the main control unit 220 .

[0135] The main control unit 220 includes a microprocessor, operates according to the control program and control data stored in the storage unit 212, and controls all the units of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. The main control unit 220 also includes a mobile communication control function that controls all the units of the communication system to perform voice communication or data communication via the wireless communication unit 210, and an application processing function.

[0136] The application processing function is realized by the main control unit 220 operating in accordance with the application software stored in the storage unit 212. Examples of the application processing function include an infrared communication function that controls the external input / output unit 213 to perform data communication with a connected device, an email function that sends and receives emails, and a web browsing function that views web pages.

[0137] The main control unit 220 also has an image processing function for displaying video on the display input unit 204 based on image data (still image or moving image data) such as received data or downloaded streaming data.

[0138] The image processing function refers to a function in which the main control unit 220 decodes the image data, performs image processing on the decoded result, and displays the image on the display input unit 204 .

[0139] Furthermore, the main control unit 220 executes display control for the display panel 202 and operation detection control for detecting user operations via the operation unit 207 and the operation panel 203 .

[0140] By executing display control, the main control unit 220 displays software keys such as icons or scroll bars for launching application software, or displays a window for creating e-mail, etc. The scroll bar refers to a software key that accepts instructions to move the displayed portion of an image, such as a large image that cannot fit in the display area of ​​the display panel 202.

[0141] In addition, by executing operation detection control, the main control unit 220 detects user operations through the operation unit 207, accepts operations on the above icons and input of character strings into the input field of the above window through the operation panel 203, or accepts requests to scroll the displayed image through the scroll bar.

[0142] Furthermore, by executing operation detection control, the main control unit 220 determines whether the operation position on the operation panel 203 is an overlapping portion (display area) that overlaps the display panel 202 or an outer edge portion (non-display area) that does not overlap the display panel 202, and is equipped with a touch panel control function that controls the sensitive area of ​​the operation panel 203 or the display position of the software key.

[0143] The main control unit 220 can also detect a gesture operation on the operation panel 203 and execute a preset function in response to the detected gesture operation.

[0144] Gesture operation is not a simple touch operation as in the past, but rather refers to an operation of drawing a path with a finger or the like, specifying multiple positions simultaneously, or combining these to draw a path for at least one of multiple positions.

[0145] The camera section 208 includes the image capturing section 50 in the digital camera 100 shown in FIG.

[0146] The captured image data generated by the camera unit 208 can be stored in the storage unit 212 or output via the external input / output unit 213 or the wireless communication unit 210 .

[0147] In the smartphone 200 shown in FIG. 20 , the camera unit 208 is mounted on the same surface as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and it may be mounted on the back surface of the display input unit 204.

[0148] The camera unit 208 can also be used for various functions of the smartphone 200. For example, an image acquired by the camera unit 208 can be displayed on the display panel 202, or an image from the camera unit 208 can be used as one of the operation inputs for the operation panel 203.

[0149] Furthermore, when the GNSS receiving unit 214 detects a position, it can also detect the position by referring to an image from the camera unit 208. Furthermore, it can also refer to an image from the camera unit 208 to determine the optical axis direction of the camera unit 208 of the smartphone 200 or determine the current usage environment without using a triaxial acceleration sensor or by using the image in combination with a triaxial acceleration sensor. Of course, the image from the camera unit 208 can also be used in application software.

[0150] In addition, image data of still images or videos can be added with location information acquired by the GNSS receiving unit 214, audio information acquired by the microphone 206 (which may be converted to text information by the main control unit, etc.), posture information acquired by the motion sensor unit 215, etc., and stored in the memory unit 212 or output via the external input / output unit 213 or wireless communication unit 210.

[0151] In the smartphone 200 configured as described above, as in the digital camera 100, it is possible to improve the readout efficiency.

[0152] Although various embodiments have been described above, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention. Furthermore, the components of the above embodiments may be combined in any manner as long as they do not deviate from the spirit of the invention.

[0153] This application is based on a Japanese patent application (Patent Application No. 2024-030162) filed on February 29, 2024, the contents of which are incorporated herein by reference.

[0154] The present invention is highly convenient and effective when applied to digital cameras and the like.

[0155] REFERENCE SIGNS LIST 1 imaging lens 2 aperture 4 lens control unit 5 imaging element 8 lens driving unit 9 aperture driving unit 10 imaging element driving unit 11 system control unit 14, 207 operation unit 15 memory control unit 16 memory 17 digital signal processing unit 20 external memory control unit 21 recording medium 22 display device 22a display controller 22b display surface 24 control bus 25 data bus 31 high gain 32 low gain 40 lens device 41 pixel area for recording image 42 pixel area for phase difference detection 50 imaging unit 51 magnification area 52 non-magnification area 60 imaging surface 60a subject 61 pixel 61a normal pixel 61b, 61c ranging pixel 62, 63 pixel line 64 driving circuit 65 signal processing circuit 65a CDS processing unit 65b AD conversion unit 70 Enlarged image 71 Whole image 81 High gain sampling 82 Low gain sampling 83 Non-imaging 100 Digital camera 100A Main body 171 Disturbance noise 181 Horizontal synchronization signal 200 Smartphone 201 Housing 202 Display panel 203 Operation panel 204 Display input unit 205 Speaker 206 Microphone 208 Camera unit 210 Wireless communication unit 211 Call unit 212 Memory unit 213 External input / output unit 214 GNSS receiving unit 215 Motion sensor unit 216 Power supply unit 217 Internal memory unit 218 External memory unit 220 Main control unit

Claims

1. An imaging device having a processor and a first imaging mode in which first image data is generated by photoelectric conversion at a first conversion efficiency, and a second imaging mode in which second image data is generated by photoelectric conversion at a second conversion efficiency different from the first conversion efficiency, wherein the processor sets a first pixel region and a second pixel region, and when imaging the first pixel region, performs imaging in the first imaging mode and the second imaging mode to obtain the first image data and the second image data, and when imaging the second pixel region, performs imaging in the second imaging mode of the first imaging mode and the second imaging mode to obtain the second image data.

2. An imaging device according to claim 1, wherein the first pixel area is an area of ​​pixels for recording an image, and the second pixel area is an area of ​​pixels for detecting a phase difference.

3. An imaging device according to claim 1, wherein the first pixel region is a region corresponding to a specific subject, and the second pixel region is a region different from the first pixel region.

4. An imaging device according to claim 1, wherein the processor, when imaging the first pixel region, alternates between imaging in the first imaging mode and imaging in the second imaging mode, and, when imaging the second pixel region, alternates between not imaging and imaging in the second imaging mode.

5. An imaging device according to claim 4, wherein the processor turns off the power supply to a conversion circuit that converts analog imaging signals into digital signals during the non-imaging period.

6. An imaging device according to claim 4, wherein the processor matches the timing of imaging in the second imaging mode in imaging the first pixel region with the timing of imaging in the second imaging mode in imaging the second pixel region.

7. An imaging device according to claim 1, wherein the processor, when imaging the first pixel region, repeatedly performs imaging in the first imaging mode and imaging in the second imaging mode, and when imaging the second pixel region, repeatedly performs imaging in the second imaging mode.

8. An imaging device according to claim 7, wherein the processor causes the timing of correlated double sampling for imaging in the second imaging mode in imaging the second pixel region to correspond to the timing of correlated double sampling for imaging in the second imaging mode in imaging the first pixel region.

9. An imaging device according to claim 7, wherein the processor sets the interval of correlated double sampling for imaging in the first imaging mode and the second imaging mode to correspond to a period that is an integer fraction of the period of periodic external disturbance noise.

10. An imaging device according to claim 1, wherein the processor corresponds the period of the horizontal synchronization signal in imaging the second pixel region to N / M times the period of the horizontal synchronization signal in imaging the first pixel region, where N and M are integers.

11. An imaging device according to any one of claims 1 to 10, wherein the processor generates a portion of the captured image corresponding to the first pixel region based on the first image data and the second image data, and generates a portion of the captured image corresponding to the second pixel region based on the second image data.

12. A control method for an imaging device having a processor and a first imaging mode in which first image data is generated by photoelectric conversion at a first conversion efficiency, and a second imaging mode in which second image data is generated by photoelectric conversion at a second conversion efficiency different from the first conversion efficiency, wherein the processor sets a first pixel region and a second pixel region, and when imaging the first pixel region, performs imaging in the first imaging mode and the second imaging mode to obtain the first image data and the second image data, and when imaging the second pixel region, performs imaging in the second imaging mode of the first imaging mode and the second imaging mode to obtain the second image data.

13. A control program for an imaging device having a processor and a first imaging mode in which first image data is generated by photoelectric conversion at a first conversion efficiency, and a second imaging mode in which second image data is generated by photoelectric conversion at a second conversion efficiency different from the first conversion efficiency, the control program causing the processor to execute the following processes: setting a first pixel region and a second pixel region; imaging the first pixel region by performing imaging in the first imaging mode and the second imaging mode to obtain the first image data and the second image data; and imaging the second pixel region by performing imaging in the second imaging mode of the first imaging mode and the second imaging mode to obtain the second image data.

Images