Image capture device and control method thereof

Abstract

To take still images while suppressing deterioration of framing performance, by appropriately controlling shooting timings and correction timings of still images and LV images.SOLUTION: The image capturing apparatus includes: actuating means that actuates an image sensor; storage means that stores an image signal read out from the image sensor; image processing means that processes the image signal; and control means that controls the actuating means, the storage means, and the image processing means by selectively using a plurality of operation modes. The plurality of operation modes include a first operation mode of performing, at a predetermined first cycle, a first operation in which a first image signal is read out from the image sensor and processed by the image processing means; and selectively performing a second operation in which a second image signal is read out from the image sensor at a timing different from that of the first operation and stored in the storage means, or a third operation in which the stored second image signal is read out from the storage means and processed by the image processing means.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus and a control method thereof, and more particularly to timing control of capturing still images and display images and correcting the images. [Background technology]

[0002] Some digital cameras and electronic devices equipped with a camera function (hereinafter collectively referred to as "imaging devices") have a live view (LV) function. The LV function is a function that captures moving images and sequentially displays the resulting frame images (hereinafter referred to as "LV images") on the display unit in parallel with the shooting. The user can check and adjust the shooting range for still image shooting and video shooting using the displayed LV image. Hereinafter, this operation of adjusting the shooting range will be referred to as "framing."

[0003] Ease of framing is important for users to take pictures as they intend. An index of this ease of framing is called "framing performance." For example, if there is a long time between taking an LV image and displaying it on the display (display delay), it may happen that the subject has already moved by the time it is displayed on the display. In such a case, framing performance will decrease. In this way, display delay plays a major role in framing performance.

[0004] In an imaging device that uses the LV function to perform framing and shoot still images, framing becomes impossible unless the LV images are continuously displayed on the display unit even during continuous shooting of still images. Therefore, in order to continuously shoot still images while successively displaying the LV images, it is necessary to switch between shooting still images and LV images.

[0005] After shooting, both still images and LV images need to be corrected, but because still images have a large amount of data, the correction process takes time. . PhotographyIf the correction process is performed in the order of the shots, it is expected that the correction process for the still images will be performed even while the LV images are being captured. As a result, the correction process for the LV images will be delayed, and the time it takes from capturing the LV images to displaying them on the display unit will be longer, resulting in a large display delay for the LV images and a decrease in framing performance.

[0006] In response to this problem, Patent Document 1 discloses a method in which still image data and video data are input and processed in an image processing device, in which the video data is displayed at a predetermined cycle while dividing still images and correcting them over several frames during the processing of the video data.

[0007] Furthermore, Patent Document 2 discloses a method in which, from multiple images obtained by continuous shooting, one image is selected, development parameters are determined, and the multiple images obtained by continuous shooting are developed using the same parameters. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Patent No. 5251684 [Patent Document 2] JP 2004-80230 A Summary of the Invention [Problem to be solved by the invention]

[0009] However, the conventional techniques disclosed in the above-mentioned Patent Documents 1 and 2 cannot capture LV images without degrading framing performance while capturing still images. In addition, the still images must be corrected during blanking of LV image capture by the number of divisions into which the still image is divided, which causes a problem of a decrease in the still image capture frame rate.

[0010] The present invention has been made in consideration of the above problems, and aims to capture still images while suppressing degradation of framing performance by appropriately controlling the shooting timing and correction timing of still images and LV images. [Means for solving the problem]

[0011] In order to achieve the above object, an imaging apparatus of the present invention comprises a driving means for driving an imaging element, a storage means for storing an image signal read out from the imaging element, an image processing means for processing the image signal read out from the imaging element, and a control means for controlling the driving means, the storage means, and the image processing means by selectively using a plurality of driving modes, the plurality of driving modes comprising a first operation in which the driving means reads out a first image signal from the imaging element and processes the first image signal by the image processing means without storing the first image signal in the storage means, and a second operation in which the driving means reads out a second image signal from the imaging element and stores it in the storage means before performing the first operation, and after performing the first operation, reads out the stored second image signal from the storage means and processes it by the image processing means. a second driving mode in which the first operation and a third operation are performed in which the driving means reads out the second image signal from the imaging element at a timing different from that of the first operation, and the second image signal is processed by the image processing means without being stored in the storage means; Including and the control means selects either the first driving mode or the second driving mode in accordance with a readout speed of the second image signal. . Effect of the Invention

[0012] According to the present invention, by appropriately controlling the shooting timing and correction timing of a still image and an LV image, it is possible to shoot a still image while suppressing degradation of framing performance. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing a configuration of an imaging apparatus according to first to third embodiments of the present invention. [Diagram 2] FIG. 3A is a circuit diagram of a unit pixel according to the first and third embodiments, and FIG. 3B is a block diagram showing a circuit configuration of an image sensor according to the first to third embodiments. [Diagram 3]FIG. 2 is a block diagram showing a configuration of a timing pulse generating circuit according to the first to third embodiments. [Figure 4] 5 is a timing chart in a first driving mode in the first embodiment. [Diagram 5] 5 is a timing chart according to a second driving mode in the first embodiment. [Figure 6] 5 is a flowchart showing control for switching between shooting modes in the first embodiment. [Figure 7] FIG. 7A is a top view showing a schematic configuration of an image sensor according to a second embodiment, and FIG. 7B is a circuit diagram of a unit pixel according to the second embodiment. [Figure 8] 10 is a timing chart according to a third driving mode in the second embodiment. [Figure 9] 10 is a timing chart according to a fourth driving mode in the second embodiment. [Figure 10] 10 is a flowchart showing control for switching between shooting modes in the second embodiment. [Figure 11] 10 is a timing chart according to a fifth driving mode in the third embodiment. [Figure 12] 13 is another timing chart according to the fifth driving mode in the third embodiment. [Figure 13] 10 is a timing chart in a sixth driving mode in the third embodiment. [Figure 14] 10 is a flowchart showing control for switching between shooting modes in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0015] Fig. 1 is a block diagram showing the configuration of an imaging device 1 according to an embodiment of the present invention. In Fig. 1, a first lens 100 is disposed at the tip of an imaging optical system 104. An aperture 101 is driven by an aperture actuator 122, and adjusts the amount of light during imaging by adjusting its aperture diameter. A second lens 102 and a third lens 103 are driven by a focus actuator 120, and adjust the focus of the imaging optical system 104 by moving them forward and backward in the optical axis direction.

[0016] The focal plane shutter 105 has a function of adjusting the exposure time when capturing a still image. However, when an electronic shutter is used to adjust the exposure time in slit rolling readout in the image sensor 107, the exposure time is not adjusted by the focal plane shutter 105. The optical low-pass filter 106 is used to reduce false colors and moire in the captured image.

[0017] The image sensor 107 photoelectrically converts an optical image of a subject formed by the photographing optical system 104 into an electric signal (image signal). The image sensor 107 also has an electronic shutter function and can obtain an image signal by slit rolling readout.

[0018] The DSP 108 receives an image signal from the image sensor 107 and performs predetermined image processing. In addition to image processing, the DSP 108 also performs calculations of information used to drive the focus lenses (the second lens 102 and the third lens 103) based on information from the image sensor 107.

[0019] The RAM 109 has both a function of storing image data that has been subjected to a predetermined image processing by the DSP 108 and a function as a work memory when the CPU 110 operates. Note that other types of memory can be used as long as the access speed is sufficiently fast and there is no operational problem. In this embodiment, the RAM 109 is disposed outside the DSP 108 and the CPU 110, but a configuration in which some or all of the functions are built into the DSP 108 and the CPU 110 may also be used.

[0020] The CPU 110 executes a program for controlling each unit of the imaging device 1, and performs overall control of the operation of the imaging device 1. The CPU 110 can control reading from the imaging element 107 by making various settings in the imaging element 107. The CPU 110 also communicates with a timing pulse generating circuit 111, and controls the generation timing of various timing pulses for controlling the operation of each module, as described below.

[0021] The timing pulse generating circuit 111 generates a plurality of synchronous signals based on a clock signal CLK1 from a first clock 112 and a clock signal CLK2 from a second clock 113, and supplies the signals to the image sensor 107 and the display unit 114. The image sensor 107 operates in synchronization with the signals from the timing pulse generating circuit 111 to perform an image capturing operation.

[0022] Moreover, the display unit 114 updates the display image by operating in synchronization with a signal from the timing pulse generating circuit 111. Note that, in this embodiment, a configuration is adopted in which a plurality of clocks are input to the timing pulse generating circuit 111 to generate two types of synchronization signals, but the present invention is not limited to this configuration. Furthermore, the CPU 110 also has a function of controlling the focus actuator 120 using the calculation result output from the DSP 108, and adjusting the focus of the photographing optical system 104.

[0023] The display unit 114 displays still images, moving images, menus, etc. generated by the DSP 108. The user can adjust the framing while checking the shooting range of still images and moving images by using the live view (LV image) displayed successively on the display unit 114. Besides a rear display, an electronic viewfinder (EVF) is also known as the display unit 114. Although FIG. 1 shows a configuration with only one display unit 114, a configuration with both a rear display and an EVF may also be used.

[0024] Operation unit 115 is made up of operation members such as buttons and levers, and includes a mode switching dial, a still image shooting button for issuing an instruction to shoot a still image, etc. Note that operation unit 115 may be configured not to have operation members, but to input various instructions via a touch panel using a rear display. Also, a user can take pictures by controlling CPU 110 through operation unit 115.

[0025] The CPU 110 receives a still image taken by the user. Shadow button When the button is pressed, a certain period of time including a preparation period for still image shooting elapses, and then a still image is shot. If the still image shooting button is pressed continuously after still image shooting has started, continuous shooting of still images is performed. The detailed operation of still image shooting will be described later in detail using a timing chart.

[0026] The recording medium 116 is configured to be removable, and still image data and moving image data are recorded in it. The ROM 117 stores a program for the CPU 110 to control the operation of each unit.

[0027] The shutter driving circuit 118 drives and controls the focal plane shutter 105. The focus driving circuit 119 controls the focus actuator 120 based on the output of the CPU 110 to drive the focus lenses (the second lens 102 and the third lens 103) forward and backward in the optical axis direction to perform focus adjustment to change the focal position. The aperture driving circuit 121 controls the aperture actuator 122 to control the opening of the aperture 101.

[0028] Next, the configuration of the image sensor 107 of this embodiment will be described with reference to FIG. 2(a) is a diagram for explaining a detailed circuit configuration of a unit pixel 206 of the image sensor 107. A photodiode (PD) 200 photoelectrically converts incident light and accumulates the generated charge. A transfer switch 201 is controlled by a control signal φtx, and transfers the charge accumulated in the PD 200 to a floating diffusion (FD) 202 by setting the control signal φtx to High (hereinafter, referred to as "H").

[0029] The reset switch 203 is a switch for initializing the FD 202, and is controlled by a control signal φres. The pixel reset operation is realized by simultaneously setting the control signals φtx and φres to H and setting the PD 200 and the FD 202 to the power supply voltage (VDD). When the control signal φsel of the selection switch 205 becomes H, the pixel amplifier transistor 204 is connected to a constant current source 209 via a vertical output line 208 to form a pixel amplifier. Then, the charge transferred from the PD 200 to the FD 202 is converted by the pixel amplifier into a voltage value according to the amount of charge, and is output to the vertical output line 208 as an image signal.

[0030] Next, the circuit configuration of the image sensor 107 will be described with reference to FIG. In the pixel array 207, a plurality of unit pixels 206 are arranged in a matrix, with (m+1) units in the horizontal direction (row direction) and (n+1) units in the vertical direction (column direction). Note that both m and n are natural numbers. The driving pulse generating circuit 210 generates a pulse signal for performing a reset operation or a read operation of the unit pixels 206 based on a synchronization signal from the timing pulse generating circuit 111.

[0031] The pulse signal generated by the drive pulse generation circuit 210 is supplied to the pixel drive circuit 212. The row selection circuit 211 selects a row to which the pulse signal generated by the drive pulse generation circuit 210 is supplied, and sets the selected row in the pixel drive circuit 212. The pixel drive circuit 212 supplies the pulse signal generated by the drive pulse generation circuit 210 to the row set by the row selection circuit 211 as the above-mentioned control signal.

[0032] An image signal is output to a vertical output line 208 from the unit pixel 206 of a row selected in response to a control signal supplied from a pixel drive circuit 212. A constant current source 209 is combined with the pixel amplifier transistor 204 to form a source follower circuit.

[0033] An AD conversion circuit (ADC) 213 converts the analog image signal (voltage value) output to the vertical output line 208 into a digital value corresponding to the signal level. The image signals converted into digital values ​​by the ADC 213 are sequentially selected by a horizontal scanning circuit 214 and transferred to an output unit 215.

[0034] It should be noted that image signals can be read out in a number of different readout methods by changing the driving of the pixel array 207 by the row selection circuit 211. For example, in this embodiment, the readout method for generating a still image is a method in which image signals are read out from the pixels in the top row, and then image signals are read out from the pixels in the next row, and this is repeated until the bottom row is reached.

[0035] In addition, the readout method used when generating moving images is to read out image signals from the pixels in the top row, then read out image signals from pixels in rows that are skipped a few rows (for example, two rows), and repeat this process until the pixels in the bottom row. Reading out image signals in this manner when generating moving images reduces the vertical resolution of the image, but allows one frame of image signals to be read out in a short time with low power consumption. There are also various other readout methods that can be used to read out image signals.

[0036] In this embodiment, an example has been described in which the pixel driving circuit 212 and the ADC 213 are built into the image sensor 107, but these circuits may be provided on a chip separate from the image sensor 107.

[0037] Next, the internal configuration of the timing pulse generating circuit 111 will be described with reference to Fig. 3. In this embodiment, the timing pulse generating circuit 111 is composed of a reference synchronization signal generating circuit 300 (hereinafter referred to as "reference SSG"), a second synchronization signal generating circuit 301 (hereinafter referred to as "second SSG"), and a third synchronization signal generating circuit 302 (hereinafter referred to as "third SSG").

[0038] The reference SSG 300 and the second SSG 301 are connected to the CPU 110, and the time of the assertion timing of each synchronization signal can be recorded. From this time, it is possible to calculate the time difference between the assertion timing of the reference SSG 300 and the second SSG 301. Furthermore, by setting each of these circuits from the CPU 110, it is possible to generate synchronization signals with various cycles, and also to change the generation timing of the synchronization signal.

[0039] The second SSG 301 outputs a second synchronization signal, and based on this second synchronization signal, the timing of the readout scanning of the image sensor 107 is controlled. Also, the third SSG 302 outputs a third synchronization signal, and based on this third synchronization signal, the timing of the display scanning of the display unit 114 is controlled.

[0040] In this configuration, the second SSG 301 operates based on a clock signal CLK1 from the first clock 112, and the reference SSG 300 and the third SSG 302 operate based on a clock signal CLK2 from the second clock 113. However, the present invention is not limited to this configuration. All the SSGs may operate based on a single clock signal, or may operate based on different clock signals. In addition, all or some of the above SSGs may form independent blocks.

[0041] <First embodiment> A first embodiment of the present invention will be described below. In this first embodiment, a case will be described in which still images are continuously shot while an LV image is displayed at different timings in two drive modes, a first drive mode and a second drive mode.

[0042] 4 is a timing chart showing the shooting operation in the first driving mode in the first embodiment. This process is performed in response to a shooting instruction issued by, for example, a user pressing a still image shooting button provided on the operation unit 115. In the figure, the solid line represents a readout scan of a still image image signal from the image sensor 107, the dashed line represents a readout scan of an LV image signal, and the dashed line represents a reset scan of the image sensor 107. Also, the readout of the still image image signal is performed at a very high speed, and it takes a longer time for the image correction circuit built into the DSP 108 to correct the still image image signal than it takes to read the still image signal.

[0043] At time t400, the second SSG 301 of the timing pulse generating circuit 111 asserts a second synchronization signal (vertical synchronization signal (VD)), and in synchronization therewith, the imaging element 107 starts reading out the image signal for a still image. It is assumed that reset scanning of the imaging element 107 starts before time t400 by an exposure time (charge accumulation time) determined in advance by photometry or a user instruction. Then, simultaneously with the start of reading out the image signal for a still image, the read out image signal for a still image is transferred to the DSP 108, and storage in the RAM 109 connected to the DSP 108 starts. When the reading out of the image signal for a still image ends at time t401, the storage of the image signal for a still image in the RAM 109 also ends.

[0044] At time t402, the timing pulse generating circuit 111 asserts the second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out the LV image signal. Note that reset scanning of the image sensor 107 starts before time t402, an exposure time (charge accumulation time) that is determined in advance by photometry or a user instruction. The exposure time at this time is set to time TLV The time required for the LV image signal to be read out is shorter than the time required for the LV image signal to be read out. The LV image signal that has been read out is then transferred to the DSP 108, and is corrected by an image correction circuit included in the DSP 108. At this time, the image signal for a still image that has been read out between time t400 and t401 is stored in the RAM 109 without being corrected. At time t403, the reading of the LV image signal is completed, and the correction process in the image correction circuit is also completed. A live view image generated based on the LV image signal that has been corrected is displayed on the display unit 114.

[0045] Next, at time t404, the still image signal stored in the RAM 109 is read out and transferred to the image correction circuit, and correction processing of the still image signal is started. This correction processing is completed at time t405. As described above, the processing speed of the image correction circuit is slower than the readout speed of the still image signal, so the time from time t404 to time t405 is longer than the time from time t400 to time t401.

[0046] At time t406, the timing pulse generation circuit 111 asserts a second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out the LV image signal of the next frame. Note that reset scanning of the image sensor 107 starts a predetermined exposure time before this time t406, so that the correction process of the image signal for a still image by the image correction circuit is completed when the reading out of the LV image signal starts at time t406. Then, the read out LV image signal is transferred to the DSP 108, and is corrected by the image correction circuit included in the DSP 108.

[0047] At the following time t408, the timing pulse generating circuit 111 asserts a second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out the image signal for a still image. The timing of this time t408 is determined by the following conditions. That is, after the start of reading out the LV image signal at time t406 enables reset scanning for charge accumulation for generating a still image, the exposure time (charge accumulation time) T STThe still image signals that have been read out are transferred to the DSP 108 and stored in sequence in the RAM 109 connected to the DSP 108.

[0048] After t409, the same control as that from time t402 to t409 is repeated. In this way, by repeating the control from time t402 to time t409, it is possible to continuously shoot still images while acquiring LV images at a video cycle. Also, control is performed so that the time from time t402 to time t406 is equal to the time from time t406 to time t409.

[0049] In this way, by controlling in the first drive mode, it is possible to acquire LV images at a constant cycle while still images are being shot. By sequentially displaying these LV images on the display unit 114, when the readout speed of the image signal for still images is faster than the correction speed of the image correction circuit, it becomes possible to shoot and correct LV images while shooting and correcting still images, and display them on the display unit 114 at a constant cycle.

[0050] Next, the shooting operation in the second driving mode will be described with reference to the timing chart of FIG. This process is performed, for example, in response to a shooting instruction issued by a user pressing a still image shooting button provided on operation unit 115. Note that the solid lines, dashed dotted lines, and broken lines in the figure are the same as those in Fig. 4, and therefore their explanations are omitted. The readout speed of the image signal for a still image in the second driving mode is assumed to be approximately the same as the time it takes for the image correction circuit built into DSP 108 to correct the image signal for a still image.

[0051] At time t500, the timing pulse generation circuit 111 asserts a second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out the LV image signal. It is assumed that reset scanning of the image sensor 107 starts before time t500 by an exposure time (charge accumulation time) that is predetermined by photometry or a user instruction. Then, simultaneously with the start of reading out the LV image signal, the read out LV image signal is transferred to the DSP 108 and corrected by an image correction circuit included in the DSP 108. At time t501, reading out of the LV image signal is completed, and at the same time, the correction process of the LV image signal in the image correction circuit is also completed.

[0052] At time t502, the timing pulse generation circuit 111 asserts a second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out an image signal for a still image. The timing of this time t502 is determined by the following conditions. That is, the image correction circuit becomes available at time t501, and reset scanning for charge accumulation for generating a still image becomes possible with the start of reading out an image signal for LV at time t500. Therefore, after reset scanning is performed at or after time t500, exposure time (charge accumulation time) T ST The timing after time t501 is the timing to start reading out the image signal for a still image, time t502. ST is longer than the period from time t500 to time t501. At the same time as the start of readout of the still image signal, the read out still image signal is transferred to the DSP 108, and correction is started by the image correction circuit included in the DSP 108. At time t503, the readout of the still image signal is completed, and at the same time, the correction process of the still image signal in the image correction circuit is also completed.

[0053] At the next time t504, the timing pulse generating circuit 111 asserts a second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out the LV image signal. Note that reset scanning of the image sensor 107 starts before time t504, an exposure time (charge accumulation time) that is determined in advance by photometry or a user instruction. The exposure time at this time is set to a time that takes into account the timing of the end of reading out the still image signal. LV The time will be shorter than

[0054] By repeating the above-mentioned control from time t500 to time t504, it is possible to continuously capture still images while acquiring LV images in a video cycle. The time from time t500 to time t504 is set as the video cycle, and control is continued so that LV images can be captured at this interval.

[0055] In this way, by controlling in the second driving mode, it is possible to acquire LV images at a constant cycle while shooting still images. By sequentially displaying these LV images on the display unit 114, it becomes possible to shoot and correct the LV images to be displayed on the display unit 114 when the readout speed of the image signal for still images is equal to or slower than the correction speed of the image correction circuit.

[0056] Note that, even if the readout speed of the image signal for still images is equal to or slower than the correction speed of the image correction circuit, the data may be stored in the RAM 109 and correction processing may be performed later as described with reference to Fig. 4, but this puts a strain on the bandwidth of the RAM 109 and increases power consumption. Therefore, by performing the control shown in Fig. 5, it is possible to eliminate these disadvantages.

[0057] 4 and 5, charge accumulation is started by reset-scanning the image sensor 107, but the present invention is not limited to this. For example, after reset-scanning the image sensor 107, the front curtain of the focal plane shutter 105 may be driven to expose the image sensor 107 to light, thereby starting charge accumulation.

[0058] When shooting is performed in the first drive mode shown in Fig. 4, it is possible to correct both the still image and the LV image even when the readout speed of the image signal for a still image is fast, but this puts a strain on the bandwidth of the RAM 109 and increases the power consumption of the RAM 109. On the other hand, when shooting is performed in the second drive mode shown in Fig. 5, it is possible to correct both the still image and the LV image without consuming the bandwidth of the RAM 109 or increasing the power consumption when the readout speed of the image signal for a still image is equal to or lower than the processing speed of the image correction circuit. Therefore, by switching between the two processing methods depending on the readout speed of the still image, it is possible to make the most of the advantages of each drive method.

[0059] FIG. 6 is a flowchart illustrating a method of switching between the shooting operation in the first driving mode shown in FIG. 4 and the shooting operation in the second driving mode shown in FIG.

[0060] When still image shooting is set, the process of Fig. 6 starts. In S601, the user selects a still image shooting mode via the operation unit 115. Note that the method for selecting the shooting mode may be explicit or implicit, and for example, the user may be provided with a shutter method as a choice.

[0061] In S602, it is determined whether the selected shooting mode is suitable for shooting in the first drive mode or the second drive mode. Here, it is determined whether the read speed of the image signal for still images in the selected shooting mode is faster than the correction processing speed by the image correction circuit. If it is faster, the process proceeds to S603, and if it is equal to or lower than the correction processing speed, the process proceeds to S604.

[0062] Note that, instead of determining the speed in S602, information linking the shooting mode with the first drive mode or the second drive mode may be stored in advance and used for the determination. Also, as described above, when the shooting mode is selected implicitly, for example, by the shutter method, it is possible to make the following determination. That is, when the mechanical front curtain method that controls the exposure time using the focal plane shutter 105 is selected, it takes time to start scanning the image sensor 107 to accumulate electric charge, so it is determined that the second drive mode is suitable. On the other hand, when the electronic shutter is selected, it is possible to scan the image sensor 107 at high speed, so it is determined that the first drive mode is suitable.

[0063] In S603, a program for the sequence in the first drive mode shown in Fig. 4 is loaded, and in S604, a program for the sequence in the second drive mode shown in Fig. 5 is loaded, and the process ends. After that, shooting is performed according to the loaded program.

[0064] In the flowchart of FIG. 6, the drive mode of the image sensor 107 is changed by the user's operation in S601, but the method of changing the drive mode is not limited to this. For example, the shooting mode of the still image to be shot in the next frame may be changed according to various conditions calculated from the shot image, and the drive mode may be changed accordingly. In this case, the process from S602 onwards is performed in the same manner as the process described above, so that the drive mode of the image sensor 107 can be switched according to the shooting conditions. Then, in S60 3 or S604 Once the drive mode switching has been completed, the image sensor 107 can be driven in the switched drive sequence.

[0065] As described above, according to the first embodiment, the first driving mode and the second driving mode are selectively used. This makes it possible to acquire LV images at a constant cycle while shooting still images, regardless of the relationship between the readout speed of the image signal for still images and the correction processing speed of the image correction circuit. 。

[0066] <Second embodiment> Next, a second embodiment of the present invention will be described. In the above-mentioned first embodiment, a method of switching an imaging sequence according to a readout speed of a still image was described. In the first embodiment, one image was obtained by one exposure of an image signal for a still image. However, some imaging elements can obtain multiple images by amplifying an image obtained by one exposure with different gains. Therefore, in the second embodiment, a control of a shooting operation in the case of obtaining multiple images amplified with different gains by one exposure will be described. In the second embodiment, the configuration of the imaging element 107 in the imaging device 1 shown in FIG. 1 is different from that shown in FIG. 2, and will be described below.

[0067] Fig. 7 is a diagram showing the configuration of an image sensor 107 in the second embodiment. In Fig. 7, the same components as those in Fig. 2 are given the same reference numerals, and the description thereof will be omitted.

[0068] 7(a) is a schematic top view showing the pixel array 207 of the image sensor 107, in which unit pixels 206 are arranged in a matrix. Each unit pixel 206 includes one microlens 700, and two photoelectric conversion units (photodiodes: PD) 701a and PD 701b are arranged below each microlens 700.

[0069] FIG. 7B shows a circuit diagram of one unit pixel 206. PD 701a is connected to a transfer switch 702a controlled by a control signal φtxa, and PD 701b is connected to a transfer switch 702b controlled by a control signal φtxb. By setting the control signal φtxa to High and turning on the transfer switch 702a, the charge accumulated in PD 701a is transferred to the FD 202 via the transfer switch 702a. By setting the control signal φtxb to High and turning on the transfer switch 702b, the charge accumulated in PD 701b is transferred to the FD 202 via the transfer switch 702b. Therefore, by setting the control signals φtxa and φtxb to High at different times, the charges of PD 701a and PD 701b can be read out independently. By setting the control signals φtxa and φtxb to High simultaneously, the charges of PD 701a and PD 701b can be read out independently. D7 01a and P D7 The charge of 01b can be added and read out. The image signal with this charge added can be used for recording still images and LV images.

[0070] 7(b), the voltage of the vertical output line 208 varies according to the charge read out from each unit pixel 206, and the voltage is amplified by the column amplifier 703 and input to the AD conversion circuit. In this configuration, the gain of the column amplifier 703 can be changed to perform AD conversion on the same pixel signal. In other words, the charge obtained by one exposure of the image sensor 107 can be read out by applying different gains, so that images with a plurality of different brightnesses can be obtained.

[0071] In this embodiment, a configuration is shown in which a plurality of pupil-divided images output from PD701a and PD701b, respectively, and a plurality of images amplified with different gains are obtained. However, the present invention is not limited to this, and any configuration that can obtain a plurality of images by one exposure is sufficient.

[0072] Next, a shooting operation in the third driving mode in which signals are read out from the PD 701a and the PD 701b of the unit pixel 206 shown in FIG. 7 will be described with reference to a timing chart in FIG. This process is performed in the same manner as the process shown in Fig. 4, for example, when a user presses a still image capture button provided on operation unit 115, and receives a capture instruction issued. Note that the solid lines, dashed dotted lines, and broken lines in the figure are the same as those in Fig. 4, and therefore their explanation will be omitted. Also, the reading of the image signal for a still image is performed at a very high speed, and it is assumed that the time required for the image correction circuit built into DSP 108 to correct the image signal for a still image is longer than the time required for reading the image signal for a still image.

[0073] At time t800, the second SSG 301 of the timing pulse generating circuit 111 asserts a second synchronization signal, and in synchronization therewith starts reading out an image signal (hereinafter referred to as "signal A") from the PD 701a of the image sensor 107. It is assumed that reset scanning of the image sensor 107 starts before time t800, an exposure time (charge accumulation time) determined in advance by photometry or a user instruction. At the same time as reading out of the signal A starts, the read signal A is transferred to the DSP 108, and storage in the RAM 109 connected to the DSP 108 starts. When reading out of the signal A ends at time t801, storage of the signal A in the RAM 109 also ends.

[0074] Subsequently, at time t802, the timing pulse generation circuit 111 asserts a second synchronization signal, and in synchronization therewith starts reading out an image signal (hereinafter referred to as a "B signal") from the PD 701b of the image sensor 107. At the same time as starting to read out the B signal, the read out B signal is transferred to the DSP 108, and storage in the RAM 109 connected to the DSP 108 starts. When reading out of the B signal ends at time t803, storage of the B signal in the RAM 109 also ends.

[0075] At time t804, the timing pulse generating circuit 111 asserts the second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out the LV image signal. Note that reset scanning of the image sensor 107 starts before time t804 by an exposure time (charge accumulation time) that is determined in advance by photometry or a user instruction. The exposure time at this time is B Considering the timing of the end of signal readout, time T LV The time required for the LV image signal to be read out is shorter than the time required for the LV image signal to be read out. The LV image signal that has been read out is then transferred to the DSP 108, and is corrected by an image correction circuit included in the DSP 108. At this time, the data of the A and B signals that have been read out between times t800 and t803 is stored in the RAM 109 without being corrected. At time t805, the reading of the LV image signal is completed, and the correction process in the image correction circuit is also completed. A live view image generated based on the LV image signal that has been corrected is displayed on the display unit 114.

[0076] Next, at time t806, the A signal stored in the RAM 109 is read out and transferred to the image correction circuit, and correction processing of the A signal is started. This correction processing is completed at time t807. As described above, the processing speed of the image correction circuit is A Since this is slower than the signal readout speed, the time from time t806 to time t807 is longer than the time from time t800 to time t801.

[0077] When the correction process of the A signal is completed, next, at time t808, the B signal stored in the RAM 109 is read out and transferred to the image correction circuit, and the correction process of the B signal is started. This correction process is completed at time t809. By transferring the image signal in this manner, it is no longer necessary to suspend / resume the process of the image correction circuit, and control is simplified.

[0078] Similarly, when applying two different gains to a single image signal from the image sensor 107 and reading it out, two image signals obtained by applying the two gains are read out, stored in the RAM 109, and correction processing is performed sequentially.

[0079] At time t810, the timing pulse generation circuit 111 asserts a second synchronization signal, and in synchronization therewith, the image sensor 107 starts reading out the LV image signal. Note that reset scanning of the image sensor 107 starts a predetermined exposure time before time t810, so that the reading out of the LV image starts after the correction process of the B signal ends. Then, the read out LV image signal is transferred to the DSP 108, and is corrected by an image correction circuit included in the DSP 108.

[0080] At the following time t811, the timing pulse generating circuit 111 asserts a second synchronization signal, and in synchronization therewith starts reading out the A signal from the PD 701a of the image sensor 107. The timing of this time t811 is determined by the following conditions. That is, after the start of reading out the LV image signal at time t810 enables reset scanning for charge accumulation for generating a still image, the exposure time (charge accumulation time) T ST The read A signal is transferred to the DSP 108 and stored in the RAM 109 connected to the DSP 108 in sequence.

[0081] Subsequently, at t812, the timing pulse generation circuit 111 asserts a second synchronization signal, and in synchronization therewith starts reading out the B signal from the PD 701b of the image sensor 107. The read out B signal is transferred to the DSP 108 and sequentially stored in the RAM 109 connected to the DSP 108.

[0082] Then, at time t813, the timing pulse generating circuit 111 asserts the second synchronizing signal, and in synchronization therewith, the image sensor 107 starts reading out the LV image signal for the next frame. Thereafter, the processes from time t804 to time t813 are repeated. In the above example, the case where the A signal and the B signal are read out has been described. However, it is also possible to control the reading out of the A signal and the A+B signal obtained by adding up the charges of PD701a and PD701b obtained by simultaneously turning on the transfer switches 702a and 702b.

[0083] Next, the shooting operation in the fourth driving mode will be described with reference to the timing chart of FIG. 9. The solid lines, dashed lines, and broken lines in the figure are the same as those in FIG. 4, and therefore their description will be omitted. Also, the reading of the image signal for a still image is performed at a very high speed, and it takes a longer time for the image correction circuit built into the DSP 108 to correct the image signal for a still image than it takes to read the image signal for a still image. Also, it is assumed that the A signal and the B signal are read out, respectively. The operation from time t800 to time t807 is the same as the operation from time t800 to time t807 shown in the timing chart of FIG. 8, and therefore its description will be omitted.

[0084] When the correction process of the A signal is completed at time t807, the timing pulse generation circuit 111 asserts the second synchronization signal at time t901, and in synchronization therewith starts the reading of the LV image signal by the image sensor 107. Note that the image correction circuit 111 asserts the second synchronization signal at time t901, which is a predetermined exposure time before the time t901, when the reading of the LV image signal starts at time t901. A As if the signal correction process had been completed, reset scanning is started for the image sensor 107. Then, the read LV image signal is transferred to the DSP 108, where it is corrected by an image correction circuit included in the DSP 108.

[0085] When the correction process of the LV image signal is completed at time t902, the B signal stored in the RAM 109 is read out and transferred to the image correction circuit at time t903, and the correction process of the B signal is started. This correction process is completed at time t904. With this correction process, all correction processes of the read still image data are completed. At time t905, the timing pulse generating circuit 111 asserts the second synchronization signal and performs the same operation as at time t804.

[0086] In the next step S906, the timing pulse generating circuit 111 asserts a second synchronization signal, and in synchronization therewith starts reading out the A signal from the PD 701a of the image sensor 107. The timing of this time t906 is determined by the following conditions. That is, after the start of reading out the LV image signal at time t905 enables reset scanning for charge accumulation for generating a still image, the exposure time (charge accumulation time) T ST The read A signal is transferred to the DSP 108 and stored in the RAM 109 connected to the DSP 108 in sequence. After time t907, the operations from time t804 to time t907 are repeated.

[0087] By controlling in this manner, it is possible to make the interval from time t804 to time t901, the interval from time t901 to time t905, and the interval from time t905 to time t907 the same. This makes it possible to capture LV images to be displayed on the display unit 114 at regular intervals while capturing still images.

[0088] However, in the fourth driving mode shown in Fig. 9, the shooting interval between still images is from time t800 to time t906, which is longer than the shooting interval between still images from time t800 to time t811 in the timing chart of Fig. 8. This is because an LV image is shot between correction processes of multiple still images.

[0089] In contrast, in the third driving mode shown in FIG. 8, the shooting interval of still images is shorter than in the fourth driving mode, but the interval from time t804 to time t810 is different from the interval from time t810 to time t813.

[0090] As described above, when shooting is performed in the third driving mode shown in Fig. 8, the shooting frame rate of still images can be improved compared to shooting in the fourth shooting mode shown in Fig. 9. On the other hand, when shooting is performed in the fourth driving mode, LV images can be shot at regular intervals while still images are being shot. Thus, by switching the process depending on whether still images or display images are to be prioritized, the advantages of each driving method can be utilized.

[0091] Fig. 10 is a flowchart for explaining a method for switching between the shooting operation in the third driving mode shown in Fig. 8 and the shooting operation in the fourth driving mode shown in Fig. 9. Note that the same step numbers are used for steps that perform the same processes as in the flowchart in Fig. 6, and explanations thereof will be omitted.

[0092] In S1000, it is determined whether the determined still image shooting mode is the frame rate priority mode for still images, and if it is the frame rate priority mode for still images, the process transitions to S1001, and if not, the process transitions to S1002. When the process transitions to S1001, a program for the sequence in the third drive mode shown in Fig. 8 is loaded, and when the process transitions to S1002, a program for the sequence in the fourth drive mode shown in Fig. 9 is loaded, and the process ends. After that, shooting is performed according to the loaded program.

[0093] As described above, according to the second embodiment, in an image sensor capable of reading out multiple images amplified with different gains for one exposure of a still image, multiple shooting sequences are prepared and switched according to the shooting mode of the still image, thereby providing the user with the option to prioritize either the frame rate of the still image or the display quality in the viewfinder, whichever is more preferable.

[0094] <Third embodiment> Next, a third embodiment of the present invention will be described. In the third embodiment, a shooting operation will be described in which it is desired to shorten the interval between continuous shooting of still images compared to the first and second embodiments.

[0095] Fig. 11 is a timing chart showing the shooting operation in the fifth driving mode in the third embodiment. The process shown in Fig. 11 is performed in response to a shooting instruction issued by, for example, a user pressing a still image shooting button provided on the operation unit 115, similar to the process shown in Fig. 4. The solid lines, dashed lines, and broken lines in the figure are the same as those in Fig. 4, and therefore their explanations are omitted. However, in the figure, the diagonal solid lines of "display unit image update scan" indicate the scan by which the display unit 114 updates the image. Also, the readout of the image signal for a still image is performed at high speed, and it is assumed that the time required for the image correction circuit built in the DSP 108 to correct the image signal for a still image is longer than the time required for the image signal for a still image to be read out.

[0096] At time t1100, the second synchronization signal is asserted from the second SSG 301 of the timing pulse generating circuit 111, and in synchronization therewith, the readout of the LV image signal is started. At the same time, the image correction circuit included in the DSP 108 starts a correction process on the readout LV image signal. At time t1101, the third synchronization signal is asserted from the third SSG 302 of the timing pulse generating circuit 111, and the LV image generated based on the LV image signal corrected at time t1100 starts to be displayed on the display unit 114. This series of operations is repeated with a period T0 as a cycle, and the LV display is updated at regular time intervals. Here, it is assumed that the captured image can be stably displayed by providing a phase difference Δt1 from capture to display by the time difference between time t1100 and time t1101.

[0097] At time t1102, a reference synchronization signal is asserted from the reference SSG 300 of the timing pulse generating circuit 111. When the third SSG 302 receives this reference synchronization signal, it resets the value of its internal counter. In this way, the third SSG 302 is synchronized with the synchronization signal of the reference SSG 300. The reference synchronization signal is also input to the CPU 110, and when the CPU 110 detects that this reference synchronization signal has been asserted, it reads a value from a counter representing the system time of the CPU 110 and records the count value.

[0098] At time t1103, the second synchronization signal is asserted from the second SSG 301, and reading of the LV image signal is started. At the same time, the second synchronization signal is input to the CPU 110, and the CPU 110 records a count value representing the system time at which the second synchronization signal is asserted. The CPU 110 calculates the assertion timing of the second synchronization signal from the count values ​​of the reference synchronization signal and the second synchronization signal by calculation. Then, based on the calculation result, the timing pulse generation circuit 111 is controlled to synchronize the second synchronization signal with the reference synchronization signal with a certain time difference. By controlling in this way, both the second synchronization signal and the third synchronization signal can be operated in synchronization with the reference synchronization signal, so that the phase difference Δt1 between the second synchronization signal and the third synchronization signal can be maintained at a constant value.

[0099] When the still image shooting button is pressed at time t1104, reset scanning for still images is started at time t1105 after the release time lag T1 has elapsed. At time t1106 after the charge accumulation time T2 has elapsed, readout of image signals for still images is started and temporarily stored in the RAM 109. Then, at time t1107, reset scanning for still images is started again, and at time t1108, readout of image signals for still images corresponding thereto is started and similarly temporarily stored in the RAM 109. This is repeated for a period T3 until the still image shooting button is released. Note that, in this embodiment, this period T3 is the period during which the still image shooting button is pressed, but is not limited thereto. For example, the user may set the number of shots and the shooting interval in advance using the UI by the operation unit 115, and the period during which shooting is repeated at the accumulation time T2 may be calculated from the set number of shots and the shooting interval.

[0100] After the period T3 has elapsed, the CPU 110 controls the timing pulse generating circuit 111 so that the second synchronization signal is asserted from the counter of the reference synchronization signal with a phase difference Δt1 with respect to the reference synchronization signal. After the second synchronization signal is asserted at time t1109, reset scanning for the LV image is started at time t1110. At time t1111, readout of the corresponding LV image signal is started, and the DSP 108 performs image processing on the LV image signal. Then, when the third synchronization signal is asserted at time t1112, the display unit 114 starts displaying the LV image generated from the LV image signal whose readout started at time t1111.

[0101] Next, at time t1113, upon completion of reading out the immediately preceding LV image signal, the still image signal that was read out during period T3 and temporarily stored in the RAM 109 is read out, and correction processing is performed by the DSP 108. Therefore, reading out of the LV image signal for updating the display unit 114 corresponding to the third synchronization signal at time t1114 is not performed. Note that correction processing performed on still image data refers to correction processing performed on the RAW data of the image sensor 107, such as processing to correct variations for each individual sensor and applying digital gain. Incidentally, development processing and JPEG compression processing may be performed together with the correction processing during the period from time t1113 to time t1115, or may be performed at a timing after time t1115.

[0102] Then, at time t1115, reset scanning for the LV image is started, and at time t1116, reading of the corresponding LV image signal is started, and correction processing of the LV image signal read by the DSP 108 is started. The period T4 from time t1109 to time t1111 is longer than the period T0 in order to perform image processing on the still image data at time t1112. In this embodiment, the DPS 108 performs image processing on only one still image data during each period T4, but this is not limited to this, and a configuration in which multiple still image data are processed continuously may be used. In this case, a period such as time t1114 in which the LV image signal for updating the display unit 114 corresponding to the third synchronization signal is not read continues. Also, even when the period T4 is extended, the timing pulse generating circuit 111 is controlled so that the second synchronization signal is synchronized with the third synchronization signal by providing a phase difference Δt1.

[0103] Then, when all processing of the image signals for still images temporarily stored in the RAM 109 during the period T3 is completed and readout of the LV image signal starts at time t1117, in response to the third synchronization signal at time t1118, the display unit 114 starts displaying the LV image generated from the LV image signal whose readout started at time t1117. After this time, readout of the LV image signal is repeated at a cycle of the period T5.

[0104] In FIG. 11, the time of the period T5 is the same length as the period T0, but this is not limited thereto. In addition, the timing of switching from the cycle of the period T4 to the cycle of the period T5 is when all processing of the image signals for still images temporarily stored in the RAM 109 is completed, but this is not limited thereto and may be based on the display format of the display unit 114. For example, if the display format of the display unit 114 is an interlace format, one image is created from a TOP field and a BOTTOM field. Therefore, when the third synchronization signal is asserted, the CPU 110 records the system time and also records field information of whether the field is a TOP field or a BOTTOM field. Thus, at the cycle switching timing, the field information and whether there is an image signal for still images in the RAM 109 may be determined and the switching may be performed.

[0105] In the third embodiment, the timing pulse generating circuit 111 is controlled so that the second synchronization signal is asserted with a phase difference Δt1 with respect to the reference synchronization signal, but the present invention is not limited to this. For example, as shown in FIG. 12, the timing pulse generating circuit 111 is set to shift the assertion timing of the third synchronization signal by Δu1 so that the second synchronization signal and the third synchronization signal are asserted with a phase difference Δt1, thereby making it possible to advance the timing of reading out the LV image signal. This Δu1 can be calculated at the timing when the still image where the period T3 is determined ends, by determining the periods T1, T2, and T4 at the timing when the still image shooting button is pressed at time t1104. In other words, Δu1 is calculated so that the third synchronization signal is asserted at time t1112 in FIG. 12, and set in the timing pulse generating circuit 111. Based on this setting, the third synchronization signal is asserted at time t1201, making it possible to advance the display timing of the LV image after shooting the still image. If the period T3 is repeated for the number of sheets previously set by the user as described above, it can be determined at time t1104. Also, the timing for calculating Δu1 and the timing for setting the timing pulse generating circuit 111 are assumed to have been completed before time t1111.

[0106] It is possible to achieve higher speed continuous shooting of still images by performing imaging operation in the fifth driving mode shown in Fig. 11 or 12. This makes it possible to provide ultra-high speed continuous shooting of still images to users who want to capture decisive moments, such as professional users.

[0107] Next, the shooting operation in the sixth driving mode will be described with reference to the timing chart of Fig. 13. In Fig. 13, the same signals and timings as those in Figs. 11 and 12 are denoted by the same reference numerals, and the description will be omitted as appropriate.

[0108] In the sixth driving mode, after the period T3 has elapsed, at time t1301, when the CPU 110 detects the completion of reading out the last still image signal, the CPU 110 then reads out multiple still image signals temporarily stored in the RAM 109 and performs correction processing in the DSP 108.

[0109] In addition, during the period T6 after time t1301, correction processing is performed on the image signals for multiple still images temporarily stored in RAM 109, and DSP 108 may perform processing such as HDR processing and filter processing using multiple still image data.

[0110] Here, taking HDR processing as an example, for example, a proper exposure image in which the accumulation time is controlled so that the subject is properly exposed, an underexposed image in which the accumulation time is controlled so that a high-luminance subject such as a blue sky is properly exposed, and an overexposed image in which the accumulation time is controlled so that a low-luminance subject that is likely to be crushed is properly exposed are continuously shot during a period T3. Then, the obtained image data of the three types of exposure are synthesized so that the high-luminance part becomes an underexposed image and the low-luminance part becomes an overexposed image, and an image reproduced from low luminance to high luminance is generated by the DSP 108 during a period T6.

[0111] Then, after the time period T6 has elapsed, the CPU 110 controls the timing pulse generating circuit 111 so that the second synchronization signal is asserted from the counter of the reference synchronization signal with a phase difference Δt1 with respect to the reference synchronization signal, and asserts the second synchronization signal at time t1302. Then, at time t1303, reset scanning for the LV image is started, and at time t1304, readout of the corresponding LV image signal is started, and the DSP 108 performs correction processing on the LV image signal. Then, when the third synchronization signal is asserted at time t1305, the display unit 114 starts displaying the LV image generated from the LV image signal readout of which was started at time t1304. After this time t1304, readout of the LV image signal is repeated at a cycle of the time period T5.

[0112] Note that HDR processing, filter processing, etc., do not necessarily occur during the period T6 To It is not necessary to perform this, and it may be configured to perform this in parallel during the readout period of the LV image signal after the end of the period T6.

[0113] In the present embodiment, the timing pulse generating circuit 111 is controlled so that the second synchronization signal is asserted with a phase difference Δt1 with respect to the reference synchronization signal. However, for example, by setting the timing pulse generating circuit 111 so that the assertion timing of the third synchronization signal is shifted by Δu1 so that the second synchronization signal and the third synchronization signal are asserted with a phase difference Δt1, it is possible to advance the time t1304. This Δu1 can be calculated from the lengths of the periods T1, T2, T3, T6, and T5 as described in FIG. 12. That is, Δu1 is calculated so that the third synchronization signal is asserted at time t1305, and is set in the timing pulse generating circuit 111. Based on this setting, the third synchronization signal is asserted, and it is possible to advance the display timing of the LV image after shooting a still image.

[0114] 13, image processing is first performed on the captured image signal for still images, and then the image signal for LV can be read out. As a result, it is possible to permit the next shooting earlier even when performing complex image synthesis processing such as HDR (high dynamic range) processing or filter processing that applies special effects to images.

[0115] Next, an example of switching the driving mode will be described with reference to the flowchart in Fig. 14. Unless otherwise specified, the processing of each step in the flowchart in Fig. 14 is realized by the CPU 110 controlling each unit of the imaging device shown in Fig. 1. When the still image capture button is pressed, the CPU 110 executes the processing of this flowchart.

[0116] In S1401, CPU 110 checks the image processing method for still images among the menu settings set by the user via operation unit 115. For example, whether HDR processing is enabled, whether filter processing for applying special effects to images is enabled, whether NR processing by shooting multiple images is enabled, whether multiple exposure settings are enabled, etc. Here, if a menu is set to execute image processing for shooting and combining multiple still images to generate a single image, the process proceeds to S1402, and otherwise the process proceeds to S1403.

[0117] In S1402, a shooting sequence for special shooting is executed. Here, a program for the sequence in the sixth driving mode shown in Fig. 13 is loaded.

[0118] Next, in S1403, the continuous shooting mode is confirmed from among the menu settings set by the user via the operation unit 115. For example, if there are five continuous shooting modes, Fastest, High+, High, Low, and Single, it is determined whether the fastest, "Fastest," is set, and if the fastest is set, the process proceeds to S1404, and if a continuous shooting mode other than the fastest is set, the process proceeds to S1405.

[0119] In S1404, a shooting sequence with priority given to frame speed is executed. Here, a program for the sequence in the fifth driving mode shown in FIG.

[0120] On the other hand, in S1405, a photographing sequence with priority given to subject tracking is executed. and second The sequence program for one of the first to fourth drive modes shown in Figures 4, 5, 8, and 9 described in the embodiment is loaded, and the process ends. Note that the method for selecting the first to fourth drive modes may follow the method described in the first or second embodiment with reference to Figures 6 and 10. After that, shooting is performed according to the loaded program.

[0121] As described above, according to the third embodiment, the shooting sequence is switched in accordance with the contents set by the user, so that it is possible to shoot in the optimal shooting sequence in accordance with the user's settings.

[0122] <Other embodiments> The present invention may be applied to a system consisting of multiple devices (such as a host computer, interface device, scanner, video camera, etc.), or to an apparatus consisting of a single device (such as a copier, facsimile machine, etc.).

[0123] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0124] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0125] 107: imaging element, 108: DSP, 109: RAM, 111: timing pulse generating circuit, 701a, 701b: photodiodes

Claims

1. A driving means for driving the imaging element; a storage means for storing an image signal read out from the imaging element; an image processing means for processing an image signal read out from the imaging element; a control means for selectively using a plurality of drive modes to control the drive means, the storage means, and the image processing means; The plurality of driving modes include: a first driving mode which performs a first operation in which the driving means reads out a first image signal from the imaging element and processes the first image signal by the image processing means without storing the first image signal in the storage means, and a second operation in which the driving means reads out a second image signal from the imaging element and stores it in the storage means before performing the first operation, and after performing the first operation, reads out the stored second image signal from the storage means and processes it by the image processing means; a second driving mode in which the first operation and a third operation in which the driving means reads out the second image signal from the imaging element at a timing different from that of the first operation, and the second image signal is processed by the image processing means without being stored in the storage means, The control means selects either the first driving mode or the second driving mode in accordance with a readout speed of the second image signal.

1. An imaging device comprising:

2. A driving means for driving an imaging element; a storage means for storing an image signal read out from the imaging element; an image processing means for processing an image signal read out from the imaging element; a control means for selectively using a plurality of drive modes to control the drive means, the storage means, and the image processing means; The plurality of driving modes include: a first driving mode which performs a first operation in which the driving means reads out a first image signal from the imaging element and processes the first image signal by the image processing means without storing the first image signal in the storage means, and a second operation in which the driving means reads out a second image signal from the imaging element and stores it in the storage means before performing the first operation, and after performing the first operation, reads out the stored second image signal from the storage means and processes it by the image processing means; a second driving mode in which the first operation and a third operation in which the driving means reads out the second image signal from the imaging element at a timing different from that of the first operation, and the second image signal is processed by the image processing means without being stored in the storage means, The control means selects either the first driving mode or the second driving mode in accordance with a set photographing mode.

1. An imaging device comprising:

3. The control means selecting the first driving mode when a read speed of the second image signal is faster than a processing speed by the image processing means; When the read speed of the second image signal is equal to or lower than the processing speed of the image processing means, the second driving mode is selected.

2. The imaging device according to claim 1 .

4. 4. The imaging device according to claim 1, wherein in the second operation of the first driving mode, the driving means reads out the second image signal from the imaging element once or multiple times per exposure and stores it in the memory means.

5. The plurality of driving modes include a third driving mode that performs the first operation, and a fourth operation in which, before performing the first operation, the second image signal is read from the imaging element a plurality of times for one exposure and stored in the storage means, and every time the first operation is performed, the stored second image signal is read from the storage means by a signal amount corresponding to each of the plurality of readouts and processed by the image processing means.

5. The imaging device according to claim 4.

6. The control means selecting the first driving mode when priority is given to reading out the second image signal; The third driving mode is selected when priority is given to the readout of the first image signal.

6. The imaging device according to claim 5.

7. the imaging element includes a plurality of pixels, each of which includes a plurality of photoelectric conversion units; The imaging device according to any one of claims 4 to 6, characterized in that the driving means selectively performs either a driving operation to read out image signals from the plurality of photoelectric conversion units in a plurality of separate operations or a driving operation to add up and read out the image signals from the plurality of photoelectric conversion units.

8. The imaging element has an amplifier means capable of applying different gains, 8. The imaging apparatus according to claim 4, wherein the driving means drives the image signal read out from the imaging element a plurality of times to be multiplied by different gains by the amplifier means to generate and output a plurality of image signals.

9. 9. The imaging device according to claim 1, wherein, in the first driving mode, when an instruction is given to read out the second image signal multiple times in succession while the first operation is being repeatedly performed, the first operation is interrupted and the second image signal is read out multiple times and stored in the memory means, and each time the first operation is performed, the stored second image signal is read out from the memory means in signal amounts corresponding to each of the multiple readouts and processed by the image processing means.

10. 10. The imaging device according to claim 9, wherein the plurality of driving modes include a fourth driving mode in which, when an instruction is given to read out the second image signal multiple times in succession while the first operation is being repeatedly performed, the first operation is interrupted, the second image signal is read out multiple times and stored in the storage means, and the stored second image signal is processed by the image processing means before resuming the first operation.

11. The control means selecting the first driving mode when a synthesis process for synthesizing a plurality of the second image signals is not performed; When the synthesis process is performed, the fourth driving mode is selected.

11. The imaging device according to claim 10.

12. 12. The imaging device according to claim 1, wherein the second image signal takes a longer time to be read out from the imaging element than the first image signal.

13. 13. The imaging device according to claim 1, wherein the first image signal is a signal of each frame of a moving image, and the second image signal is a signal of a still image.

14. A method for controlling an imaging device having a driving means for driving an imaging element, a storage means for storing an image signal read out from the imaging element, and an image processing means for processing the image signal read out from the imaging element, comprising: a control step in which a control unit selectively uses a plurality of drive modes to control the drive unit, the storage unit, and the image processing unit; The plurality of driving modes include: a first driving mode which performs a first operation in which the driving means reads out a first image signal from the imaging element and processes the first image signal by the image processing means without storing the first image signal in the storage means, and a second operation in which the driving means reads out a second image signal from the imaging element and stores it in the storage means before performing the first operation, and after performing the first operation, reads out the stored second image signal from the storage means and processes it by the image processing means; a second driving mode in which the first operation and a third operation in which the driving means reads out the second image signal from the imaging element at a timing different from that of the first operation, and the second image signal is processed by the image processing means without being stored in the storage means, In the control step, either the first driving mode or the second driving mode is selected in accordance with a readout speed of the second image signal.

23. A method for controlling an imaging apparatus comprising the steps of:

15. A method for controlling an imaging device having a driving means for driving an imaging element, a storage means for storing an image signal read out from the imaging element, and an image processing means for processing the image signal read out from the imaging element, comprising: a control step in which a control unit selectively uses a plurality of drive modes to control the drive unit, the storage unit, and the image processing unit; The plurality of driving modes include: a first driving mode which performs a first operation in which the driving means reads out a first image signal from the imaging element and processes the first image signal by the image processing means without storing the first image signal in the storage means, and a second operation in which the driving means reads out a second image signal from the imaging element and stores it in the storage means before performing the first operation, and after performing the first operation, reads out the stored second image signal from the storage means and processes it by the image processing means; a second driving mode in which the first operation and a third operation in which the driving means reads out the second image signal from the imaging element at a timing different from that of the first operation, and the second image signal is processed by the image processing means without being stored in the storage means, In the control step, either the first driving mode or the second driving mode is selected according to a set photographing mode.

23. A method for controlling an imaging apparatus comprising the steps of:

16. A program for causing a computer to function as a control unit of the imaging apparatus according to any one of claims 1 to 13.

17. A computer-readable storage medium storing the program according to claim 16.

Images