Image pickup apparatus

Abstract

An image pickup apparatus includes: an image sensor including a pixel section, a readout section for reading out a first image signal and a second image signal from the pixel section, the resolution of the second image signal being lower than the resolution of the first image signal, a first output section for outputting the first image signal, and a second output section for outputting the second image signal; the image processor is used for generating a third image signal according to the first image signal, the resolution of the third image signal is lower than that of the first image signal, the first output part is used for outputting the first image signal, and the second output part is used for outputting the second image signal or the third image signal; a first integrated circuit for processing the output first image signal; and a second integrated circuit for processing the second image signal or the third image signal.

Description

Technical Field

[0001] The present invention relates to an imaging device. Background Art

[0002] Generally, digital cameras and electronic devices equipped with camera functions generally have a "live view function (LV function)". The "live view function" is a function of displaying a captured image in real time on a display unit while capturing a moving image or the like. A user can adjust a capturing range for still image capturing and moving image capturing while checking a moving image displayed by the live view function.

[0003] In order for the user to perform the shooting intended by the user, the above-mentioned ease of adjustment of the shooting range is crucial. For example, in the case of shooting a still image during the shooting of a moving image, the moving image cannot be obtained in the frame in which the still image has been shot. In this case, since the image to be displayed as a live view cannot be obtained, the ease of adjustment of the shooting range is weakened. In order to make the adjustment of the shooting range easy, it is also crucial to prevent the loss of the live view frame even when shooting a still image. For this reason, it is known to perform processing on the still image that is different from the processing on the live view image.

[0004] For example, Japanese Patent No. 6757199 discloses an image sensor having a first mode in which image data read from a pixel portion is transferred to a memory built into the image sensor and a second transfer mode in which the image data is transferred to the outside of the image sensor.

[0005] Japanese Patent Laid-Open No. 2023-106041 discloses a data processing method for a case where a plurality of data has been read out inside an image sensor. Specifically, the following method is described: still image data from a pixel unit is output after being reduced to a live view image by a conversion circuit inside the image sensor, and when live view image data has been read out from the pixel unit, the live view image data is output without being reduced.

[0006] However, according to the technology disclosed in the above-mentioned Japanese Patent No. 6757199, image data having a large data volume, such as a still image, cannot be used for live view display, which results in omission of display frames of live view.

[0007] Furthermore, although Japanese Patent Laid-Open No. 2023-106041 discloses the processing of data inside an image sensor, no clear description is provided of a processing method related to a signal processing circuit connected to the image sensor. Furthermore, no clear description is provided of a method for configuring appropriate output interfaces corresponding to still images and live view images, respectively. Summary of the invention

[0008] The present invention has been made in view of the above problems, and the present invention provides an imaging device capable of preventing the loss of a live view image when a user takes a picture while viewing the live view image.

[0009] According to one aspect of the present invention, there is provided an imaging device including: an image sensor including: a pixel unit in which a plurality of pixels are arranged in a matrix, a readout unit for reading out a first image signal and a second image signal from the pixel unit, the resolution of the second image signal being lower than that of the first image signal, an image processor for generating a third image signal based on the first image signal, the resolution of the third image signal being lower than that of the first image signal, a first output unit for outputting the first image signal, and a second output unit for outputting the second image signal or the third image signal; a first integrated circuit for processing the first image signal output from the first output unit of the image sensor; and a second integrated circuit for processing the second image signal or the third image signal output from the second output unit of the image sensor.

[0010] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram showing the configuration of an imaging device according to a first embodiment of the present invention.

[0012] Figure 2A and Figure 2B is a diagram showing the configuration of an image sensor.

[0013] Figure 3A and Figure 3B is a diagram showing a stacked structure of an image sensor.

[0014] Figure 4A and Figure 4B is a diagram showing data paths of a still image and an LV image.

[0015] Figure 5 is a timing diagram showing the operation of an image sensor according to the first embodiment.

[0016] Figure 6 is a timing diagram showing the operation of an image sensor according to a second embodiment.

[0017] Figure 7 is a timing diagram showing the operation of an image sensor according to a third embodiment. DETAILED DESCRIPTION

[0018] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. In the embodiments, a plurality of features are described, but the invention is not limited to the invention that requires all such features, and a plurality of such features can be appropriately combined. Further, in the drawings, the same reference numerals are given to the same or similar configurations, and redundant descriptions thereof are omitted.

[0019] (First Embodiment)

[0020] Figure 1 is a block diagram showing the configuration of an imaging device according to the first embodiment of the present invention. As Figure 1 shown, the imaging device 150 includes an imaging optical system 104. The imaging optical system 104 includes a first lens 100, an aperture 101, a second lens 102, and a third lens 103. The first lens 100 is arranged at the front end of the imaging optical system 104. The aperture 101 adjusts the amount of light during shooting by adjusting its opening diameter. The opening diameter of the aperture 101 is adjusted by driving an aperture actuator 126. Driving a focus actuator 124 described later causes the second lens 102 and the third lens 103 to move forward and backward in the optical axis direction, thereby adjusting the focus of the imaging optical system 104.

[0021] A focal plane shutter 105, an optical low-pass filter 106, and an image sensor 107 are arranged in sequence behind the imaging optical system 104. The focal plane shutter 105 has a function of adjusting the exposure period during still image shooting. The optical low-pass filter 106 has a function of reducing false colors and moiré patterns in the captured image. The image sensor 107 converts the optical image of the subject formed by the imaging optical system 104 into an electrical signal.

[0022] The imaging device 150 includes a first integrated circuit 110 and a second integrated circuit 114. The first integrated circuit 110 includes a first digital signal processor (DSP) 111.

[0023] The first DSP 111 is connected to the image sensor 107 via a first interface (IF) 108 serving as a first output unit, receives image data transmitted from the image sensor 107, and performs image processing. Examples of the image processing performed by the first DSP 111 include processing for correcting still image data and the like.

[0024] A first RAM 112 is connected to the first integrated circuit 110 and stores the image data processed by the first DSP 111. Note that although the first RAM 112 is arranged outside the first integrated circuit 110 in the present embodiment, a configuration in which part or all of its functions are provided in the first integrated circuit 110 or the first DSP 111 is allowed.

[0025] At least a second digital signal processor (DSP) 115 and a CPU 117 are provided in the second integrated circuit 114. Although in this embodiment, the shutter drive circuit 122, the focus drive circuit 123, and the aperture drive circuit 125 are further provided in the second integrated circuit 114, a configuration in which these components are provided in other integrated circuits is allowed.

[0026] The second DSP 115 is connected to the image sensor 107 via a second interface (IF) 109 serving as a second output unit, receives image data transmitted from the image sensor 107, and performs image processing.

[0027] Examples of the image processing performed by the second DSP 115 include processing for correcting LV image data and generation of a display image to be displayed on the display unit 119.

[0028] The second RAM 116 is connected to the second integrated circuit 114, stores the image data processed by the second DSP 115, and further serves as a working memory when the following CPU 117 operates.

[0029] Note that although this embodiment employs a configuration in which two functions are implemented using a RAM, other types of memories may also be used as long as they have a sufficiently high access speed and no problems in terms of operation. In addition, although in this embodiment the second RAM 116 is arranged outside the second integrated circuit 114, a configuration in which some or all of its functions are provided in the second integrated circuit 114, the second DSP 115, or the CPU 117 is allowed.

[0030] Here, the first integrated circuit 110 and the second integrated circuit 114 are connected via a third interface (IF) 113. The still image data corrected by the first DSP 111 inside the first integrated circuit 110 is transmitted to the second integrated circuit 114 via the third IF 113 and is recorded in the recording medium 120 via the second DSP 115.

[0031] The CPU 117 overall controls the operation of the imaging device 150 and executes a program for controlling each component of the imaging device. The CPU 117 controls the shooting timing of moving images and still images to be described later by configuring various types of settings for the image sensor 107. In addition, it has a function of adjusting the focus of the imaging optical system 104 by driving and controlling the focus drive circuit 123 to be described later using the calculation result output from the second DSP 115.

[0032] The CPU 117 is connected to an operation unit 118, a display unit 119, a recording medium 120, a ROM 121, a shutter drive circuit 122, a focus drive circuit 123, and an aperture drive circuit 125.

[0033] The operation unit 118 includes operation members such as buttons and levers. The user inputs instructions to the CPU 117 and performs shooting by operating the operation members of the operation unit 118. The operation unit 118 includes a still image shooting start button. When a user operation of pressing the still image shooting start button is detected, the CPU 117 controls to shoot a still image after a specific period of time has elapsed.

[0034] The display unit 119 displays the image processed by the second DSP 115 and menus and the like. Not only a display can be used as the display unit 119, but also an electronic viewfinder (EVF) and the like can be used as the display unit 119. The recording medium 120 is a detachable recording medium that records still image data and moving image data, and can be implemented by, for example, a memory card or the like. The ROM 121 stores, for example, programs for the CPU 117 to control the operations of various components.

[0035] The shutter drive circuit 122 drives and controls the focal plane shutter 105. The focus drive circuit 123 changes the focus position of the imaging optical system 104. That is, the focus drive circuit 123 performs focus adjustment by driving and controlling the focus actuator 124 based on the output of the CPU 117 and moving the focus lens (the second lens 102 and the third lens 103) in the optical axis direction. The aperture drive circuit 125 changes the opening diameter of the aperture 101 by driving and controlling the aperture actuator 126, thereby adjusting the amount of light incident on the image sensor 107.

[0036] Figure 2A is a circuit diagram of a unit pixel, and Figure 2B is a diagram showing the configuration of a pixel array and peripheral circuits. In particular, reference will be made to Figure 2A and Figure 2B to describe the circuits around the pixels in the image sensor 107.

[0037] First, reference will be made to Figure 2A to describe the circuit configuration of the unit pixel 206. A photodiode (hereinafter referred to as PD) 200 serving as a photoelectric conversion unit is arranged under the microlens and generates and accumulates charges corresponding to the incident light amount. The transfer switch 201 is a switch controlled by a control signal φtx. Setting the value of the control signal φtx to high (hereinafter referred to as H) turns on the transfer switch 201 and causes the charges accumulated in the PD 200 to be transferred to the floating diffusion unit (hereinafter referred to as FD) 202.

[0038] The reset switch 203 is a switch controlled by the control signal φres and resets the FD 202. Both the control signals φtx and φres are set to H, thereby placing both the PD 200 and the FD 202 at the power supply voltage (VDD); in this way, the pixel reset operation is performed.

[0039] The transistor 204, which also serves as a pixel amplifier, is connected to the selection switch 205 and is also connected to the constant current source 209 via the column output line 208 (see Figure 2B ) (see Figure 2B ). When the value of the control signal φsel of the selection switch 205 is set to H, the transistor 204 is connected to the constant current source 209, thereby forming a pixel amplifier. The charge transferred from the PD 200 to the FD 202 is converted by the pixel amplifier into a voltage value corresponding to the amount of charge and is output as a pixel signal to the column output line 208.

[0040] Next, the overall configuration of the image sensor 107 will be described with reference to Figure 2B the following.

[0041] In the pixel array (pixel section) 207, a plurality of unit pixels 206 are arranged in a matrix. Specifically, (m + 1) unit pixels 206 and (n + 1) unit pixels 206 are arranged in the horizontal direction and the vertical direction, respectively. Note that m and n are natural numbers. With this configuration, a plurality of photoelectric conversion elements are arranged in a matrix in the pixel array 207.

[0042] The drive pulse generation circuit 210 generates pulses for performing a reset operation and a readout operation on the unit pixel 206. The pulses generated by the drive pulse generation circuit 210 are supplied to the pixel drive circuit 212. The row selection circuit 211 selects a specific row to which the pulses generated by the drive pulse generation circuit 210 will be supplied and sets the selected specific row in the pixel drive circuit 212. The pixel drive circuit 212 supplies the pulses generated by the drive pulse generation circuit 210 to the specific row that has been selected and set by the row selection circuit 211.

[0043] The row selection circuit 211 allows signals to be read out from the pixel array 207 in a variety of ways. For example, in the case of reading pixel signals to be used for a still image, since high-resolution image data is necessary, the row selection circuit 211 selects each row of the pixel array 207 and reads out the pixel signals.

[0044] On the other hand, when reading out pixel signals to be used for the LV moving image (live view moving image), since high resolution is not required, the row selection circuit 211 selects rows of the pixel array 207 at an interval of two rows and reads out pixel signals. In this case, after the 0th row, the pixel signals of the 3rd row are read out. Reading out pixel signals in this way enables the acquisition of LV image data with a vertical resolution reduced to 1 / 3 of the vertical resolution of the still image described above.

[0045] Comparing the still image with the LV image, although the still image has excellent resolution, the characteristics of the LV image are such that the readout period is shortened as the vertical resolution decreases, and the power required for readout can be reduced. In this way, the row selection / control mode of the row selection circuit 211 enables the acquisition of multiple types of images suitable for their respective purposes. Note that there is no limitation on the configuration in which pixel rows are read out with an interval; for example, a configuration is allowed in which LV image data with a low vertical resolution is obtained by adding pixel signals of multiple pixel rows.

[0046] According to the pulses supplied from the pixel drive circuit 212, pixel signals are output to the column output line 208 row by row. The constant current source 209 and the transistor 204 together constitute a source follower circuit. The AD conversion circuit 213 converts the analog signal output to the column output line 208 into a digital signal.

[0047] Note that although this embodiment employs a configuration in which pixel signals are read out from the pixel array 207 row by row, the present invention is not limited thereto. For example, when two column output lines are prepared for each column, by changing the connection between the pixels and the column output lines between even rows and odd rows, pixel signals of two rows can be read out simultaneously for both the still image and the LV image. By adopting such a configuration that allows simultaneous readout of multiple rows of pixel signals, the readout period can be shortened. On the other hand, the larger the number of column output lines, the more complex the circuit. Therefore, it is desirable to determine an appropriate number of column output lines according to the usage method.

[0048] Figure 3A and Figure 3B is a diagram showing the configuration of an image sensor composed of multiple semiconductor substrates. The configuration of the image sensor 107 according to this embodiment will be described with reference to Figure 3A and Figure 3B description.

[0049] Generally speaking, as a method of configuring an image sensor, a method is known in which a single image sensor is configured by providing a plurality of semiconductor substrates each having an independent circuit, stacking these plurality of semiconductor substrates on top of each other, and connecting them to each other. Similarly, in the image sensor 107 according to the present embodiment, a single image sensor chip is configured by stacking a first semiconductor substrate 300 and a second semiconductor substrate 301 on top of each other.

[0050] Reference will be made Figure 3A to describe the circuit configurations respectively mounted on the first semiconductor substrate 300 and the second semiconductor substrate 301.

[0051] The pixel array 207 is provided on the first semiconductor substrate 300. In addition, an AD conversion circuit 213, an image processing circuit 302, an image memory 303, a switch circuit 305, a first IF 108, and a second IF 109 are provided on the second semiconductor substrate 301.

[0052] The image processing circuit 302 is an information processing component that processes the information read out from the pixel array 207 to convert the information into other types of information. Although the image processing circuit 302 can perform various types of processing, in the present embodiment, it is assumed that the image processing circuit 302 converts a still image into an LV image by reducing the still image.

[0053] The image memory 303 is a memory that temporarily stores the pixel data read out from the pixel array 207 and converted into digital values in the AD conversion circuit 213, as well as the image data processed by the image processing circuit 302. In the present embodiment, any memory can be used as this memory, as long as it is configured to be able to store data and allow the already stored data to be read out at a speed sufficiently faster than the speed of reading pixel signals from the pixel array 207 and the processing speed in the image processing circuit 302.

[0054] The first IF 108 is connected to the AD conversion circuit 213 and outputs the read-out image data to the outside without storing it in the image memory 303. The second IF 109 is connected to the image memory 303 and outputs the image data stored in the image memory 303 to the outside.

[0055] The switch circuit 305 is controlled by the CPU 117 and has a function of switching the output destination of the image data output from the AD conversion circuit 213 to one of the image processing circuit 302, the image memory 303, and the first IF 108. The switching will be described later with reference to Figure 4A and Figure 4B for the description of this switching.

[0056] Note that circuits other than the circuits shown may be arranged between the discrete blocks. For example, the following configuration is also allowed: a correction processing circuit that performs correction processing on the image read out from the AD conversion circuit 213 is arranged between the AD conversion circuit 213 and the first IF 108 or between the AD conversion circuit 213 and the image processing circuit 302.

[0057] Figure 3B is a schematic diagram showing an example in which the first semiconductor substrate 300 and the second semiconductor substrate 301 are stacked to configure the image sensor 107 as a single chip. This figure shows the state in which the first semiconductor substrate 300 is stacked on the second semiconductor substrate 301. Any known technique can be used as the technique for electrically connecting the stacked semiconductor substrates to each other.

[0058] Note that although this embodiment presents a configuration in which the pixel array 207 is mounted on the first semiconductor substrate 300 and all other circuit blocks are mounted on the second semiconductor substrate 301, the present invention is not limited thereto. For example, a configuration in which all pixel arrays and circuit blocks are mounted on the same semiconductor substrate or a configuration in which circuit blocks are further mounted on a plurality of semiconductor substrates is allowed.

[0059] Figure 4A and Figure 4B is a diagram schematically showing the paths of the image data of a still image, a reduced still image, and an LV image. Referring to Figure 4A and Figure 4B , the following describes which circuit blocks (data paths) the image data of a still image, a reduced still image, and an LV image pass through when output to the outside of the image sensor 107.

[0060] Figure 4A is a data path diagram showing the data paths through which a still image and a reduced still image pass.

[0061] The pixel signals of the still image are read out from the pixel array 207, and the pixel signals are converted into digital values in the AD conversion circuit 213. Thereafter, the image data reaches a fork between the path through which the image data is output as it is from the first IF 108 and the path through which the image data is input to the image processing circuit 302. These paths are set by the switch circuit 305 according to instructions from the CPU 117. The image processing circuit 302 performs processing for reducing the image data input to the image processing circuit 302 from a still image to a reduced still image. Thereafter, the image data is output from the second IF 109 via the image memory 303.

[0062] The reduced still image can have various resolutions. For example, by reducing the resolution to the same resolution as the LV image in the image processing circuit 302, the second DSP 115 in the subsequent stage can perform the same processing on the reduced still image and the LV image. Note that regarding the resolution of the reduced still image, as a result of the reduction process, it can be a resolution equal to or higher than the resolution of the LV image, or as a result of the reduction process, it can be a resolution lower than the resolution of the LV image.

[0063] In addition, in the image processing circuit 302, the resolution in the horizontal direction can also be reduced to the resolution of the LV image, and then the reduced still image is transmitted. In this case, although the second DSP 115 needs to perform the reduction process in the vertical direction, the circuit scale of the image processing circuit 302 can be reduced.

[0064] Figure 4B It is a data path diagram showing the data path through which the LV image passes. The pixel signal of the LV image output from the pixel array 207 is converted into a digital value by the AD conversion circuit 213 and accumulated in the image memory 303. Thereafter, the pixel signal is output from the second IF 109. This path is set by the switch circuit 305 according to an instruction from the CPU 117.

[0065] As described above, when the data path is configured as Figure 4A and Figure 4B shown, the high-resolution data that has been read out as a still image is output from the first IF 108, and the low-resolution data that has been read out as a reduced still image or an LV image is output from the second IF 109. Then, the still image is output to the first DSP111, and the reduced still image and the LV image are output to the second DSP 115. By performing processing on different types of data using different DSPs in the foregoing manner, the data processing of the still image and the data processing of the LV display can be executed completely in parallel.

[0066] Generally, when transmitting high-resolution data, the transmission takes time. In addition, it is difficult to share still images with the LV display because these still images are different from the LV image in terms of resolution and the like. For this reason, in the case of transmitting still images and LV images using the same IF and performing data processing by the same DSP, there is a problem that it is difficult to always update at a constant timing without intermittently eliminating the LV display. To solve this problem, the configuration of this embodiment allows for always updating at a constant timing without intermittently eliminating the LV display.

[0067] In addition, since the first IF 108 is required to transmit image data at a communication speed equal to or higher than the readout speed of still images, a high-speed interface needs to be prepared. In contrast, the second IF 109 for transmitting image data for LV display does not require such a high-speed interface. It is sufficient for the second IF 109 to transmit image data at a speed required for display on the display unit 119. That is, since an interface with a speed lower than that of the first IF 108 can be used as the second IF 109, the cost of the image sensor can be reduced. In this case, the following configuration is adopted: the reduced still image and the LV image are stored in the image memory 303 so that the transmission speed of the second IF 109 is slower than the readout speed.

[0068] Figure 5 is a timing chart showing the operation of the image sensor according to the first embodiment. Refer to Figure 5 , and the shooting operation and reduction operation of the still image and the shooting operation of the LV image in the first embodiment are described below.

[0069] Generally, the cycle required to update the LV display is different from the shooting cycle required to continuously shoot still images. This embodiment will be described by taking an example in which the duration of the shooting cycle required to continuously shoot still images is twice the cycle required to update the LV display. In the slashes in the figure, the thin dashed line indicates the scan for resetting the LV image in each unit pixel 206 included in the pixel array 207, and the thin solid line indicates the scan for reading out the LV image from the pixel array 207. In addition, the thick dashed line indicates the scan for resetting the still image in each unit pixel 206 included in the pixel array 207, and the thick solid line indicates the scan for reading out the still image from the pixel array 207.

[0070] When the vertical synchronization signal (hereinafter referred to as VD) is input to the image sensor 107 at time t500, the image sensor 107 reads out the LV image. In addition, with the start of the readout, writing of the LV image to the image memory 303 is started. In addition, the LV image is read out from the image memory 303, and data of the LV image starts to be output from the second IF 109 to the second DSP 115. The second DSP 115 starts processing for displaying the received image data on the display unit 119. When the readout of the LV image is completed at time t501, the writing to the image memory 303 is also stopped.

[0071] When a still image shooting instruction is issued at time t502, the CPU 117 controls the image sensor 107 to start accumulating the still image at the next VD timing or a later timing. At time t503, the readout of the LV image from the image memory 303 is completed, and the transmission of image data by the second IF 109 is also completed. At this timing or a later timing, the second DSP 115 completes the image display.

[0072] When VD is input to the image sensor 107 at time t504, the image sensor 107 reads out the LV image again. By repeatedly inputting VD to the image sensor 107 in the foregoing manner, the LV image can be repeatedly obtained. The time interval from time t500 to time t504 is regarded as one cycle, and the input of VD is repeated while maintaining this predetermined cycle; as a result, the LV image can be obtained at a constant cycle. By displaying the LV image that has been obtained at a predetermined cycle on the display unit 119 in the foregoing manner, a moving image with the screen updated at a predetermined cycle can be provided to the user as the LV display.

[0073] At time t505, the accumulation of the still image starts. When VD is input to the image sensor 107 at time t506 after one cycle (i.e., the foregoing predetermined cycle) has elapsed since time t504, the image sensor 107 reads out the still image. With the start of the readout, the still image is input to the image processing circuit 302, and at the same time, the output of data from the first IF 108 to the first DSP 111 starts. The first DSP 111 starts the process for recording the received image data as a still image. After the image processing circuit 302 has applied the reduction process to the image data input to the image processing circuit 302, the writing of the image data into the image memory 303 starts, and at the same time, the output of the image data from the second IF 109 to the second DSP 115 starts. The second DSP 115 starts the process for displaying the received image data on the display unit 119.

[0074] When the readout of the still image is completed at time t507, the input of the image data to the image processing circuit 302 also stops, and the writing into the image memory 303 from the image processing circuit 302 also stops. In addition, at the same time, the output of the image data from the first IF 108 to the first DSP 111 also stops. At this timing or a later timing, the first DSP 111 completes the process of the still image.

[0075] At time t508, the output of the image data written into the image memory 303 is completed, and the transmission of the image data from the second IF 109 is also completed. At this timing or a later timing, the second DSP 115 completes the image display. By controlling in the foregoing manner, even when a still image has been obtained, the periodic update of the LV display can be provided without interruption.

[0076] This embodiment assumes that the duration of the shooting cycle required for continuously shooting still images is twice the cycle required for updating the LV display. Therefore, even when the still image shooting instruction (continuous shooting instruction) continues from time t506 to time t509, the LV image is read out for the next frame.

[0077] When VD is input to the image sensor 107 at time t509, the image sensor 107 reads out the LV image. When VD is input to the image sensor 107 at the subsequent time t510, the image sensor 107 reads out the still image. After time t510, the operations from time t506 to t510 are repeated.

[0078] Note that although the timing chart of Figure 5 has been described for the case of alternately obtaining still images and LV images, the present invention is not limited thereto. For example, a configuration in which three LV images are obtained between acquisitions of still images is allowed. In this case, the period for obtaining the LV images may be the same as or different from that before the start of the shooting instruction.

[0079] As described above, according to the first embodiment, in a configuration capable of shooting still images and LV images, the still image can be further output as a reduced still image. In addition, the still image can be received in the first DSP 111 and the correction process for the still image can be applied thereto, and in parallel therewith, the reduced still image and the LV image can be received in the second DSP 115 and the correction process for the LV display image can be applied thereto. As a result, a periodic update of the display image without interval rejection can be provided to the display unit 119, and the shooting and correction processes of the still image can be performed in parallel. In addition, in such a configuration, an increase in the circuit scale of the second IF 109 can also be suppressed.

[0080] <Modification Example>

[0081] Although it is assumed in the first embodiment that the image processing circuit 302 reduces the still image, the present invention is not limited thereto. The image processing circuit 302 can perform other processes. First, imaging for LV display can be performed based on at least one of the LV image data and the reduced still image data, and then the luminance information of each color can be calculated. In addition, subject detection information can be calculated for the subject tracking function based on at least one of the LV image data and the reduced still image data.

[0082] In addition, photometric calculation for the automatic exposure adjustment function can be performed based on at least one of the LV image data and the reduced still image data. In addition, flicker detection information can be calculated for the flicker-free shooting function based on at least one of the LV image data and the reduced still image data.

[0083] In addition, phase difference information can be calculated for the automatic focus adjustment function based on at least one of the LV image data and the reduced still image data. Various other calculations are also possible. In addition, in the case of performing these various calculations, the image processing circuit 302 can also perform calculations on the image obtained as the LV image.

[0084] Therefore, by calculating various types of information in the image processing circuit 302, the configuration of the second DSP 115 can be simplified and made more versatile.

[0085] (Second Embodiment)

[0086] A second embodiment will be described for the case where the frame rate of still image capture during continuous shooting is higher than the frame rate of LV display. In this case, during the continuation of the still image capture instruction, no LV image is obtained, and LV display is generally performed based on the reduced still image generated from the still image.

[0087] Figure 6 is a timing chart showing the operation of the image sensor according to the second embodiment. Refer to Figure 6 , and the still image capture operation and reduction operation in the second embodiment will be described below.

[0088] As described above, generally, the period required to update the LV display is different from the capture period required to continuously capture still images. This embodiment will be described by taking an example where the duration of the period required to continuously capture still images is 1 / 4 of the period required to update the LV display. In the slanted lines in the figure, the thin dashed line indicates the scan for resetting the LV image in each unit pixel 206 included in the pixel array 207, and the thin solid line indicates the scan for reading out the LV image from the pixel array 207. In addition, the thick dashed line indicates the scan for resetting the still image in each unit pixel 206 included in the pixel array 207, and the thick solid line indicates the scan for reading out the still image from the pixel array 207. In addition, the description of the part that performs operations similar to the Figure 5 timing chart is omitted.

[0089] Note that Figure 6 the timing chart indicates a configuration where the readout speed of the still image is faster than the readout speed of the LV image. This can be achieved by installing additional column output lines for improving the readout speed as shown in Figure 2A and Figure 2B only in the still image mode. Note that the readout speed of the still image does not have to be faster than the readout speed of the LV image, and it is sufficient that the readout period of the still image is shorter than the duration of 1 / 4 of the period required to update the LV display.

[0090] When the vertical synchronization signal (VD) is input to the image sensor 107 at time t600, the image sensor 107 reads out the LV image and also starts updating the display of the display unit 119 at the same time. When a still image shooting instruction is issued at time t601, preparations for starting still image shooting are started. When the vertical synchronization signal (VD) is input to the image sensor 107 at time t602, the image sensor 107 reads out the LV image and also starts updating the display of the display unit 119 at the same time. The time interval (image acquisition period) from time t600 to time t602 has been set to be consistent with the update period of the LV display presented on the display unit 119.

[0091] At time t603, the image sensor 107 starts accumulating a still image. When the vertical synchronization signal (VD) is input to the image sensor 107 at time t604, the image sensor 107 reads out the still image. As the reading of the still image starts, the still image is input to the image processing circuit 302, and at the same time, image data starts to be output from the first IF 108 to the first DSP 111.

[0092] The first DSP 111 starts processing for recording the received image data as a still image. After the image processing circuit 302 has applied a reduction process to the image data input to the image processing circuit 302, writing of the image data to the image memory 303 starts, and at the same time, data starts to be output from the second IF 109 to the second DSP 115. The second DSP 115 starts processing for displaying the received image data on the display unit 119.

[0093] When the reading of the still image is completed at time t605, the input of the image data to the image processing circuit 302 also stops, and the writing from the image processing circuit 302 to the image memory 303 also stops. In addition, at the same time, the output of the image data from the first IF 108 to the first DSP 111 also stops. At this timing or a later timing, the first DSP 111 completes the processing of the still image. In addition, at the same time t605, accumulation for the next still image also starts.

[0094] Time t606 is the time when a time period equivalent to 1 / 4 of the update period of the LV display has elapsed since time t604. When the vertical synchronization signal (VD) is input to the image sensor 107 at time t606, the image sensor 107 reads out the still image. As the reading of the still image starts, image data starts to be output from the first IF 108 to the first DSP 111. The first DSP 111 starts processing for recording the received image data as a still image.

[0095] On the other hand, at this timing, a still image is not input to the image processing circuit 302. At this time, the reduced still image stored at time t605 is continuously read out from the image memory 303. When the reading of the still image is completed at time t607, the output of image data from the first IF 108 to the first DSP 111 is also stopped. By controlling the image sensor 107 in the foregoing manner, it is possible to execute processing in which a still image is not displayed even when a still image is captured.

[0096] Time t608 is a time when a period equivalent to the update period of the LV display has elapsed since time t604. Therefore, the still image read out as a result of inputting a vertical synchronization signal (VD) to the image sensor 107 at time t608 is input to the image processing circuit 302, and a reduced still image is generated therefor and used for display. Other processing is similar to the processing of the still image read out at time t604.

[0097] Here, between time t604 and time t608, four images including the images read out from time t606 to time t608 are read out. The readout period of the still image therebetween is equal to 1 / 4 of the update period of the LV display.

[0098] Note that although the acquisition period of the still image is 1 / 4 of the update period of the LV display in this embodiment, the present invention is not limited thereto. Further, in order to keep the update period of the LV display constant, it is sufficient to configure such that an integral multiple of the acquisition period of the still image coincides with one update period of the LV display.

[0099] For example, when the update period of the LV display is 60 fps, by setting the acquisition period of the still image to 120 fps or 180 fps, the update period of the LV display can be maintained at 60 fps. When the acquisition period of the still image is set to, for example, 100 fps and the update period of the LV display is 60 fps, there may be a case where the still image is not read out at the timing when an image for the LV display is to be acquired. This prevents the update period of the LV display from being maintained constant before and after a still image capture instruction.

[0100] As described above, according to the second embodiment, while keeping the period of the LV display constant, the acquisition period of the still image can be set to a period shorter than the period of the LV display.

[0101] (Third Embodiment)

[0102] A third embodiment will be described for the case of obtaining an image that is neither used as a still image nor for LV display while displaying an LV image. In a camera system, there are cases where an image that is not presented to the user is obtained to use information, for example, in flicker detection. Here, such an image is called a sub-scan image, and the scan of the image sensor for obtaining the sub-scan image is called sub-scanning. The pixel usage method, image data path, and timing chart of imaging will be described below for the sub-scan image.

[0103] According to Figure 2B the description, when obtaining an LV image that does not require high resolution, the row selection circuit 211 operates to sequentially select and read out the pixel array 207 at an interval of two rows. In this case, there are unused pixels in the pixel array 207. In view of this, for example, during sub-scanning, an operation is performed to sequentially select and read out such unused pixels at an interval of eight rows. In this way, pixels different from those used for the LV are used for the sub-scan image; thus, the accumulation for sub-scanning can be performed independently of the accumulation of the LV image.

[0104] In addition, by newly preparing column output lines for sub-scanning and connecting the pixels to the column output lines for sub-scanning during sub-scanning, the accumulation for sub-scanning can also be performed in parallel with the reading out of the LV image. Note that in this embodiment, it is assumed that such a configuration is not adopted, and the reading out timing of the LV image and the reading out timing of the sub-scan image are in an exclusive relationship.

[0105] The path diagram of the data of the sub-scan image is exactly the same as the path diagram of the LV image in Figure 4B and thus its description is omitted. The signal of the sub-scan image output from the pixel array 207 is converted into a digital value by the AD conversion circuit 213 and accumulated in the image memory 303. Thereafter, the signal is output from the second IF 109.

[0106] Figure 7 is a timing chart showing the operation of the image sensor according to the third embodiment. Referring to Figure 7 , the shooting operation of the LV image and the shooting operation of the sub-scan image in the third embodiment will be described below. In the slashes in the figure, the thin dashed lines indicate the scan for resetting the LV image in each unit pixel 206 included in the pixel array 207, and the thin solid lines indicate the scan for reading out the LV image from the pixel array 207. In addition, the thick dashed lines indicate the scan for resetting the sub-scan image in each unit pixel 206 included in the pixel array 207, and the thick solid lines indicate the scan for reading out the sub-scan image from the pixel array 207.

[0107] When the vertical synchronization signal (VD) is input to the image sensor 107 at time t700, the image sensor 107 reads out the LV image. In addition, with the start of the readout, writing of the LV image to the image memory 303 is started. Further, the LV image is read out from the image memory 303, and data output from the second IF 109 to the second DSP 115 is started. The second DSP 115 starts processing for displaying the received data on the display unit 119.

[0108] When the vertical synchronization signal (VD) is input to the image sensor 107 again at time t701, the image sensor 107 reads out the LV image again. The time interval from time t700 to time t701 is the update period of the LV display.

[0109] At time t702, accumulation of the sub-scan image is started. At time t703, the readout of the LV image is completed, and at the same time, the readout of the sub-scan image is started. Further, with the start of the readout of the sub-scan image, writing of the sub-scan image to the image memory 303 is started. At this time, the LV image is read out from the image memory 303 and output from the second IF 109. Therefore, at this timing, the sub-scan image is not read out from the image memory 303.

[0110] At time t704, accumulation of the next sub-scan image is started. At time t705, the readout of the sub-scan image is completed, and the writing to the image memory 303 is also completed. At this time, similarly, the sub-scan image is not read out from the image memory 303.

[0111] At time t706, the readout of the sub-scan image is started again. In this way, during the blanking period of the LV image acquisition, the sub-scan image is read out and written to the image memory 303. Thereafter, at time t707, the readout of the LV image from the image memory 303 is completed. At the same time, the readout of the sub-scan image from the image memory 303 is started. At time t708, the readout of the sub-scan image from the image memory 303 is completed.

[0112] When the vertical synchronization signal (VD) is input to the image sensor 107 at time t709, the image sensor 107 reads out the LV image. In this frame, the sub-scan image is not read out.

[0113] As described above, according to the third embodiment, while the LV image is acquired at a constant period, the sub-scan image that is not provided to the user as an image can be read out. Note that although this embodiment has presented a method in which the image processing circuit 302 does not process the sub-scan image, similar to the case of the LV image according to the modification of the first embodiment, various types of data can be obtained by processing the sub-scan image in the image processing circuit.

[0114] Other Embodiments

[0115] Embodiments of the present invention can also be implemented by the following method, that is, software (program) that executes the functions of the above embodiments is provided to a system or device through a network or various storage media, and a method for a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device to read and execute the program.

[0116] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims is to be accorded the broadest interpretation so as to encompass all such modifications, equivalent structures and functions.

Claims

1. A camera device, comprising: An image sensor comprising: a pixel section in which a plurality of pixels are arranged in a matrix, a readout section configured to read out a first image signal and a second image signal from the pixel section, wherein a resolution of the second image signal is lower than a resolution of the first image signal, an image processor, configured to generate a third image signal according to the first image signal, wherein the resolution of the third image signal is lower than the resolution of the first image signal, a first output unit, configured to output the first image signal, and A second output unit, used for outputting the second image signal or the third image signal; a first integrated circuit for processing a first image signal output from the first output portion of the image sensor; and The second integrated circuit is used to process the second image signal or the third image signal output from the second output portion of the image sensor.

2. The imaging device according to claim 1, wherein: The second image signal or the third image signal is used for real-time view display.

3. The imaging device according to claim 1, wherein: A resolution of the third image signal is lower than a resolution of the first image signal and is equal to or higher than a resolution of the second image signal.

4. The imaging device according to claim 3, wherein: The resolution of the third image signal is equal to the resolution of the second image signal.

5. The imaging device according to claim 1, wherein: A resolution of the third image signal is lower than a resolution of the second image signal.

6. The imaging device according to claim 1, further comprising: A switch component is used to switch between a first state in which the first image signal is output from the first output section and the third image signal is output from the second output section and a second state in which the second image signal is output from the second output section.

7. The imaging device according to claim 6, further comprising: A display unit is used to display an image, wherein the display unit displays an image based on the second image signal before a user issues a shooting instruction and displays an image based on the third image signal after the user issues the shooting instruction.

8. The imaging device according to claim 1, wherein: A communication speed at which a signal is output from the first output portion is faster than a communication speed at which a signal is output from the second output portion.

9. The imaging device according to claim 1, further comprising: A storage unit is arranged between the image processor and the second output unit.

10. The imaging device according to claim 1, further comprising: A transmission unit is used to transmit the signal processed by the first integrated circuit to the second integrated circuit.

11. The imaging device according to claim 1, wherein: The image processor calculates brightness information of an image using at least one of the second image signal and the third image signal.

12. The imaging device according to claim 1, wherein: The image processor detects a subject using at least one of the second image signal and the third image signal.

13. The imaging device according to claim 1, wherein: The image processor detects photometric information using at least one of the second image signal and the third image signal.

14. The imaging device according to claim 1, wherein: The image processor detects flicker using at least one of the second image signal and the third image signal.

15. The imaging device according to claim 1, wherein: The image processor detects phase difference information using at least one of the second image signal and the third image signal.

Images