Imaging apparatus

Abstract

To provide an imaging apparatus that can prevent chipping of a live view image when a user performs photographing while watching the live view image.SOLUTION: An imaging apparatus has: an image pick-up device comprising a pixel unit in which a plurality of pixels are arranged in matrix, a reading unit that reads first image signals from the pixel unit and second image signals having lower resolution than the first image signals, an image processing unit that generates, from the first signals, third image signals having lower resolution than the first image signals, and a first output unit that outputs the first image signals, and a second output unit that outputs the second image signals or the third image signals; a first integrated circuit that processes the first image signals output from the first output unit of the image pick-up device; and second integrated circuit that processes the second image signals or the third image signals output from the second output unit of the image pick-up device.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an imaging device.

Background Art

[0002] Generally, digital cameras and electronic devices equipped with a camera function often have a "Live View function (LV function)". The "Live View function" is a function that displays the captured image in real time on the display unit while shooting a moving image or the like. The user can adjust the shooting range for still image shooting or moving image shooting while checking the moving image displayed by the Live View function.

[0003] In order for the user to perform shooting as intended, it is important that the above-described adjustment of the shooting range is easy. For example, when shooting a still image while shooting a moving image, a moving image cannot be acquired in the frame in which the still image is shot. In this case, since the image displayed in the Live View cannot be acquired, the ease of adjusting the shooting range is reduced. In order to facilitate the adjustment of the shooting range, it is important to prevent the Live View frame from being missing even when shooting a still image. Therefore, it is known that different processing is performed on the still image from the image for Live View.

[0004] For example, Patent Document 1 discloses an image sensor having a first mode of transferring image data read from a pixel unit to a memory built in the image sensor and a second transfer mode of transferring it outside the image sensor.

[0005] Patent Document 2 discloses a method of data processing when a plurality of data are read in the image sensor. Specifically, a method is described in which still image data from the pixel unit is reduced and output to an image for Live View by a conversion circuit in the image sensor, and is output without reduction when image data for Live View is read from the pixel unit.

Prior Art Documents

Patent Documents

[0006] [Patent Document 1] Japanese Patent No. 6757199 [Patent Document 2] Japanese Unexamined Patent Application Publication No. 2023-106041 [Summary of the Invention] [Problems to be Solved by the Invention]

[0007] However, in the technology disclosed in Patent Document 1 described above, image data with a large data amount such as a still image cannot be used for live view display, so the display frame of the live view is missing.

[0008] Further, although Patent Document 2 discloses the processing of data in the imaging element, the processing method of the signal processing circuit connected to the imaging element is not clarified. Also, the configuration method of an appropriate output interface for each of the still image and the live view image is not clarified.

[0009] The present invention has been made in view of the above problems, and an object thereof is to provide an imaging device capable of preventing the loss of a live view image when a user takes a picture while viewing the live view image. [Means for Solving the Problems]

[0010] The imaging device according to the present invention is an image sensor, comprising a pixel unit in which a plurality of pixels are arranged in a matrix, a first image signal from the pixel unit, a reading unit that reads a second image signal having a lower resolution than the first image signal, an image processing unit that generates a third image signal having a lower resolution than the first image signal from the first image signal, a first output unit that outputs the first image signal, and a second output unit that outputs the second image signal or the third image signal; an image sensor; a first integrated circuit that processes the first image signal output from the first output unit of the image sensor; and a second integrated circuit that processes the second image signal or the third image signal output from the second output unit of the image sensor.

Effect of the Invention

[0011] According to the present invention, when a user takes a picture while viewing a live view image, it is possible to prevent the loss of the live view image.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0014] (First Embodiment) FIG. 1 is a block diagram showing the configuration of an imaging device according to a first embodiment of the present invention. As shown in FIG. 1, the imaging device 150 includes an imaging optical system 104. The imaging optical system 104 includes a first lens 100, a diaphragm 101, a second lens 102, and a third lens 103. The first lens 100 is disposed at the tip of the imaging optical system 104. The diaphragm 101 adjusts the amount of light during shooting by adjusting its aperture diameter. The aperture diameter of the diaphragm 101 is adjusted by driving a diaphragm actuator 126. The second lens 102 and the third lens 103 move forward and backward in the optical axis direction by driving a focus actuator 124 described later to adjust the focus of the imaging optical system 104.

[0015] Behind the imaging optical system 104, a focal plane shutter 105, an optical low-pass filter 106, and an image sensor 107 are arranged in this order. The focal plane shutter 105 has a function of adjusting the exposure time during still image shooting. The optical low-pass filter 106 has a function of reducing false colors and moiré in the captured image. The image sensor 107 converts the optical image of the subject formed by the imaging optical system 104 into an electrical signal.

[0016] The imaging device 150 includes a first integrated circuit 110 and a second integrated circuit 114. The first integrated circuit 110 includes a first DSP (Digital Signal Processing Unit) 111. The first DSP 111 is connected to the imaging device 107 via the first IF (interface) 108 as the first output unit, and receives the image data transmitted from the imaging device 107 and performs image processing. Examples of the image processing performed by the first DSP 111 include correction processing of still image data and the like.

[0017] The first RAM 112 is connected to the first integrated circuit 110 and stores the image data processed by the first DSP 111. In this embodiment, the first RAM 112 is arranged outside the first integrated circuit 110, but a configuration in which part or all of its functions are mounted on the first integrated circuit 110 or the first DSP 111 may also be adopted.

[0018] At least the second DSP (digital signal processing unit) 115 and the CPU 117 are mounted on the second integrated circuit 114. In this embodiment, the shutter drive circuit 122, the focus drive circuit 123, and the aperture drive circuit 125 are also mounted on the second integrated circuit 114, but they may be mounted on separate integrated circuits. The second DSP 115 is connected to the imaging device 107 via the second IF (interface) 109 as the second output unit, and receives the image data transmitted from the imaging device 107 and performs image processing.

[0019] Examples of the image processing performed by the second DSP 115 include correction processing of LV image data and generation of display images for display on the display unit 119.

[0020] The second RAM 116 is connected to the second integrated circuit 114, stores the image data processed by the second DSP 115, and functions as a work memory when the CPU 117 described later operates.

[0021] In the present embodiment, both functions are realized using a RAM. However, other types of memory can also be used as long as it has a sufficiently high access speed and operates without problems. Also, in the present embodiment, the second RAM 116 is arranged outside the second integrated circuit 114, but a part or all of its functions may be incorporated into the second integrated circuit 114, the second DSP 115, or the CPU 117.

[0022] Here, the first integrated circuit 110 and the second integrated circuit 114 are connected by a third IF (interface) 113. The still image data corrected by the first DSP 111 in the first integrated circuit 110 is transmitted to the second integrated circuit 114 via the third IF 113 and recorded on the recording medium 120 via the second DSP 115.

[0023] The CPU 117 comprehensively controls the operation of the imaging device 150 and executes a program for controlling each part of the imaging device. The CPU 117 controls the shooting timing of video and still images, which will be described later, by performing various settings on the imaging element 107. Also, the CPU 117 has a function of adjusting the focus of the imaging optical system 104 by driving and controlling the focus drive circuit 123, which will be described later, using the calculation result output from the second DSP 115.

[0024] Connected to the CPU 117 are an operation unit 118, a display unit 119, a recording medium 120, a ROM 121, a shutter drive circuit 122, a focus drive circuit 123, and a diaphragm drive circuit 125.

[0025] The operation unit 118 has operation members such as buttons and levers. The user inputs instructions to the CPU 117 by operating the operation members of the operation unit 118 to perform shooting. The operation unit 118 includes a still image shooting start button. When the CPU 117 detects a pressing operation of the still image shooting start button by the user, it controls to perform still image shooting after a certain period of time has elapsed.

[0026] The display unit 119 displays the images and menus processed by the second DSP 115. As the display unit 119, in addition to a display, an electronic viewfinder (EVF) or the like may be used. The recording medium 120 is a detachable recording medium for recording still image data and moving image data, and can be realized by, for example, a memory card or the like. The ROM 121 stores programs and the like for the CPU 117 to control the operations of each unit.

[0027] The shutter drive circuit 122 drives and controls the focal plane shutter 105. The focus drive circuit 123 changes the focal position of the imaging optical system 104. That is, the focus drive circuit 123 drives and controls the focus actuator 124 based on the output of the CPU 117, and performs focus adjustment by moving the focus lens (the second lens 102, the third lens 103) in the optical axis direction. The aperture drive circuit 125 drives and controls the aperture actuator 126 to change the aperture diameter of the aperture 101, thereby adjusting the amount of light incident on the imaging element 107.

[0028] FIG. 2(a) is a circuit diagram of a unit pixel, and FIG. 2(b) is a diagram showing the configuration of a pixel array and a peripheral circuit. With reference to FIG. 2, the circuit particularly around the pixel of the imaging element 107 will be described.

[0029] First, with reference to FIG. 2(a), the circuit configuration of the unit pixel 206 will be described. The photodiode (hereinafter, PD) 200 as a photoelectric conversion unit is disposed under the microlens, and generates and accumulates charges corresponding to the incident light amount. The transfer switch 201 is a switch controlled by the control signal φtx. By setting the value of the control signal φtx to High (hereinafter, H), the transfer switch 201 is turned on, and the charges accumulated in the PD 200 are transferred to the floating diffusion unit (hereinafter, FD) 202.

[0030] The reset switch 203 is a switch controlled by the control signal φres and initializes FD202. By setting both the control signals φtx and φres to H, both PD200 and FD202 are set to the power supply voltage (VDD), and the reset operation of the pixel is performed.

[0031] The transistor 204 that functions as a pixel amplifier is connected to the select switch 205 and is also connected to the constant current source 209 (see Fig. 2(b)) via the column output line 208 (see Fig. 2(b)). When the value of the control signal φsel of the select switch 205 becomes H, the transistor 204 is connected to the constant current source 209 to form a pixel amplifier. The charge transferred from PD200 to FD202 is converted by the pixel amplifier into a voltage value corresponding to the amount of charge and output as a pixel signal to the column output line 208.

[0032] Next, with reference to Fig. 2(b), the overall configuration of the imaging device 107 will be described. The pixel array (pixel section) 207 has a plurality of unit pixels 206 arranged in a matrix. Specifically, (m + 1) unit pixels 206 are arranged in the horizontal direction and (n + 1) unit pixels 206 are arranged in the vertical direction. Note that m and n are natural numbers. With such a configuration, the pixel array 207 is formed by arranging a plurality of photoelectric conversion elements in a matrix.

[0033] The drive pulse generation circuit 210 generates pulses for performing the reset operation and readout operation of the unit pixel 206. The pulses generated by the drive pulse generation circuit 210 are supplied to the pixel drive circuit 212. The row selection circuit 211 selects a specific row to which the pulses generated by the drive pulse generation circuit 210 are supplied and sets the selected specific row to the pixel drive circuit 212. The pixel drive circuit 212 supplies the pulses generated by the drive pulse generation circuit 210 to the specific row selected and set by the row selection circuit 211.

[0034] The row selection circuit 211 can read signals from the pixel array 207 in multiple ways. For example, when reading pixel signals used for a still image, since high-resolution image data is required, the row selection circuit 211 selects each row of the pixel array 207 to read the pixel signals.

[0035] On the other hand, when reading pixel signals used for a moving image for LV (live view moving image), since the resolution is not required so much, the row selection circuit 211 performs an operation of selecting every other row of the pixel array 207 to read the pixel signals. In this case, the pixel signals of the third row will be read after the zeroth row. By reading the pixel signals in this way, it is possible to obtain LV image data with a vertical resolution reduced to one-third compared to the still image described above.

[0036] When comparing the still image and the LV image, the still image is excellent in resolution, while the LV image has the feature that the reading time is shortened by the amount of the reduced vertical resolution, and the power required for reading can be reduced. Thus, by the row selection control mode of the row selection circuit 211, a plurality of types of images suitable for each purpose can be obtained. Note that the configuration is not limited to thinning out the pixel rows and reading them. For example, it may be configured to obtain LV image data with a low vertical resolution by adding the pixel signals of a plurality of pixel rows.

[0037] The pixel signals are output row by row on the column output line 208 according to the pulses supplied from the pixel driving circuit 212. The constant current source 209 is combined with the transistor 204 to form a source follower circuit. The AD conversion circuit 213 converts the analog signal output to the column output line 208 into a digital signal.

[0038] In the present embodiment, the pixel signals are read out from the pixel array 207 one row at a time. However, the present invention is not limited to this. For example, if two output lines are provided for each column, by changing the connection between the pixels and the output lines for even rows and odd rows, it becomes possible to read out the pixel signals of two rows simultaneously for both the still image and the LV image. By adopting a configuration that can read out pixel signals of multiple rows simultaneously in this way, the readout time can be shortened. On the other hand, the complexity of the circuit increases as the number of output lines increases. Therefore, it is desirable to determine an appropriate number of output lines according to the usage method.

[0039] FIG. 3 is a diagram showing the configuration of an image pickup device including a plurality of semiconductor substrates. With reference to FIG. 3, the configuration of the image pickup device 107 of the present embodiment will be described.

[0040] Generally, as a method of configuring an image pickup device, a method of configuring a single image pickup device by providing separate circuits on a plurality of semiconductor substrates and laminating and connecting those plurality of semiconductor substrates to each other is known. Also in the image pickup device 107 of the present embodiment, a single image pickup device chip is configured by laminating the first semiconductor substrate 300 and the second semiconductor substrate 301 on each other.

[0041] With reference to FIG. 3(a), the configuration of the circuits mounted on each of the first semiconductor substrate 300 and the second semiconductor substrate 301 will be described.

[0042] A pixel array 207 is provided on the first semiconductor substrate 300. Also, on the second semiconductor substrate 301, an AD conversion circuit 213, an image processing circuit 302, an image memory 303, a switch circuit 305, a first IF 108, and a second IF 109 are provided.

[0043] The image processing circuit 302 is information processing means for processing the information read out from the pixel array 207 and converting it into other information. Although various processes can be performed, in the present embodiment, it is assumed that the image processing circuit 302 converts the still image into an LV image by reducing it.

[0044] The image memory 303 is a memory that temporarily stores pixel data read from the pixel array 207 and converted into digital values by the AD conversion circuit 213, and image data processed by the image processing circuit 302. In this embodiment, any memory may be adopted as long as it can store data sufficiently faster than the reading speed of pixel signals from the pixel array 207 and the processing speed in the image processing circuit 302, and can read out the stored data once.

[0045] The first IF 108 is connected to the AD conversion circuit 213 and outputs the read image data to the outside without temporarily storing it in the image memory 303. The second IF 109 is connected to the image memory 303 and outputs the image data temporarily stored in the image memory 303 to the outside.

[0046] The switch circuit 305 is controlled by the CPU 117 and has a function of switching to which of the image processing circuit 302, the image memory 303, and the first IF 108 to output the image data output from the AD conversion circuit 213. This switching will be described later with reference to FIG. 4.

[0047] Note that circuits other than the illustrated circuits may be arranged between the respective blocks. For example, a correction processing circuit that performs correction processing on the image read by the AD conversion circuit 213 may be arranged between the AD conversion circuit 213 and the first IF 108 or between the AD conversion circuit 213 and the image processing circuit 302.

[0048] FIG. 3(b) is a schematic diagram showing an example in which the first semiconductor substrate 300 and the second semiconductor substrate 301 are stacked to form the image sensor 107 which is a single chip. The state where the first semiconductor substrate 300 is stacked on the second semiconductor substrate 301 is shown. Any known technique may be used for electrically connecting the stacked semiconductor substrates.

[0049] Note that, in this embodiment, a configuration is shown in which the pixel array 207 is mounted on the first semiconductor substrate 300 and all other circuit blocks are mounted on the second semiconductor substrate 302. However, the present invention is not limited to this. For example, a configuration in which the pixel array and the circuit blocks are all mounted on the same semiconductor substrate may be used, or a configuration in which the circuit blocks are further mounted on a plurality of semiconductor substrates may be used.

[0050] FIG. 4 is a diagram schematically showing the paths of the image data of the still image, the reduced still image, and the LV image. With reference to FIG. 4, an explanation will be given of through which circuit blocks (data paths) the image data of each of the still image, the reduced still image, and the LV image is output to the outside of the image sensor 107.

[0051] FIG. 4(a) is a data path diagram for explaining the data path through which the still image and the reduced still image pass.

[0052] The pixel signal of the still image is read out from the pixel array 207 and converted into a digital value by the AD conversion circuit 213. Thereafter, this image data is branched into a path that is directly output from the first IF 108 and a path that is input to the image processing circuit 302. This path is set by the switch circuit 305 according to an instruction from the CPU 117. The image data input to the image processing circuit 302 is reduced from the still image to the reduced still image in the image processing circuit 302. Thereafter, it is output from the second IF 109 via the image memory 303.

[0053] The resolutions of the reduced still images can be various. For example, by reducing the image in the image processing circuit 302 to the same resolution as the LV image, the processing in the subsequent second DSP 115 can be made common with the LV image. Note that the resolution of the reduced still image may be reduced to a resolution equal to or higher than the resolution of the LV image, or may be reduced to a resolution lower than the resolution of the LV image.

[0054] Also, in the image processing circuit 302, it is also conceivable to reduce only the horizontal resolution to the resolution of the LV image and then transmit it. In this case, although the second DSP 115 needs to perform vertical reduction processing, the circuit scale of the image processing circuit 302 can be reduced.

[0055] FIG. 4(b) is a data path diagram for explaining the data path through which the LV image passes. The pixel signal of the LV image output from the pixel array 207 is converted into a digital value by the AD conversion circuit 213 and stored in the image memory 303. Then, it is output from the second IF 109. This path is set by the switch circuit 305 according to an instruction from the CPU 117.

[0056] As described above, when the data path is configured as shown in FIG. 4, the high-resolution data read as a still image is output from the first IF 108, and the low-resolution data read as a reduced still image and an LV image is output from the second IF 109. Then, the still image is output to the first DSP 111, and the reduced still image and the LV image are output to the second DSP 115. In this way, by separating the DSPs for processing according to the type of data, the data processing for still images and the data processing for LV display can be performed completely in parallel.

[0057] Generally, when transmitting high-resolution data, it takes a long time for transmission. Also, since the still image has a different resolution and the like from the LV image, it is difficult to share it for LV display. Therefore, when transmitting a still image and an LV image using the same IF and performing data processing with the same DSP, there is a problem that it becomes difficult to update at a constant timing without thinning out the LV display. In response to this problem, with a configuration like this embodiment, it becomes possible to update at a constant timing without thinning out the LV display.

[0058] Furthermore, regarding the first IF108, since it is required to transmit image data at a communication speed equal to or higher than the reading speed of still images, it is necessary to provide a high-speed interface. In contrast, the second IF109 for transmitting image data for LV display does not require such a high-speed interface. It is sufficient for the second IF109 to be able to send image data at a speed required for display on the display unit 119. That is, since the second IF109 can adopt an interface slower than the first IF108, the cost of the imaging device can be reduced. In this case, in order to make the transmission speed by the second IF109 slower than the reading speed, a configuration is adopted in which the reduced still image and the LV image are saved in the image memory 303 at once.

[0059] FIG. 5 is a timing chart showing the operation of the imaging device in the first embodiment. With reference to FIG. 5, the still image shooting operation and reduction operation, and the LV image shooting operation in the first embodiment will be described.

[0060] Generally, the period required for updating the LV display and the shooting period required for continuous shooting of still images are different. In this embodiment, an example will be described in which the shooting period required for continuous shooting of still images is twice as long as the period required for updating the LV display. Among the hatched lines in the figure, the thin dashed line represents the reset scan of the LV image in each unit pixel 206 included in the pixel array 207, and the thin solid line represents the read scan of the LV image from the pixel array 207. Furthermore, the thick dashed line represents the reset scan of the still image in each unit pixel 206 included in the pixel array 207, and the thick solid line represents the read scan of the still image from the pixel array 207.

[0061] At time t500, when a vertical synchronization signal (hereinafter referred to as VD) is input to the imaging device 107, the imaging device 107 reads out the LV image. At the same time as the start of the reading, writing of the LV image to the image memory 303 is started. Further, the LV image is read out from the image memory 303, and output of the LV image data from the second IF 109 to the second DSP 115 is started. In the second DSP 115, processing for displaying the received image data on the display unit 119 is started. When the reading of the LV image is completed at time t501, writing to the image memory 303 also stops.

[0062] When a still image shooting instruction is given at time t502, the CPU 117 controls the imaging device 107 to start accumulating the still image at the timing after the timing of the next VD. At time t503, the reading of the LV image from the image memory 303 is completed, and the transmission of the image data by the second IF 109 is also completed. At the subsequent timing, the second DSP 115 completes the display of the image.

[0063] At time t504, when VD is input to the imaging device 107, the imaging device 107 reads out the LV image again. By repeating the input of VD to the imaging device 107 in this way, the LV image can be repeatedly acquired. Taking the time interval from time t500 to time t504 as one cycle, and repeating the input of VD so as to follow this predetermined cycle, the LV image can be acquired at a fixed cycle. By displaying the LV image acquired at such a predetermined cycle on the display unit 119, a moving image with the screen updated at a predetermined cycle can be provided to the user as LV display.

[0064] At time t505, the accumulation of a still image starts. When VD is input to the imaging device 107 at time t506 after one period of the above-described predetermined period has elapsed since time t504, the imaging device 107 reads out the still image. When starting the readout, the still image is input to the image processing circuit 302, and at the same time, the output of data from the first IF 108 to the first DSP 111 is started. In the first DSP 111, processing for recording the received image data as a still image is started. The image data input to the image processing circuit 302 is subjected to reduction processing in the image processing circuit 302 and then starts to be written into the image memory 303. At the same time, the output of image data from the second IF 109 to the second DSP 115 is started. In the second DSP 115, processing for displaying the received image data on the display unit 119 is started.

[0065] When the readout of the still image is completed at time t507, the input of image data to the image processing circuit 302 is also stopped, and the writing from the image processing circuit 302 to the image memory 303 is also stopped. At the same time, the output of image data from the first IF 108 to the first DSP 111 is also stopped. At a timing after this, the first DSP 111 completes the processing of the still image.

[0066] At time t508, the output of the image data written in the image memory 303 is completed, and the transmission of the image data from the second IF 109 is also completed. At a timing after this, the second DSP 115 completes the display of the image. By controlling in this way, even when a still image is acquired, it is possible to provide the periodic LV display update without interruption.

[0067] In the present embodiment, it is assumed that the shooting period required for continuous shooting of still images is twice as long as the period required for the update of the LV display. Therefore, even when a still image shooting instruction (continuous shooting instruction) continues from time t506 to time t509, the readout of the next frame is an image for LV.

[0068] At time t509, when VD is input to the imaging device 107, the imaging device 107 reads out an LV image. Further, at a subsequent time t510, when VD is input to the imaging device 107, the imaging device 107 reads out a still image. After time t510, the operations at times t506 to t510 are repeated.

[0069] Note that in the timing chart of FIG. 5, the case of alternately acquiring a still image and an LV image has been described, but the present invention is not limited to this. For example, a configuration in which three LV images are acquired during the acquisition of a still image may be adopted. In this case, the period for acquiring the LV image may be the same as or different from that before the start of the shooting instruction.

[0070] As described above, according to the first embodiment, in a configuration capable of shooting a still image and an LV image, the still image can be further output as a reduced still image. Further, the still image is received by the first DSP 111 and correction processing for the still image is performed, and in parallel therewith, the reduced still image and the LV image are received by the second DSP 115 and correction processing for the LV display image is performed. As a result, while providing periodic update of the display image that is not thinned out for the display unit 119, it is possible to perform still image shooting and correction processing in parallel. Further, even in such a configuration, an increase in the circuit scale of the second IF 109 can be suppressed.

[0071] <Modification Example> In the first embodiment, the image processing circuit 302 is configured to reduce a still image, but the present invention is not limited thereto. The image processing circuit 302 may perform other processing. Development for LV display may be performed first based on at least one of the LV image data and the reduced still image data, and luminance information for each color may be calculated. Further, subject detection information may be calculated for the subject tracking function based on at least one of the LV image data and the reduced still image data.

[0072] Further, photometric calculations for the automatic exposure adjustment function may be performed based on at least one of the LV image data and the reduced still image data. Further, flicker detection information may be calculated for the flickerless shooting function based on at least one of the LV image data and the reduced still image data.

[0073] Also, phase difference information may be calculated for the autofocus adjustment function based on at least one of the LV image data and the reduced still image data. Various other calculations are possible. Also, when performing various calculations in this way, it is possible to perform calculations on the image acquired as the LV image by the image processing circuit 302.

[0074] In this way, by calculating various information by the image processing circuit 302, it is possible to simplify the configuration of the second DSP 115 and make it a more general-purpose configuration.

[0075] (Second Embodiment) In the second embodiment, a case where the shooting frame rate of still images in continuous shooting is higher than the frame rate of LV display will be described. In that case, while the still image shooting instruction continues, LV display is performed based on the reduced still images generated entirely from still images without acquiring LV images.

[0076] FIG. 6 is a timing chart showing the operation of the imaging device in the second embodiment. With reference to FIG. 6, the still image shooting operation and reduction operation in the second embodiment will be described.

[0077] As described above, generally, the period required for updating the LV display and the shooting period required for continuous shooting of still images are different. In the present embodiment, an example will be described in which the period required for continuous shooting of still images is one-fourth the length of the period required for updating the LV display. Among the hatched lines in the figure, the thin dashed line represents the reset scan of the LV image in each unit pixel 206 included in the pixel array 207, and the thin solid line represents the read scan of the LV image from the pixel array 207. Further, the thick dashed line represents the reset scan of the still image in each unit pixel 206 included in the pixel array 207, and the thick solid line represents the read scan of the still image from the pixel array 207. Also, the description of the part that operates in the same manner as the timing chart of FIG. 5 will be omitted.

[0078] Note that in the timing chart of FIG. 6, the read speed of the still image is faster than the read speed of the LV image. This can be realized by enabling the addition of the column output line for improving the read speed as described in FIG. 2 only in the still image mode. However, it is not necessarily required that the read of the still image is faster than the read of the LV image, and the read time of the still image may be shorter than one-fourth the length of the period required for updating the LV display.

[0079] At time t600, when the vertical synchronization signal (VD) is input to the image sensor 107, the image sensor 107 reads the LV image, and at the same time, the update of the display on the display unit 119 also starts. At time t601, when a still image shooting instruction is given, the preparation for starting the still image shooting is started. At time t602, when the vertical synchronization signal (VD) is input to the image sensor 107, the image sensor 107 reads the LV image, and at the same time, the update of the display on the display unit 119 also starts. The time interval (image acquisition period) from time t600 to time t602 is set to match the update period of the LV display displayed on the display unit 119.

[0080] At time t603, the imaging device 107 starts accumulating a still image. When a vertical synchronization signal (VD) is input to the imaging device 107 at time t604, the imaging device 107 reads out the still image. While starting to read out the still image, the still image is input to the image processing circuit 302, and at the same time, the output of image data from the first IF108 to the first DSP111 is started.

[0081] In the first DSP111, processing for recording the received image data as a still image is started. The image data input to the image processing circuit 302 is subjected to reduction processing in the image processing circuit 302 and then starts to be written into the image memory 303. At the same time, the output of data from the second IF109 to the second DSP115 is started. In the second DSP115, processing for displaying the received image data on the display unit 119 is started.

[0082] When the readout of the still image is completed at time t605, the input of image data to the image processing circuit 302 is also stopped, and the writing from the image processing circuit 302 to the image memory 303 is also stopped. At the same time, the output of image data from the first IF108 to the first DSP111 is also stopped. At a timing after this, the first DSP111 completes the processing of the still image. Further, at time t605 as well, accumulation for the next still image is started.

[0083] Time t606 is the time when a time of 1 / 4 of the update period of the LV display has elapsed from time t604. When a vertical synchronization signal (VD) is input to the imaging device 107 at time t606, the imaging device 107 reads out the still image. While starting to read out the still image, the output of image data from the first IF108 to the first DSP111 is started. In the first DSP111, processing for recording the received image data as a still image is started.

[0084] On the other hand, at this timing, no still image is input to the image processing circuit 302. At this time, the reduced still image that was stored at time t605 is continuously read from the image memory 303. When the reading of the still image is completed at time t607, the output of the image data from the first IF108 to the first DSP111 also stops. By controlling the imaging device 107 in this way, it is possible to execute processing such that no still image is displayed even when a still image is captured.

[0085] Time t608 is the time when the same amount of time as the update period of the LV display has elapsed since time t604. Therefore, for the still image read out when the vertical synchronization signal (VD) is input to the imaging device 107 at time t608, it is input to the image processing circuit 302, a reduced still image is generated, and it is used for display. Other processing is the same as the still image read out at time t604.

[0086] Here, between time t604 and time t608, four images are read out, including the images read out at times t606 to t608. The readout period of the still image during this period is equal to 1 / 4 of the update period of the LV display.

[0087] Note that in this embodiment, the acquisition period of the still image is set to 1 / 4 times the update period of the LV display, but the present invention is not limited to this. Also, in order to keep the update period of the LV display constant, it is advisable to set it such that an integer multiple of the acquisition period of the still image coincides with one period of the update period of the LV display.

[0088] For example, when the update period of the LV display is 60 fps, by setting the acquisition period of the still image to 120 fps or 180 fps, it is possible to keep the update period of the LV display at 60 fps. If the update period of the LV display is 60 fps, but the acquisition period of the still image is set to 100 fps or the like, there may be a case where the still image is not read out at the timing when an image for the LV display is desired to be acquired. Therefore, it becomes impossible to keep the update period of the LV display constant before and after the still image shooting instruction.

[0089] As described above, according to the second embodiment, it is possible to set the acquisition period of the still image to a shorter period while keeping the period of the LV display constant.

[0090] (Third Embodiment) In the third embodiment, a case where an image that is not used for either still image or LV display is acquired while displaying the LV image will be described. In a camera system, for example, in order to use information for flicker detection or the like, there may be a case where an image that is not shown to the user is acquired. Here, such an image is called a sub-scanned image, and the scanning of the imaging device for acquiring the sub-scanned image is called sub-scanning. Hereinafter, the method of using pixels, the path of image data, and the imaging timing chart for the sub-scanned image will be described.

[0091] In the description of FIG. 2(b), when acquiring the LV image for which resolution is not required, it was described that the row selection circuit 211 selects and reads out the pixel array 207 one row at a time, skipping every other row. In this case, there will be pixels in the pixel array 207 that are not used. Therefore, for example, in sub-scanning, an operation of selecting and reading out such unused pixels one row at a time, skipping every eight rows, is performed. In this way, since pixels different from those for the LV image are used for the sub-scanned image, it is possible to perform sub-scanning accumulation independently of the accumulation of the LV image.

[0092] Furthermore, if a new output line for sub-scanning is prepared and the pixels are connected to the output line for sub-scanning during sub-scanning, it is also possible to perform sub-scanning accumulation in parallel with the reading of the LV image. However, in this embodiment, such a configuration is not adopted, and it is assumed that the reading timing of the LV image and the reading timing of the sub-scanned image are in an exclusive relationship.

[0093] Regarding the route diagram of the data of the sub-scanned image, since it is exactly the same as the route diagram of the LV image in Fig. 4(b), it is omitted. The signal of the sub-scanned image output from the pixel array 207 is converted into a digital value by the AD conversion circuit 213 and stored in the image memory 303. Then, it is output from the second IF 109.

[0094] Fig. 7 is a timing chart showing the operation of the imaging device in the third embodiment. With reference to Fig. 7, the imaging operation of the LV image and the imaging operation of the sub-scanned image in the third embodiment will be described. Among the hatched lines in the figure, the thin dashed line represents the reset scan of the LV image in each unit pixel 206 included in the pixel array 207, and the thin solid line represents the read scan of the LV image from the pixel array 207. Further, the thick dashed line represents the reset scan of the sub-scanned image in each unit pixel 206 included in the pixel array 207, and the thick solid line represents the read scan of the sub-scanned image from the pixel array 207.

[0095] At time t700, when a vertical synchronization signal (VD) is input to the imaging device 107, the imaging device 107 reads out the LV image. Also, when starting the readout, it starts writing the LV image to the image memory 303. Further, it reads out the LV image from the image memory 303 and starts outputting data from the second IF 109 to the second DSP 115. The second DSP 115 starts processing the received data for display on the display unit 119.

[0096] At time t701, when a vertical synchronization signal (VD) is input to the imaging device 107 again, the imaging device 107 reads out the LV image again. The time interval from time t700 to time t701 is the update period of the LV display.

[0097] At time t702, the accumulation of the sub-scan image starts. At time t703, when the reading of the LV image is completed, the reading of the sub-scan image is started simultaneously. Also, when starting the reading of the sub-scan image, the writing of the sub-scan image to the image memory 303 is started. At this time, the LV image is read from the image memory 303 and output from the second IF109. Therefore, the sub-scan image is not read from the image memory 303 at this timing.

[0098] At time t704, the accumulation of the next sub-scan image starts. At time t705, the reading of the sub-scan image is completed, and the writing to the image memory 303 is also completed. Even at this point, the sub-scan image is not read from the image memory 303.

[0099] At time t706, the reading of the sub-scan image is started again. In this way, during the blanking period for acquiring the LV image, the reading of the sub-scan image and the writing to the image memory 303 are performed. Then, at time t707, the reading of the LV image from the image memory 303 is completed. At the same time, the reading of the sub-scan image from the image memory 303 is started. At time t708, the reading of the sub-scan image from the image memory 303 is completed.

[0100] At time t709, when a vertical synchronization signal (VD) is input to the imaging device 107, the imaging device 107 reads the LV image. In this frame, the sub-scan image is not read.

[0101] As described above, according to the third embodiment, while acquiring the LV image at a fixed cycle, it is possible to read the sub-scan image that is not provided to the user as an image. In this embodiment, a method in which the sub-scan image is not processed by the image processing circuit 302 is shown, but it is the same as in the case of the LV image in the modification of the first embodiment that various data may be obtained by processing the sub-scan image by the image processing circuit.

[0102] The disclosure of this specification includes the following imaging device and its control method.

[0103] (Item 1) An imaging device, a pixel section in which a plurality of pixels are arranged in a matrix, a reading section that reads a first image signal from the pixel section and a second image signal having a lower resolution than the first image signal, an image processing section that generates a third image signal having a lower resolution than the first image signal from the first image signal, a first output section that outputs the first image signal, a second output section that outputs the second image signal or the third image signal, and an imaging device comprising the same, a first integrated circuit that processes the first image signal output from the first output section of the imaging device, a second integrated circuit that processes the second image signal or the third image signal output from the second output section of the imaging device, and an imaging apparatus characterized by having the same.

[0104] (Item 2) The imaging apparatus according to claim 1, wherein the second image signal or the third image signal is used for live view display.

[0105] (Item 3) The imaging apparatus according to item 1 or 2, wherein the resolution of the third image signal is lower than the resolution of the first image signal and equal to or higher than the resolution of the second image signal.

[0106] (Item 4) The imaging apparatus according to item 3, wherein the resolution of the third image signal is equal to the resolution of the second image signal.

[0107] (Item 5) The imaging apparatus according to claim 1, wherein the resolution of the third image signal is lower than the resolution of the second image signal.

[0108] (Item 6) The imaging device according to any one of claims 1 to 5, further comprising switching means for switching between a first state of outputting the first image signal from the first output unit and outputting the third image signal from the second output unit, and a second state of outputting the second image signal from the second output unit.

[0109] (Item 7) The imaging device according to any one of items 1 to 6, further comprising a display unit for displaying an image, wherein before a shooting instruction is given by a user, the display unit displays an image based on the second image signal, and after a shooting instruction is given by the user, the display unit displays an image based on the third image signal.

[0110] (Item 8) The imaging device according to any one of items 1 to 7, wherein a communication speed for outputting a signal from the first output unit is faster than a communication speed for outputting a signal from the second output unit.

[0111] (Item 9) The imaging device according to any one of items 1 to 8, further comprising a storage unit disposed between the image processing unit and the second output unit.

[0112] (Item 10) The imaging device according to any one of items 1 to 9, further comprising a transmission unit for transmitting a signal processed by the first integrated circuit to the second integrated circuit.

[0113] (Item 11) The imaging device according to any one of items 1 to 10, wherein the image processing unit calculates luminance information of an image using at least one of the second image signal and the third image signal.

[0114] (Item 12) The imaging device according to any one of items 1 to 11, wherein the image processing unit detects a subject using at least one of the second image signal and the third image signal.

[0115] (Item 13) The imaging device according to any one of items 1 to 12, wherein the image processing unit detects photometric information using at least one of the second image signal and the third image signal.

[0116] (Item 14) The imaging device according to any one of items 1 to 13, wherein the image processing unit detects flicker using at least one of the second image signal and the third image signal.

[0117] (Item 15) The imaging device according to any one of items 1 to 14, wherein the image processing unit detects phase difference information using at least one of the second image signal and the third image signal.

[0118] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0119] The invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of reference numerals

[0120] 107: Image sensor, 108: First interface, 109: Second interface, 110: First integrated circuit, 114: Second integrated circuit, 207: Pixel array, 212: Pixel driving circuit, 302: Image processing circuit, 305: Switching circuit

Claims

1. An imaging device comprising: a pixel section in which a plurality of pixels are arranged in a matrix; a reading section that reads a first image signal from the pixel section and a second image signal having a lower resolution than the first image signal; an image processing section that generates a third image signal having a lower resolution than the first image signal from the first image signal; a first output section that outputs the first image signal; a second output section that outputs the second image signal or the third image signal; an imaging device comprising the above; a first integrated circuit that processes the first image signal output from the first output section of the imaging device; a second integrated circuit that processes the second image signal or the third image signal output from the second output section of the imaging device; An imaging apparatus characterized by comprising the above.

2. The imaging device according to claim 1, wherein the second image signal or the third image signal is used for live view display.

3. The imaging device according to claim 1, wherein the resolution of the third image signal is lower than the resolution of the first image signal and equal to or higher than the resolution of the second image signal.

4. The imaging device according to claim 3, wherein the resolution of the third image signal is equal to the resolution of the second image signal.

5. The imaging device according to claim 1, wherein the resolution of the third image signal is lower than the resolution of the second image signal.

6. The imaging device according to claim 1, further comprising switching means for switching between a first state in which the first image signal is output from the first output section and the third image signal is output from the second output section, and a second state in which the second image signal is output from the second output section.

7. The imaging device according to claim 6, further comprising a display section for displaying an image, wherein the display section displays an image based on the second image signal before a shooting instruction is given by a user, and displays an image based on the third image signal after the shooting instruction is given by the user.

8. The imaging device according to claim 1, wherein the communication speed for outputting a signal from the first output section is faster than the communication speed for outputting a signal from the second output section.

9. The imaging device according to claim 1, further comprising a storage section disposed between the image processing section and the second output section.

10. The imaging device according to claim 1, further comprising a transmission unit that transmits a signal processed by the first integrated circuit to the second integrated circuit.

11. The imaging device according to claim 1, wherein the image processing unit calculates luminance information of an image using at least one of the second image signal and the third image signal.

12. The imaging device according to claim 1, wherein the image processing unit detects a subject using at least one of the second image signal and the third image signal.

13. The imaging device according to claim 1, wherein the image processing unit detects information for photometry using at least one of the second image signal and the third image signal.

14. The imaging device according to claim 1, wherein the image processing unit detects flicker using at least one of the second image signal and the third image signal.

15. The imaging device according to claim 1, wherein the image processing unit detects phase difference information using at least one of the second image signal and the third image signal.

Images