IMAGE CAPTURE DEVICE

Abstract

An image pickup device comprises an image sensor having a pixel portion, a readout portion that reads out a first image signal and a second image signal from the pixel portion, the second image signal having a lower resolution than the first image signal, an image processor that generates a third image signal from the first image signal, the third image signal having a lower resolution than the first image signal, a first output portion that outputs the first image signal and a second output portion that outputs the second image signal or the third image signal, a first integrated circuit that processes the first image signal output; and a second integrated circuit that processes the second image signal or the third image signal.

Description

BACKGROUND OF THE INVENTIONField of the invention

[0001] The present invention relates to an image pickup device. Description of the related art

[0002] Generally, digital cameras and electronic devices equipped with a camera function often have a "live view (LV) function." The "live view" function is a function that displays captured images on a display unit in real time while shooting motion pictures or similar. A user can adjust the shooting range of a still image and a motion picture while checking the motion images displayed by the live view function.

[0003] In order for the user to perform the shooting intended by the user, it is crucial that the aforementioned adjustment of the shooting range be easy. For example, if a still image is captured while shooting moving images, moving images cannot be obtained in a frame in which the still image was captured. In this case, since images to be displayed as a live view cannot be obtained, ease of adjusting the shooting range is impaired. To make adjustment of the shooting range easy, it is crucial that even when a still image is captured, missing frames of a live view be prevented. For this reason, it is known that processing different from processing for live view images is performed for a still image.

[0004] For example, Japanese Patent No. 6757199 discloses an image sensor having a first mode in which image data read out from a pixel portion is transferred to a memory built into the image sensor, and a second transfer mode in which the image data is transferred outside the image sensor.

[0005] Japanese Patent Laid-Open No. 2023-106041 discloses a data processing method for a case where a plurality of data has been read out within an image sensor. Specifically, a method is described in which still image data from a pixel section is output after being reduced into a live view image by a conversion circuit within the image sensor, and in which, when live view image data has been read out from the pixel section, the live view image data is output without being reduced.

[0006] However, according to the technique disclosed in the above-mentioned Japanese Patent No. 6757199, image data having a large data amount such as a still image cannot be used in the live view display, causing the absence of a display frame of a live view.

[0007] Although Japanese Patent Laid-Open No. 2023-106041 discloses data processing within the image sensor, it does not provide a clear description of a processing method regarding signal processing circuits connected to the image sensor. Furthermore, it does not provide a clear description of a method for configuring appropriate output interfaces corresponding to a still image and a live view image, respectively. SUMMARY OF THE INVENTION

[0008] The present invention has been made in view of the problems described above and provides an image pickup device capable of preventing a lack of live view images in a case where a user performs shooting while looking at live view images.

[0009] According to one aspect of the present invention, there is provided an image pickup device comprising: an image sensor having a pixel portion in which a plurality of pixels are arranged in a matrix; a readout portion that reads out a first image signal and a second image signal from the pixel portion, the second image signal having a lower resolution than the first image signal; an image processor that generates a third image signal from the first image signal, the third image signal having a lower resolution than the first image signal; a first output portion that outputs the first image signal; and a second output portion that outputs the second image signal or the third image signal; a first integrated circuit that processes the first image signal output from the first output portion of the image sensor;and a second integrated circuit that processes the second image signal or the third image signal output from the second output section of the image sensor;

[0010] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing a configuration of an image pickup apparatus according to a first embodiment of the present invention. Fig. 2A and Fig. 2B are diagrams showing a configuration of an image sensor. Fig. 3A and Fig. 3B are diagrams showing a stack structure of the image sensor. Fig. 4A and Fig. 4B are diagrams showing data paths of still images and LV images. Fig. 5 is a timing chart showing the operations of the image sensor according to the first embodiment. Fig. 6 is a timing chart showing the operations of the image sensor according to a second embodiment. Fig. 7 is a timing chart showing the operations of the image sensor according to a third embodiment. DESCRIPTION OF THE EMBODIMENTS

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. In the embodiments, several features are described, but no limitation is imposed on an invention requiring all such features, and several such features may be combined as needed. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant description thereof will be omitted. (First embodiment)

[0012] Fig. 1 is a block diagram showing a configuration of an image pickup device according to a first embodiment of the present invention. As shown in Fig. 1, an image pickup device 150 includes an image pickup optical system 104. The image pickup optical system 104 includes a first lens 100, a diaphragm 101, a second lens 102, and a third lens 103. The first lens 100 is disposed at the front end of the image pickup optical system 104. The diaphragm 101 adjusts the amount of light at the time of photography by adjusting the diaphragm aperture diameter. The diaphragm aperture diameter of the diaphragm 101 is adjusted by driving a diaphragm actuator 126. Driving a focus actuator 124 described later causes the second lens 102 and the third lens 103 to move forward and backward in the optical axis direction, thereby adjusting a focal point of the image pickup optical system 104.

[0013] Behind the optical image pickup system 104, a focal plane shutter 105, an optical low-pass filter 106, and an image sensor 107 are arranged in this order. The focal plane shutter 105 functions to adjust an exposure time period at the time of still image capture. The optical low-pass filter 106 functions to reduce false colors and moiré in a captured image. The image sensor 107 converts an optical image of an object generated by the optical image pickup system 104 into electrical signals.

[0014] The image capture device 150 includes a first integrated circuit 110 and a second integrated circuit 114. The first integrated circuit 110 includes a first digital signal processor (DSP) 111.

[0015] The first DSP 111 is connected to the image sensor 107 via a first interface (IF) 108, which functions as a first output section, receives image data transmitted from the image sensor 107, and performs image processing. Examples of the image processing performed by the first DSP 111 include processes for correcting still image data and the like.

[0016] A first RAM 112 is connected to the first integrated circuit 110 and stores image data processed by the first DSP 111. Note that, although the first RAM 112 is disposed outside the first integrated circuit 110 in the present embodiment, it is permissible to employ a configuration in which part or all of its functions are provided in the first integrated circuit 110 or the first DSP 111.

[0017] At least a second digital signal processor (DSP) 115 and a CPU 117 are provided in the second integrated circuit 114. Furthermore, although a shutter drive unit 122, a focus drive unit 123, and a diaphragm drive unit 125 are provided in the second integrated circuit 114 in the present embodiment, it is permissible to employ a configuration in which these are provided in another integrated circuit.

[0018] The second DSP 115 is connected to the image sensor 107 via a second interface (IF) 109, which functions as a second output section, receives image data transmitted from the image sensor 107, and performs image processing.

[0019] Examples of the image processing performed by the second DSP 115 include processing for correcting LV image data, generation of display images to be displayed on a display unit 119, and the like.

[0020] A second RAM 116 is connected to the second integrated circuit 114, stores the image data processed by the second DSP 115, and continues to function as a working memory when the CPU 117 described later is operating.

[0021] Note that although the present embodiment adopts a configuration in which both functions are realized using RAMs, another type of memory may be used as long as it is a memory that has a sufficiently high access speed and is trouble-free in terms of operations. Although the second RAM 116 is arranged outside the second integrated circuit 114 in the present embodiment, it is permissible to adopt a configuration in which part or all of its functions are provided in the second integrated circuit 114, the second DSP 115, or the CPU 117.

[0022] Here, the first integrated circuit 110 and the second integrated circuit 114 are connected via a third interface (IF) 113. Still image data corrected by the first DSP 111 within the first integrated circuit 110 is transmitted to the second integrated circuit 114 via the third IF 113 and recorded on a recording medium 120 via the second DSP 115.

[0023] The CPU 117 integrally controls the operations of the image pickup device 150 and executes a program for controlling each component of the image pickup device. The CPU 117 controls the shooting timing of moving images and still images, described later, by configuring various types of settings on the image sensor 107. Furthermore, it also has the function of adjusting a focal point of the image pickup optical system 104 by driving and controlling the focus drive unit 123, described later, using a calculation result output from the second DSP 115.

[0024] The CPU 117 is connected to an operation unit 118, a display unit 119, a recording medium 120, a ROM 121, a shutter drive unit 122, a focus drive unit 123, and a diaphragm drive unit 125.

[0025] The operation unit 118 includes control elements such as buttons and levers. A user inputs an instruction to the CPU 117 and performs recording by operating a control element of the operation unit 118. The operation unit 118 includes a still image recording start button. After detecting the user's operation to press the still image recording start button, the CPU 117 performs control to capture a still image after a specified period of time has elapsed.

[0026] The display unit 119 displays images processed by the second DSP 115, menus, and the like. Not only a display but also an electronic viewfinder (EVF) or the like can be used as the display unit 119. The recording medium 120 is an attachable and removable recording medium on which still image and moving image data are recorded, and can be implemented, for example, by a memory card or the like. The ROM 121 stores, for example, a program intended for the CPU 117 to control the operations of each component.

[0027] The shutter drive unit 122 drives and controls the focal plane shutter 105. The focus drive unit 123 changes a focus position of the image pickup optical system 104. That is, the focus drive unit 123 performs focus adjustment by driving and controlling the focus actuator 124 based on the output of the CPU 117 and moving the focus lenses (the second lens 102 and the third lens 103) in the optical axis direction. The diaphragm drive unit 125 changes the aperture diameter of the diaphragm 101 by driving and controlling the diaphragm actuator 126, thereby adjusting the amount of light incident on the image sensor 107.

[0028] Fig. 2A is a circuit diagram of a unit pixel, and Fig. Fig. 2B is a diagram showing configurations of a pixel array and peripheral circuits. Specifically, circuits around the pixels in the image sensor 107 will be described with reference to Fig. 2A and Fig. 2B.

[0029] First, with reference to Fig. 2A describes a circuit configuration of a unit pixel 206. A photodiode (hereinafter, PD) 200 as a photoelectric unit is arranged below a microlens and generates and accumulates charges according to the amount of incident light. A transfer switch 201 is a switch controlled by a control signal φtx. When the value of the control signal φtx is set to high (hereinafter, H), the transfer switch 201 is turned on, causing the charges accumulated in the PD 200 to be transferred to a floating diffusion unit (hereinafter, FD) 202.

[0030] A reset switch 203 is a switch controlled by a control signal φres that resets the FD 202. By setting both control signals φtx and φres to H, both the PD 200 and the FD 202 are set to a power supply voltage (VDD); thus, a pixel reset operation is performed.

[0031] A transistor 204, which also functions as a pixel amplifier, is connected via a column output line 208 (see Fig. 2B) with both a selector switch 205 and a constant current source 209 (see Fig. 2B). When the value of a control signal φsel of the select switch 205 is set to H, the transistor 204 is connected to the constant current source 209, thus forming the pixel amplifier. The charges transferred from the PD 200 to the FD 202 are converted by the pixel amplifier into a voltage value corresponding to the amount of charge and output as a pixel signal to the column output line 208.

[0032] Next, an overall configuration of the image sensor 107 will be described with reference to Fig. 2B.

[0033] In a pixel array (pixel section) 207, a plurality of unit pixels 206 are arranged in a matrix. Specifically, (m+1) unit pixels 206 and (n+1) unit pixels 206 are arranged in the horizontal direction and vertical direction, respectively. Note that m and n are natural numbers. In this configuration, a plurality of photoelectric conversion elements are arranged in a matrix in the pixel array 207.

[0034] A drive pulse generation unit 210 generates pulses for performing a reset operation and a readout operation for the unit pixels 206. The pulses generated by the drive pulse generation unit 210 are supplied to a pixel drive circuit 212. A row selection circuit 211 selects a specific row to which the pulses generated by the drive pulse generation unit 210 are to be supplied and sets the selected specific row in the pixel drive circuit 212. The pixel drive circuit 212 supplies the pulses generated by the drive pulse generation unit 210 to the specific row selected and set by the row selection circuit 211.

[0035] The row selection circuit 211 enables reading out signals from the pixel array 207 in a variety of ways. For example, in a case where pixel signals to be used for a still image are read out because high-resolution image data is required, the row selection circuit 211 selects each row of the pixel array 207 and reads out pixel signals.

[0036] On the other hand, in a case where pixel signals to be used for LV (live view) moving images are read out, since high resolution is not required, the line selection circuit 211 performs an operation of selecting lines of the pixel array 207 at a pitch of two lines and reading out pixel signals. In this case, after the 0 ten Line the pixel signals of the 3 tenLine read out. Reading pixel signals in this way makes it possible to obtain LV image data whose vertical resolution has been reduced to 1 / 3 of that of the previously described still image.

[0037] Comparing the still image with the LV image, although the still image has excellent resolution, the characteristics of the LV image are such that the readout time is shortened along with a reduction in vertical resolution, and the electrical power required for readout can be reduced. In this way, the line selection / control mode of the line selection circuit 211 enables a variety of image types to be obtained, suitable for specific purposes. Note that no limitation is intended with a configuration that performs readout by thinning pixel lines; for example, it is permissible to adopt a configuration that obtains LV image data with a low vertical resolution by adding pixel signals from a plurality of pixel lines.

[0038] Pixel signals are output to column output lines 208 row by row according to the pulses supplied by pixel drive circuit 212. Constant current source 209, in combination with transistors 204, forms a source-follower circuit. An AD converter circuit 213 converts analog signals output to column output lines 208 into digital signals.

[0039] Note that although the present embodiment adopts a configuration in which pixel signals are read out from the pixel array 207 on a per-row basis, the present invention is not limited thereto. For example, if two column output lines are prepared for each column, pixel signals from two rows can be read out simultaneously for both the still image and the LV image by changing the connection between pixels and the column output lines between even-numbered and odd-numbered rows. By adopting such a configuration that enables simultaneous readout of pixel signals from a plurality of rows, the time period for readout can be shortened. However, the larger the number of column output lines, the more complex the circuits become. Therefore, it is desirable to determine an appropriate number of column output lines according to the method of use.

[0040] Fig. 3A and Fig. 3B are diagrams showing a configuration of an image sensor composed of a plurality of semiconductor substrates. A configuration of the image sensor 107 according to the present embodiment will be described with reference to FIG. Fig. 3A and Fig. 3B.

[0041] Generally, as a method for configuring an image sensor, a method is known in which a single image sensor is configured by providing a plurality of semiconductor substrates with separate circuits, stacking these plurality of semiconductor substrates, and connecting them together. Also, in the image sensor 107 according to the present embodiment, a single image sensor chip is configured by stacking a first semiconductor substrate 300 and a second semiconductor substrate 301.

[0042] Configurations of circuits mounted on the first semiconductor substrate 300 and the second semiconductor substrate 301, respectively, will be described with reference to Fig. 3A.

[0043] The pixel array 207 is provided on the first semiconductor substrate 300. Furthermore, an AD converter circuit 213, an image processing circuit 302, an image memory 303, a switching circuit 305, a first IF 108, and a second IF 109 are provided on the second semiconductor substrate 301.

[0044] The image processing circuit 302 is an information processing device for processing information read from the pixel array 207, thereby converting this information into another type of information. While the image processing circuit 302 can perform various types of processing, in the present embodiment, it is assumed that it converts a still image into an LV image by reducing the still image.

[0045] The image memory 303 is a memory that temporarily stores pixel data read from the pixel array 207 and converted into digital values ​​in the AD converter circuit 213, as well as image data processed by the image processing circuit 302. In the present embodiment, any memory can be used as this memory as long as it is configured to store data and enable stored data to be read out sufficiently faster than the speed of reading pixel signals from the pixel array 207 and the processing speed in the image processing circuit 302.

[0046] The first IF 108 is connected to the AD converter circuit 213 and outputs read-out image data to the outside without storing it in the image memory 303. The second IF 109 is connected to the image memory 303 and outputs image data stored in the image memory 303 to the outside.

[0047] The switching circuit 305 is controlled by the CPU 117 and has the function of switching the output destination of the image data output from the AD converter circuit 213 to one of the image processing circuits 302, the image memory 303, and the first IF 108. This switching will be explained later with reference to the Fig. 4A and Fig. 4B.

[0048] It should be noted that circuits other than those shown may also be arranged between the discrete blocks. For example, a configuration is also permissible in which a correction processing circuit is arranged between the AD conversion circuit 213 and the first IF 108 or between the AD conversion circuit 213 and the image processing circuit 302, which performs correction processing with respect to the images read out from the AD conversion circuit 213.

[0049] Fig. 3B is a schematic diagram showing an example in which the first semiconductor substrate 300 and the second semiconductor substrate 301 are stacked to configure the image sensor 107, which is a single chip. The figure shows a state in which the first semiconductor substrate 300 is stacked on the second semiconductor substrate 301. Any known technique can be used for electrically connecting the stacked semiconductor substrates to each other.

[0050] Note that although the present embodiment illustrates a configuration in which the pixel array 207 is mounted on the first semiconductor substrate 300 and all other circuit blocks are mounted on the second semiconductor substrate 301, the present invention is not limited thereto. For example, it is permissible to adopt a configuration in which all pixel arrays and circuit blocks are mounted on the same semiconductor substrate, or a configuration in which the circuit blocks are further mounted on a plurality of semiconductor substrates.

[0051] Fig. 4A and Fig. 4B are diagrams schematically showing the paths of image data elements of still images, reduced still images, and LV images. With reference to the Fig. 4A and Fig. 4B, the type of circuit blocks (data path) through which each of the image data items of still images, reduced still images, and LV images passes when output to the outside of the image sensor 107 is described below.

[0052] Fig. Figure 4A is a data path diagram illustrating a data path traversed by still images and reduced still images.

[0053] Pixel signals of a still image are read out from the pixel array 207 and converted into digital values ​​in the AD converter circuit 213. This image data then reaches a junction between a path through which the image data is output as is from the first IF 108 and a path through which the image data is input to the image processing circuit 302. These paths are set by the switching circuit 305 according to an instruction from the CPU 117. The image processing circuit 302 executes a process of reducing the image data input to the image processing circuit 302 from the still image into a reduced still image. The image data is then output from the second IF 109 via the image memory 303.

[0054] The reduced still image may have different resolutions. For example, by reducing the resolution in the image processing circuit 302 to the same resolution as the LV images, the second DSP 115 can perform the same processing for the reduced still image and the LV images at a later stage. Note that the resolution of the reduced still image may be equal to or higher than the resolution of the LV images as a result of the reduction processing, or it may be lower than the resolution of the LV images as a result of the reduction processing.

[0055] Furthermore, in the image processing circuit 302, it is also possible to only reduce the resolution in the horizontal direction to the resolution of the LV images and then transmit the reduced still image. In this case, although the second DSP 115 must perform reduction processing in the vertical direction, the circuit scale of the image processing circuit 302 can be reduced.

[0056] Fig. Figure 4B is a data path diagram illustrating a data path through which LV images traverse. Pixel signals of an LV image output from pixel array 207 are converted into digital values ​​by AD converter circuit 213 and accumulated in image memory 303. The pixel signals are then output from second IF 109. This path is set by switching circuit 305 according to an instruction from CPU 117.

[0057] As described above, if the data paths are configured as described in the Fig. 4A and Fig. As shown in Fig. 4B, high-resolution data read out as a still image is output from the first IF 108, and low-resolution data read out as a reduced still image or an LV image is output from the second IF 109. Then, the still image is output to the first DSP 111, and the reduced still image and the LV image are output to the second DSP 115. By executing the processing using different DSPs for different data types in the above manner, the data processing for still images and the data processing for LV displays can be executed completely in parallel.

[0058] When transmitting high-resolution data, transmission generally takes time. Furthermore, it is difficult to share still images with LV displays because they differ from LV images in resolution and the like. For this reason, in a case where still images and LV images are transmitted via the same IF and data processing is performed by the same DSP, there is a problem that it is difficult to always perform an update at constant timing without thinning an LV display. To solve this problem, the configuration of the present embodiment enables an update to always be performed at constant timing without thinning the LV display.

[0059] Furthermore, since the first IF 108 needs to transmit image data at a communication speed equal to or higher than the readout speed of still images, a high-speed interface must be prepared therefor. In contrast, the second IF 109, which transmits image data for LV display, does not require such a high-speed interface. It is sufficient for the second IF 109 to transmit image data at a speed required for displaying on the display unit 119. That is, since an interface having a lower speed than the first IF 108 can be used as the second IF 109, the cost of the image sensor can be reduced. In this case, a configuration is adopted in which a reduced still image and an LV image are stored in the image memory 303 to make the transmission speed of the second IF 109 slower than the readout speed.

[0060] Fig. 5 is a timing chart showing the operations of the image sensor according to the first embodiment. Referring to Fig. 5, a capture operation and a reduction operation for still images and a capture operation for LV images in the first embodiment will be described below.

[0061] In general, a cycle required to update the LV display is different from a capture cycle required when continuously capturing still images. The present embodiment will be described with reference to an example in which the duration of the capture cycle required when continuously capturing still images is twice as long as the cycle required to update the LV display. Among the diagonal lines in the figure, thin dashed lines indicate the sampling for resetting an LV image in each unit pixel 206 included in the pixel array 207, and thin solid lines indicate the sampling for reading out an LV image from the pixel array 207.In addition, thick dashed lines indicate a scan for resetting a still image in each unit pixel 206 included in the pixel array 207, and thick solid lines indicate a scan for reading out a still image from the pixel array 207.

[0062] When a vertical synchronization signal (hereinafter, VD) is input to the image sensor 107 at time t500, the image sensor 107 reads out an LV image. Along with the start of the readout, the writing of the LV image to the image memory 303 also starts. Furthermore, the LV image is read out from the image memory 303, and the output of data of the LV image from the second IF 109 to the second DSP 115 starts. The second DSP 115 starts the process of displaying the received image data on the display unit 119. When the reading out of the LV image is completed at time t501, the writing to the image memory 303 also stops.

[0063] When a still image capture command is issued at time t502, the CPU 117 controls the image sensor 107 to start accumulating a still image at the next VD time or later. At time t503, the readout of the LV image from the image memory 303 is completed, and the transfer of the image data by the second IF 109 is also completed. At this time or later, the second DSP 115 completes or terminates the display of the image.

[0064] When a VD is input to the image sensor 107 at time t504, the image sensor 107 again reads out an LV image. By repeatedly inputting a VD to the image sensor 107 in the above-mentioned manner, an LV image can be repeatedly obtained. A time interval from time t500 to time t504 is regarded as one cycle, and the input of a VD is repeated while maintaining this predetermined cycle; as a result, LV images can be obtained in a constant cycle. By displaying the LV images obtained in the predetermined cycle in the above-mentioned manner on the display unit 119, moving images in which a screen is updated at the predetermined cycle can be provided to a user as LV display.

[0065] At time t505, accumulation of a still image is started. When a VD is input to the image sensor 107 at time t506 following the lapse of one cycle, that is, the aforementioned predetermined cycle, since time t504, the image sensor 107 reads out a still image. With the start of readout, the still image is input to the image processing circuit 302, and simultaneously, output of data from the first IF 108 to the first DSP 111 is started. The first DSP 111 starts processing to record the received image data as the still image. After the image processing circuit 302 applies reduction processing to the image data input to the image processing circuit 302, writing of the image data to the image memory 303 is started, and simultaneously, output of the image data from the second IF 109 to the second DSP 115 is started.The second DSP 115 starts the process of displaying the received image data on the display unit 119.

[0066] When the readout of the still image is completed at time t507, the input of the image data to the image processing circuit 302 is also stopped, and the writing from the image processing circuit 302 to the image memory 303 is also stopped. Furthermore, at the same time, the output of the image data from the first IF 108 to the first DSP 111 is also stopped. At this time or later, the first DSP 111 completes the processing of the still image.

[0067] At time t508, the output of the image data written to the image memory 303 is completed, and the transfer of the image data from the second IF 109 is also completed. At this time or later, the second DSP 115 completes the image display. By performing control in the above manner, the cyclic update of the LV display can be provided without interruption even when a still image has been obtained.

[0068] In the present embodiment, it is assumed that the duration of the capture cycle required for continuous still image capture is twice the duration of the cycle required for updating the LV display. Therefore, even in a case where the still image capture instruction (continuous capture instruction) runs from time t506 to time t509, an LV image is read out for the next frame.

[0069] When a VD is input to the image sensor 107 at time t509, the image sensor 107 reads out an LV image. When a VD is input to the image sensor 107 at the subsequent time t510, the image sensor 107 reads out a still image. Starting at time t510, the operations of times t506 to t510 are repeated.

[0070] It should be noted that the timing diagram in Fig. 5 has been described with respect to a case where a still image and an LV image are alternately acquired, but the present invention is not limited thereto. For example, it is permissible to employ a configuration in which three LV images are acquired between the acquisition of still images. In this case, the cycle of obtaining an LV image may be the same as or different from that before the start of the acquisition instruction.

[0071] As described above, according to the first embodiment, in a configuration capable of acquiring still images and LV images, the still images can still be output as reduced still images. Furthermore, it is possible to receive a still image in the first DSP 111 and apply correction processing for still images thereto, and in parallel, to receive a reduced still image and an LV image in the second DSP 115 and apply correction processing for LV display images thereto. As a result, cyclic updating of the display images without thinning can be provided to the display unit 119, and the acquisition and correction processing for still images can be performed in parallel. Furthermore, in such a configuration, an increase in the circuit scale of the second IF 109 can also be suppressed. <modifikationsbeispiel>

[0072] Although the first embodiment assumes that the image processing circuit 302 reduces still images, the present invention is not limited thereto. The image processing circuit 302 may perform other processing. Design for an LV display may first be performed based on at least one of the LV image data and the reduced still image data, and then luminance information may be calculated for each color. Furthermore, subject detection information for a subject tracking function may be calculated based on at least one of the LV image data and the reduced still image data.

[0073] Furthermore, a photometric calculation for an automatic exposure adjustment function can be performed based on at least one of the LV image data and reduced still image data. Furthermore, flicker detection information for a flicker-free recording function can be calculated based on at least one of the LV image data and reduced still image data.

[0074] Phase difference information for an autofocus adjustment function may also be calculated based on at least one of LV image data and reduced still image data. Other various calculations are possible. Furthermore, in a case where these various calculations are performed, the image processing circuit 302 may also perform the calculations with respect to images obtained as LV images.

[0075] By calculating various types of information in the image processing circuit 302, the configuration of the second DSP 115 can be simplified and made more versatile. (Second embodiment)

[0076] A second embodiment will be described with respect to a case where the capture frame rate of still images during continuous shooting is higher than the frame rate of LV display. In this case, no LV images are obtained during a still image capture instruction, and LV display is performed entirely based on reduced still images generated from still images.

[0077] Fig. Fig. 6 is a timing chart showing the operations of the image sensor according to the second embodiment. Referring to Fig. 6, a recording operation and a reducing operation of still images in the second embodiment will be described below.

[0078] As described above, the cycle required for updating the LV display is generally different from the shooting cycle required for continuously shooting still images. The present embodiment will be described with reference to an example in which the duration of the cycle required for continuously shooting still images is 1 / 4 of the cycle required for updating the LV display. Among the diagonal lines in the figure, thin dashed lines indicate a scan for resetting an LV image in each unit pixel 206 included in the pixel array 207, and thin solid lines indicate a scan for reading out an LV image from the pixel array 207.In addition, thick dashed lines indicate a scan for resetting a still image in each unit pixel 206 included in the pixel array 207, and thick solid lines indicate a scan for reading out a still image from the pixel array 207. Also, the description of portions having similar operations to the timing chart in FIG. Fig. 5, omitted.

[0079] It should be noted that the timing diagram in Fig. 6 indicates a configuration in which the readout speed of a still image is faster than the readout speed of an LV image. This can be achieved by installing additional column output lines to improve the readout speed, which is Fig. 2A and Fig. 2B, is only possible in a freeze-frame mode. Note that the readout speed of a freeze-frame does not necessarily have to be faster than the readout speed of an LV image, and it is sufficient that the time period of reading a freeze-frame is shorter than the duration corresponding to 1 / 4 of the cycle required to update the LV display.

[0080] When a vertical synchronization signal (VD) is input to the image sensor 107 at time t600, the image sensor 107 reads out an LV image and simultaneously starts updating the displays of the display unit 119. When a freeze frame shooting instruction is issued at time t601, preparation for starting the freeze frame shooting is started. When a vertical synchronization signal (VD) is input to the image sensor 107 at time t602, the image sensor 107 reads out an LV image and simultaneously starts updating the display of the display unit 119. A time interval from time t600 to time t602 (one image retention cycle) has been set to correspond to the update cycle of the LV display presented on the display unit 119.

[0081] At time t603, the image sensor 107 starts accumulating a still image. When a vertical synchronization signal (VD) is input to the image sensor 107 at time t604, the image sensor 107 reads out a still image. With the start of the still image readout, the still image is input to the image processing circuit 302, and simultaneously, the output of image data from the first IF 108 to the first DSP 111 starts.

[0082] The first DSP 111 starts processing to record the received image data as the still image. After the image processing circuit 302 applies reduction processing to the image data input to the image processing circuit 302, the writing of the image data to the image memory 303 starts, and at the same time, the output of the data from the second IF 109 to the second DSP 115 starts. The second DSP 115 starts the process of displaying the received image data on the display unit 119.

[0083] When the readout of the still image is completed at time t605, the input of image data to the image processing circuit 302 is also stopped, and writing from the image processing circuit 302 to the image memory 303 is also stopped. At the same time, the output of image data from the first IF 108 to the first DSP 111 is also stopped. At this time or a later time, the first DSP 111 completes processing for the still image. Furthermore, at the same time t605, accumulation for the next still image is started.

[0084] Time t606 is the time at which a time period corresponding to 1 / 4 of the LV display update cycle has elapsed since time t604. When a vertical synchronization signal (VD) is input to the image sensor 107 at time t606, the image sensor 107 reads out a still image. Simultaneously with the start of the still image readout, the output of image data from the first IF 108 to the first DSP 111 starts. The first DSP 111 starts processing to record the received image data as the still image.

[0085] Meanwhile, the still image is not input to the image processing circuit 302 at this time. At this time, the reduced still image, the storage of which was completed at time t605, is continuously read out from the image memory 303. When the readout of the still image is completed at time t607, the output of image data from the first IF 108 to the first DSP 111 is also stopped. By controlling the image sensor 107 in the above manner, processing can be performed in which no still image is displayed even if the still image has been captured.

[0086] Time t608 is the time at which a time period corresponding to the update cycle of the LV display has elapsed since time t604. Therefore, a still image read out as a result of inputting a vertical synchronization signal (VD) to the image sensor 107 at time t608 is input to the image processing circuit 302, and a reduced still image is generated therefor and used for display. Further processing is similar to the processing for the still image read out at time t604.

[0087] Here, four images are read out between time t604 and time t608, which include the images read out from time t606 to time t608. The cycle of reading the still images between these two frames is equal to 1 / 4 of the refresh cycle of the LV display.

[0088] Note that although the freeze frame acquisition cycle in the present embodiment is 1 / 4 of the LV display refresh cycle, the present invention is not limited to this. To keep the LV display refresh cycle constant, it is also sufficient to configure a setting such that an integer multiple of the freeze frame acquisition cycle corresponds to one LV display refresh cycle.

[0089] For example, in a case where the LV display refresh cycle is 60 fps, the LV display refresh cycle can be maintained at 60 fps by setting the still image acquisition cycle to 120 fps or 180 fps. For example, if the still image acquisition cycle is set to 100 fps while the LV display refresh cycle is 60 fps, there may be a case where a still image is not read out at the time when an image for the LV display should be acquired. This prevents the LV display refresh cycle from being maintained constant before and after a still image capture instruction.

[0090] As described above, according to the second embodiment, the cycle of obtaining still images can be set to a cycle shorter than the cycle of the LV display while keeping the cycle of the LV display constant. (Third embodiment)

[0091] A third embodiment will be described with respect to a case where images that are neither used as still images nor for LV displays are obtained during the display of LV images. In a camera system, there are cases where images that are not presented to a user are obtained, for example, to use information in detecting flicker and the like. Such images are referred to as sub-sampling images, and the sampling of the image sensor to obtain the sub-sampling images is referred to as sub-sampling. A method of using pixels, a path of image data, and a timing chart of image acquisition with respect to sub-sampling images will be described below.

[0092] According to the description of Fig. 2B, the row selection circuit 211 performs an operation of selecting and reading out the pixel array 207 on a per-row basis at an interval of two lines at the time of obtaining LV images that do not need to have high resolution. In this case, unused pixels are present in the pixel array 207. In view of this, for example, during sub-scanning, an operation of selecting and reading out such unused pixels is performed on a per-row basis at an interval of eight lines. In this way, pixels different from pixels for the LV images are used for the sub-scan images; accordingly, the accumulation for the sub-scan can be performed independently of the accumulation of the LV images.

[0093] Furthermore, accumulation for sub-sampling can also be performed in parallel with the readout of LV images by re-preparing the column output lines for sub-sampling and connecting the pixels to the column output lines for sub-sampling at the time of sub-sampling. Note that in the present embodiment, it is assumed that such a configuration is not employed, and the timing of reading LV images and the timing of reading sub-sampled images are in an exclusive relationship.

[0094] A path diagram of data from subsampled images is exactly the same as the path diagram of LV images in Fig. 4B, and therefore, a description thereof will be omitted. The signals of a sub-scan image output from the pixel array 207 are converted into digital values ​​by the AD converter circuit 213 and accumulated in the image memory 303. Thereafter, the signals are output from the second IF 109.

[0095] Fig. Fig. 7 is a timing chart showing the operations of the image sensor according to the third embodiment. Referring to Fig. 7, a capturing operation of LV images and a capturing operation of sub-sampled images in the third embodiment will be described below. Among the diagonal lines in the figure, thin dashed lines indicate a scan for resetting an LV image in each unit pixel 206 included in the pixel array 207, and thin solid lines indicate a scan for reading out an LV image from the pixel array 207. Furthermore, thick dashed lines indicate a scan for resetting a sub-sampled image in each unit pixel 206 included in the pixel array 207, and thick solid lines indicate a scan for reading out a sub-sampled image from the pixel array 207.

[0096] When a vertical synchronization signal (VD) is input to the image sensor 107 at time t700, the image sensor 107 reads out an LV image. Along with the start of the readout, the writing of the LV image to the image memory 303 also starts. Furthermore, the LV image is read out from the image memory 303, and data is output from the second IF 109 to the second DSP 115. The second DSP 115 starts processing to display the received data on the display unit 119.

[0097] When a vertical synchronization signal (VD) is input to the image sensor 107 again at time t701, the image sensor 107 reads out an LV image again. This time interval from time t700 to time t701 is the update cycle of the LV display.

[0098] At time t702, accumulation of a sub-scan image starts. At time t703, the readout of the LV image is completed, and at the same time, the readout of the sub-scan image starts. Simultaneously with the start of the readout of the sub-scan image, the writing of the sub-scan image to the image memory 303 also starts. At this time, the LV image is read out from the image memory 303 and output from the second IF 109. Therefore, the sub-scan image is not read out from the image memory 303 at this time.

[0099] At time t704, accumulation of the next sub-scan image is started. At time t705, the reading of the sub-scan image is completed, and writing to the image memory 303 is also completed. Even at this time, the sub-scan image is not read from the image memory 303.

[0100] At time t706, the readout of the sub-scan image is restarted. In this way, the sub-scan image is read out and written to the frame memory 303 during a blanking interval in which an LV image is obtained. Thereafter, at time t707, the readout of the LV image from the frame memory 303 is completed. At the same time, the readout of the sub-scan image from the frame memory 303 is started. At time t708, the readout of the sub-scan image from the frame memory 303 is completed.

[0101] When a vertical synchronization signal (VD) is input to the image sensor 107 at time t709, the image sensor 107 reads out an LV image. No sub-scan image is read out in this frame.

[0102] As described above, according to the third embodiment, sub-scan images that are not provided to a user as images can be read out while LV images are obtained in a constant cycle. Note that, although a method in which sub-scan images are not processed by the image processing circuit 302 was introduced in the present embodiment, various types of data can be obtained by processing sub-scan images in the image processing circuit, similarly to the case of LV images according to the modification example of the first embodiment. Other embodiments

[0103] Embodiment(s) of the present invention may also be implemented by a computer of a system or apparatus that retrieves and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transitory computer-readable storage medium") to perform the functions of one or more of the embodiments described above, and / or that includes one or more circuits (e.g.,application-specific integrated circuit (ASIC)) for performing the functions of one or more of the embodiments described above, and by a method performed by the computer of the system or device, for example, by reading the computer-executable instructions from the storage medium and executing them to perform the functions of one or more of the embodiments described above and / or controlling the one or more circuits to perform the functions of one or more of the embodiments described above. The computer may have one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)) and may include a network of separate computers or separate processors to read and execute the computer-executable instructions. The computer-executable instructions may, for example, be provided to the computer.provided via a network or a storage medium. The storage medium may, for example, include one or more of the following devices: a hard disk, random access memory (RAM), read-only memory (ROM), distributed computing system memory, an optical disk (such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.

[0104] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims should be interpreted as broadly as possible to encompass all such modifications and equivalent structures and functions. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 6757199 [0004, 0006] JP 2023-106041 [0005, 0007]< / modifikationsbeispiel>

Claims

An image pickup device comprising: an image sensor having a pixel section in which a plurality of pixels are arranged in a matrix; a readout section that reads out a first image signal and a second image signal from the pixel section, the second image signal having a lower resolution than the first image signal; an image processor that generates a third image signal from the first image signal, the third image signal having a lower resolution than the first image signal; a first output section that outputs the first image signal, and a second output section that outputs the second image signal or the third image signal; a first integrated circuit that processes the first image signal output from the first output section of the image sensor; and a second integrated circuit that processes the second image signal or the third image signal output from the second output section of the image sensor. The image pickup device according to claim 1, wherein the second image signal or the third image signal is used in a live view display. The image pickup device according to claim 1, wherein a resolution of the third image signal is lower than a resolution of the first image signal and is equal to or higher than a resolution of the second image signal. An image pickup device according to claim 3, wherein the resolution of the third image signal is equal to the resolution of the second image signal. The image pickup device according to claim 1, wherein a resolution of the third image signal is lower than a resolution of the second image signal. An image pickup device according to claim 1, further comprising switching means for switching between a first state in which the first image signal is output from the first output section and the third image signal is output from the second output section, and a second state in which the second image signal is output from the second output section. An image pickup device according to claim 6, further comprising display means for displaying an image, wherein the display means displays an image based on the second image signal before a user issues a pickup instruction, and displays an image based on the third image signal after the user issues the pickup instruction. The image pickup device according to claim 1, wherein a communication speed at which a signal is output from the first output section is faster than a communication speed at which a signal is output from the second output section. An image pickup device according to claim 1, further comprising a memory device arranged between the image processor and the second output section. An image pickup device according to claim 1, further comprising transmission means for transmitting a signal processed by the first integrated circuit to the second integrated circuit. The image pickup device according to claim 1, wherein the image processor calculates luminance information of an image using at least one of the second image signal and the third image signal. The image pickup device according to claim 1, wherein the image processor detects an object using at least one of the second image signal and the third image signal. The image pickup device according to claim 1, wherein the image processor acquires photometric information using at least one of the second image signal and the third image signal. The image pickup device according to claim 1, wherein the image processor detects flicker using at least one of the second image signal and the third image signal. The image pickup device according to claim 1, wherein the image processor acquires phase difference information using at least one of the second image signal and the third image signal.

Images