Pixel read circuit and image pickup device

Abstract

To provide a pixel read circuit that reads out a pixel value by a global shutter method from an image pickup device that can switch between presence and absence of binning in a local area.SOLUTION: A pixel read circuit reads out a pixel value by a global shutter method from a pixel block which has n-pixels (n is a positive integer of 1 or larger) and floating diffusion that is shred by the n-pixels, from which a pixel value corresponding can be taken out in accordance with presence or absence of binning, comprises: a pixel signal reading switch provided for each pixel; and a GS signal transferring switch that reads out a pixel value at a frequency at which a global shutter signal that is controlled at a frequency corresponding to a frame rate is divided into 1 / n when not performing the binning, and reads out a pixel value at the same frequency as the frequency of the global shutter signal, when performing the binning.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a pixel readout circuit and an image sensor. [Background technology]

[0002] In conventional imaging systems, the resolution, frame rate, and dynamic range characteristics are generally set to be constant within the screen. Here, in imaging systems that capture wide-viewing-angle images that significantly exceed conventional display angles, such as head-mounted displays and LED (Light Emitting Diode) domes, it is expected that objects with various movements and brightnesses will be captured simultaneously within the same image. Examples of objects with various movements and brightnesses include objects with fine details, fast-moving objects, and objects with large differences in brightness, such as between sunlight and shade.

[0003] To capture images of all subjects captured in such wide-field-of-view images while suppressing aliasing, motion blur, blown-out highlights, blocked-up shadows, and degradation of the signal-to-noise ratio (SNR), an image sensor that simultaneously satisfies performance requirements such as high resolution, high frame rate, and high dynamic range is required. However, achieving high resolution and high frame rate requires high-speed readout of pixel signals within the image sensor, but the faster the pixel signals are readout, the more likely the dynamic range and SNR are to degrade. Furthermore, increasing the pixel signal readout speed also leads to problems such as increased readout circuit size and power consumption. Therefore, it is not easy to achieve the high performance described above with conventional image sensors, such as complementary metal oxide semiconductor (CMOS) image sensors.

[0004] On the other hand, from the perspective of improving subjective image quality, the drive mode of the image sensor does not necessarily need to be constant across the screen. In general image capture, the brightness and spatiotemporal frequency components of a subject are not necessarily uniform across the screen. For example, while it is desirable to capture areas where a stationary, finely detailed subject is present at high resolution, it is not necessary to capture at a high frame rate. Furthermore, even if an area where a fast-moving object is present is captured at high resolution, motion blur may occur depending on the frame rate, reducing the spatial frequency components. Furthermore, in areas where a highly bright subject is present, maintaining gradation in dark areas and considering the impact of noise on image quality are given lower priority, while in areas where a low-brightness subject is present, maintaining gradation in bright areas is given lower priority. Thus, conventional imaging methods often result in excess or deficiency in the imaging parameters of the image sensor depending on the characteristics of the subject, leaving room for improvement.

[0005] Furthermore, since the required imaging parameter levels differ depending on the characteristics of each of the multiple subjects included in the same image, these imaging parameters do not necessarily need to be constant within the screen. For example, areas where stationary subjects exist do not need to be imaged at a high frame rate, and it is desirable to image them at a high resolution. Also, areas where fast-moving objects exist do not need to be imaged at a high resolution, and it is desirable to image them at a high frame rate, because the spatial frequency decreases due to motion blur.

[0006] Furthermore, by partially shortening the exposure time in areas where high-brightness objects exist and partially extending the exposure time in areas where low-brightness objects exist, the dynamic range of the entire screen can be improved. In this way, by appropriately controlling the imaging parameters according to the characteristics of each of the multiple objects contained in the same image, it is possible to obtain image quality that is essentially equivalent to that achieved when high resolution, frame rate, and dynamic range characteristics are achieved, even without achieving these. An imaging element using a pixel structure described in Patent Document 1 is known as a technology for achieving such imaging.

[0007] Furthermore, as a technique for controlling the pixel structure described in Patent Document 1, a technique such as that described in Patent Document 2 is known. According to the technique described in Patent Document 2, it is possible to switch between the on and off states of a read operation from a photodiode. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2022-123539 [Patent Document 2] Japanese Patent Application Publication No. 2023-33911 Summary of the Invention [Problem to be solved by the invention]

[0009] However, the technology described in Patent Document 2 cannot be used with the global shutter method. Furthermore, reading pixel values ​​from the pixel structure described in Patent Document 1 using the global shutter method is not generally performed.

[0010] In view of the above problems, the present invention provides a pixel readout circuit and an image sensor that are capable of reading pixel values ​​using a global shutter method from an image sensor that can switch between binning and non-binning in local regions. [Means for solving the problem]

[0011] [1] In order to solve the above problem, one aspect of the present invention is a pixel readout circuit that reads pixel values ​​by a global shutter method from a pixel block that has n pixels (n is a natural number greater than or equal to 1), and a floating diffusion shared by the n pixels, and that is capable of extracting pixel values ​​depending on whether or not binning is performed. The pixel readout circuit includes a pixel signal readout switch provided for each pixel, and a GS signal transfer switch that, when binning is not performed, reads pixel values ​​at a frequency that is obtained by dividing a global shutter signal, which is controlled at a frequency depending on the frame rate, by n, and, when binning is performed, reads pixel values ​​at the same frequency as the global shutter signal.

[0012] [2] Furthermore, in one aspect of the present invention, the pixel readout circuit described in [1] above further includes a reset signal transfer switch that controls transfer of a reset signal for each pixel block, the reset signal being controlled according to a selectable exposure time, and the reset signal transfer switch transferring the reset signal to the pixel block according to the exposure time of the corresponding pixel block.

[0013] [3] Furthermore, in one aspect of the present invention, in the pixel readout circuit described in [1] or [2] above, pixel values ​​are read out from a back-side laminated circuit having a pixel array layer having a plurality of the pixel blocks, and a global control circuit layer that performs control to read out from the pixel blocks using a global shutter.

[0014] [4] Furthermore, according to one aspect of the present invention, in the pixel readout circuit described in [3] above, a mode control circuit is arranged in the pixel array layer and outputs a control signal to the pixel signal readout switch according to a readout mode.

[0015] [5] Furthermore, according to one aspect of the present invention, the pixel readout circuit described in [3] above further comprises a vertical scanning circuit disposed in the pixel array layer, which outputs, depending on the readout mode, a signal for reading out the charge read out by the GS signal transfer switch, a reset signal for resetting the charge accumulated in the floating diffusion, and a pixel selection signal for selecting the pixel to be read out.

[0016] [6] Another aspect of the present invention is an imaging device comprising: a pixel readout circuit according to any one of [1] to [5] above; a pixel array layer having a plurality of the pixel blocks; and a global control circuit layer that controls reading from the pixel blocks using a global shutter. [Effects of the Invention]

[0017] According to the present invention, it is possible to provide a pixel readout circuit and an image sensor that can read out pixel values ​​by a global shutter method from an image sensor that can switch between the presence and absence of binning in a local area. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a functional configuration diagram illustrating an example of a functional configuration of an imaging system according to an embodiment. [Figure 2] FIG. 2 is a conceptual diagram illustrating an example of a circuit configuration of the image sensor according to the present embodiment. [Figure 3] FIG. 2 is a schematic diagram showing an example of a pixel structure in a pixel block according to the present embodiment. [Figure 4] FIG. 2 is a circuit diagram showing an example of a pixel readout circuit according to the embodiment. [Figure 5] FIG. 2 is a schematic diagram showing an example of an arrangement of pixel blocks according to the embodiment. [Figure 6] 4 is a first timing chart showing an example of pixel reading performed by the pixel reading circuit according to the present embodiment. [Figure 7] 10 is a second timing chart showing an example of pixel readout performed by the pixel readout circuit according to the embodiment. [Figure 8] 10 is a third timing chart showing an example of pixel reading performed by the pixel reading circuit according to the embodiment. [Figure 9] 10 is a fourth timing chart showing an example of pixel reading performed by the pixel reading circuit according to the embodiment. [Figure 10] 10 is a fifth timing chart showing an example of pixel reading performed by the pixel reading circuit according to the embodiment. [Figure 11] FIG. 2 is a block diagram showing an example of the internal configuration of the imaging device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] [Embodiment] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the present invention is not limited to these embodiments and includes various modifications and improvements. In other words, the components described below include those that would be easily conceivable to a person skilled in the art or that are substantially identical, and the components described below can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the present invention. Furthermore, in the drawings, the scale and number of components may differ from the scale and number of the actual structures to make each configuration easier to understand.

[0020] 1 is a functional configuration diagram showing an example of the functional configuration of an imaging system according to an embodiment. First, with reference to the diagram, an imaging system 1 according to this embodiment will be described. The imaging system 1 includes an imaging device 3 and a video display device 5.

[0021] The imaging device 3 captures an image. A single image captured by the imaging device 3 may contain multiple subjects with different characteristics. An example of multiple subjects with different characteristics is a stationary object and a fast-moving object. That is, a single image captured by the imaging device 3 may contain a mixture of areas where stationary objects exist and areas where fast-moving objects exist. The imaging device 3 sets suitable imaging conditions for each of these multiple areas, thereby optimally capturing images of multiple subjects with different characteristics.

[0022] Specifically, the imaging device 3 includes a lens 31, a prism 32, a sub-sensor 33, an image sensor 34, a sensor drive board 35, a sensor power supply board 36, a scene information / image acquisition board 37, a signal processing board 38, and a power supply module 39. In the following description, the imaging device 3 captures a color image. In this case, the image sensor 34, the sensor drive board 35, the sensor power supply board 36, and the scene information / image acquisition board 37 are provided for each of the colors R (Red), G (Green), and B (Blue). This embodiment is not limited to this example, and can also be applied to monochrome images and grayscale images.

[0023] The lens 31 guides incident light to the prism 32. The figure shows the optical axis OA of the light incident on the lens 31. The lens 31 is preferably a wide-angle lens or an ultra-wide-angle lens in order to simultaneously capture images of multiple subjects with different characteristics. In this embodiment, lenses 31 with various angles of view can be used, and the angle of view is not limited in any way. A turret-type structure may also be provided to allow selection of multiple lenses with different angles of view.

[0024] The prism 32 splits the incident light according to the wavelength of the light. For example, the prism 32 may split the incident light into infrared light and send it to the sub-sensor 33, and visible light and send it to the image sensor 34. The prism 32 may also split the light into multiple image sensors 34 according to the wavelengths of each of the R, G, and B components. In the example shown in the figure, the prism 32 guides light having a wavelength corresponding to the G component to the image sensor 34G, guides light having a wavelength corresponding to the B component to the image sensor 34B, and guides light having a wavelength corresponding to the R component to the image sensor 34R.

[0025] The sub-sensor 33 is a sensor provided auxiliary to the image sensor 34 that generates an image. The sub-sensor 33 may be, for example, an event detection sensor that detects changes in luminance of each pixel arranged on a two-dimensional coordinate system. The sub-sensor 33 may be, for example, a distance sensor (Time-of-Flight Sensor: ToF sensor) that detects distance according to changes in the amount of infrared light emitted from the prism 32.

[0026] In the illustrated example, a sensor that detects infrared light is used as the sub-sensor 33, but this embodiment is not limited to this example. For example, the sub-sensor 33 may be configured to detect visible light. By separating (assigning) infrared light that is not necessary for detecting an image to the sub-sensor 33, it is possible to obtain the effect of reducing image degradation, so it is preferable that the sub-sensor 33 be a sensor that detects changes in the luminance of infrared light.

[0027] The image sensor 34 is an image sensor that detects the brightness of visible light emitted from the prism 32. In the example shown in the figure, an image sensor 34 for each of the R, B, and C colors is provided. Specifically, the image sensor 34G is shown as a sensor that detects green (G), the image sensor 34B is shown as a sensor that detects blue (B), and the image sensor 34R is shown as a sensor that detects red (R).

[0028] The image sensor 34 has multiple pixel blocks. Each pixel block includes at least multiple pixels and one floating diffusion. A pixel block may include, for example, 2×2=4 pixels or 4×4=16 pixels. According to this embodiment, it is possible to determine whether or not to perform binning for each pixel block. When binning is performed, i.e., when pixel values ​​of multiple pixels are simultaneously extracted, the resolution in the three-dimensional direction can be improved (i.e., a higher frame rate can be achieved) at the expense of the resolution in the two-dimensional direction. When binning is not performed, i.e., when pixel values ​​of multiple pixels are individually extracted, the resolution in the two-dimensional direction can be improved at the expense of the resolution in the three-dimensional direction (i.e., the frame rate). The image sensor 34 does not perform binning in areas where stationary objects exist, but performs binning in areas where fast-moving objects exist, thereby enabling imaging with a mixture of areas where binning is performed and areas where binning is not performed.

[0029] The sensor drive boards 35 are provided corresponding to the image sensors 34. In the example shown, a sensor drive board 35G is provided corresponding to the image sensor 34G, a sensor drive board 35B is provided corresponding to the image sensor 34B, and a sensor drive board 35R is provided corresponding to the image sensor 34R. The sensor drive boards 35 control the image sensors 34 in accordance with the drive mode and read out pixel values ​​from the image sensors 34.

[0030] Sensor power supply board 36 supplies power to sensor drive board 35. Sensor power supply board 36 may be provided corresponding to sensor drive board 35. In the example shown in the figure, sensor power supply board 36G is provided corresponding to sensor drive board 35G, sensor power supply board 36B is provided corresponding to sensor drive board 35B, and sensor power supply board 36R is provided corresponding to sensor drive board 35R.

[0031] The scene information and image acquisition boards 37 are provided corresponding to the sensor drive boards 35. In the example shown in the figure, a scene information and image acquisition board 37G is provided corresponding to the sensor drive board 35G, a scene information and image acquisition board 37B is provided corresponding to the sensor drive board 35B, and a scene information and image acquisition board 37R is provided corresponding to the sensor drive board 35R.

[0032] The scene information / image acquisition board 37 generates scene information based on pixel values ​​read by the sensor drive board 35. The scene information is generated from a brightness map and a motion map. The brightness map may be generated based on an image captured by the image sensor 34. For example, the brightness map may indicate the degree of brightness for each pixel block. Examples of the degree of brightness include bright determination (Bright), dark determination (Dark), and neutral determination (Normal). The motion map may also be generated based on an image captured by the sub-sensor 33. For example, the brightness map may indicate the presence or degree of motion for each pixel corresponding to a pixel block of the image sensor 34. Examples of the presence or degree of motion include moving determination (Fast) and still determination (Slow). The scene information may indicate the degree of brightness or the degree of motion for each pixel block.

[0033] For example, as an example of scene information, pixel blocks determined to be moving may be determined to be moving, and pixel blocks determined to be still may be determined to be bright, dark, or neutral based on a brightness map.

[0034] Note that one piece of scene information may be generated in common for all RGB colors, or different scene information may be generated for each RGB color. When one piece of scene information is generated in common for all RGB colors, the result of G, which has the greatest influence on luminance among the RGB colors, may be prioritized, or one piece of scene information common to all RGB colors may be generated using other logic. The luminance map and scene information may be shared among the RGB colors.

[0035] The signal processing board 38 generates video data based on the pixel values ​​read out by the sensor driving board 35 and the scene information generated by the scene information / video acquisition board 37. Note that the image information captured by the image sensor 34 according to this embodiment includes areas that have been binned and areas that have not been binned, so in order to display the video information on the video display device 5, it is preferable to perform spatiotemporal interpolation processing to generate the video information. However, the video data generated by the signal processing board 38 may be information in a state before the spatiotemporal interpolation processing is performed.

[0036] The power supply module 39 supplies power to each component included in the imaging device 3 .

[0037] The video display device 5 displays the video captured by the imaging device 3. The video display device 5 includes a spatio-temporal interpolation processing unit 51, a signal processing unit 52, and a display unit 53.

[0038] The spatiotemporal interpolation processing unit 51 acquires video data and scene information from the imaging device 3. The spatiotemporal interpolation processing unit 51 performs interpolation processing in time and space based on the acquired video data and scene information. Specifically, the spatiotemporal interpolation processing unit 51 interpolates missing pixels in two dimensions or three dimensions based on the video data and scene information. An example of an interpolation method is linear interpolation.

[0039] The signal processing unit 52 generates a video signal based on the information interpolated by the spatio-temporal interpolation processing unit 51 .

[0040] The display unit 53 performs display based on the video signal generated by the signal processing unit 52. The display unit 53 may be, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

[0041] The video display device 5 does not need to be a device that includes all of the components of the spatiotemporal interpolation processing unit 51, the signal processing unit 52, and the display unit 53, but may be a device in which the spatiotemporal interpolation processing unit 51 and the signal processing unit 52 are added to an existing display unit 53.

[0042] FIG. 2 is a conceptual diagram showing an example of the circuit configuration of an image sensor according to this embodiment. An image of the circuit configuration of the image sensor 34 will be described with reference to the same figure. As shown in the figure, the image sensor 34 has a back-side stacked circuit configuration. Specifically, the image sensor 34 has a pixel array layer and a global control circuit layer. The pixel array layer has a pixel array configured to include a plurality of pixel blocks. The global control circuit layer performs control for reading out data from the pixel blocks using a global shutter. According to the illustrated configuration, it can also be said that the pixel array layer is the upper layer and the global control circuit layer is the lower layer.

[0043] The pixel array layer is equipped with a mode control circuit and a vertical scanning circuit. The mode control circuit controls readout from pixel blocks by outputting pixel signal readout signals TC1 to TC4 to the pixel array according to a readout mode corresponding to the scene information generated by the scene information / image acquisition board 37. The vertical scanning circuit selects pixels to be readout and controls the resetting of pixel blocks by outputting a pixel selection signal SL, a reset signal RT, and a transfer signal TX to the pixel array according to the readout mode. The pixel selection signal SL is a signal that selects pixels to be readout. The reset signal RT is a signal that resets the charge accumulated in the floating diffusion FD. The transfer signal TX is a signal that reads out the charge read out by the global shutter. The transfer signal TX is a signal that transfers pixel signals to perform readout, and therefore can also be referred to as a pixel signal readout signal.

[0044] The global control circuit layer includes a global control circuit that outputs, to the pixel array layer, a GS transfer signal GCX that controls whether or not to transmit a global shutter signal to a pixel block, and a reset signal transfer signal RCP that transfers a reset signal to a pixel block according to the exposure time of the pixel block, thereby performing readout control using the global shutter method.

[0045] In the following description, a configuration including a mode control circuit, a vertical scanning circuit, and a global control circuit may be referred to as a pixel readout circuit. The pixel readout circuit reads pixel values ​​from pixel blocks using a global shutter system. The pixel blocks from which the pixel readout circuit reads pixel values ​​are assumed to have a structure that enables extraction of pixel values ​​depending on whether binning is performed or not.

[0046] FIG. 3 is a schematic diagram showing an example of a pixel structure in a pixel block according to this embodiment. The figure shows a schematic diagram of the pixel structure of a pixel block included in the pixel block. As shown in the figure, for example, the pixel array includes four pixels, photodiodes PD1 to PD4. These four pixels share one floating diffusion FD. While the figure shows an example in which the pixel array includes four pixels, this embodiment is not limited to this example. The pixel block may include n pixels (n is a natural number greater than or equal to 1) and at least one floating diffusion shared by the n pixels. The shape, size, and arrangement of the configuration shown in the figure are schematic and do not limit the physical shape, etc., of the pixel block according to this embodiment.

[0047] 4 is a circuit diagram showing an example of a pixel readout circuit according to this embodiment, and an example of the circuit configuration of the pixel readout circuit will be described with reference to the same figure.

[0048] First, the circuit configuration shared by each pixel in the pixel block will be described. The pixel readout circuit includes transistors Tr11, Tr12, Tr20, Tr41, a floating diffusion FD, Tr51, and Tr52 as circuits shared by each pixel in the pixel block.

[0049] The transistor Tr11 controls whether or not to transfer the reset signal RTP to the pixel block. A reset signal transfer signal RCP is input to the gate of the transistor Tr11 from the global control circuit.

[0050] The transistor Tr12 resets the pixel values ​​by supplying a power supply voltage to the photodiodes PD1 to PD4 under the AND condition of the reset signal RTP and the reset signal transfer signal RCP.

[0051] The transistor Tr20 controls whether or not a global shutter signal GTX is supplied to a pixel block. A GS transfer signal GCX is input to the gate of the transistor Tr20. The global shutter signal GTX is a signal common to all pixel blocks, and the GS transfer signal GCX is a signal controlled for each pixel block. The global shutter signal GTX is supplied to a pixel block for which the GS transfer signal GCX is on.

[0052] The transistor Tr41 resets the charge of the floating diffusion FD. While the transistors Tr11 and Tr12 reset the pixels, the transistor Tr41 can also be said to be a switch that resets the floating diffusion FD shared by each pixel in the pixel block.

[0053] The floating diffusion FD is a floating diffusion layer that temporarily stores electric charge read from the photodiode by the global shutter signal in order to convert the amount of electric charge into a voltage value. The floating diffusion FD is shared by multiple pixels in the pixel block.

[0054] The gate of transistor Tr51 is connected to the floating diffusion FD, the source is connected to the power supply voltage, and the drain is connected to transistor Tr52. Transistor Tr52 is a selection switch for the pixel block. While the pixel selection signal SL is on, the signal voltage is read out through source follower SF.

[0055] Next, the pixel-specific circuit configuration in a pixel block will be described. The n pixels included in a pixel block have a common configuration. Below, a generalized description will be given using an example of a configuration including n in the reference symbol (n is a natural number equal to or greater than 1, and in the illustrated example, is 1 to 4). The pixel readout circuit includes a photodiode PDn, a transistor Tr2n, a capacitor CMn, a transistor Tr3n1, and a transistor Tr3n2 as a pixel-specific circuit in the pixel block.

[0056] The photodiode PDn converts the intensity of incident light into electrical energy. In other words, the photodiode PDn stores an electric charge corresponding to the intensity of the incident light. The photodiode PDn may also be referred to as a pixel.

[0057] The transistor Tr2n turns on when both the global shutter signal GTX and the GS transfer signal GCX are on, and stores the charge converted into electrical energy by the photodiode PDn in the capacitor CMn.

[0058] The capacitor CMn stores the charge for each photodiode PDn. The charge stored in the capacitor CMn is read out by a method that depends on whether binning is performed (at a timing that depends on whether binning is performed).

[0059] A pixel signal readout signal TCn is input to the gate of transistor Tr3n1, which selects whether or not to transmit a transfer signal TX to transistor Tr3n2. While pixel signal readout signal TCn is on, the transfer signal TX is valid. Note that transistor Tr3n1 may also be referred to as a pixel signal readout switch.

[0060] The conduction state of the transistor Tr3n2 is controlled by the transistor Tr3n1, and the amount of charge in the capacitor CMn is transferred to the floating diffusion FD. Note that the control signal for the transistor Tr3n1 may be output by the above-mentioned mode control circuit in accordance with the read mode.

[0061] Next, an example of readout timing in each imaging mode will be described with reference to FIGS.

[0062] FIG. 5 is a schematic diagram showing an example of the arrangement of pixel blocks according to this embodiment. The diagram schematically illustrates the arrangement of pixel blocks. The diagram shows an example of four pixel blocks adjacent in the vertical direction. Specifically, the diagram shows pixel block m, pixel block m+1, pixel block m+2, and pixel block m+3. A method for reading pixel blocks arranged in this manner will now be described.

[0063] 6 is a first timing chart showing an example of pixel readout performed by the pixel readout circuit according to this embodiment. The horizontal axis indicates time, and the vertical axis indicates the voltage value of each signal. CM1 to CM4 indicate the amount of charge stored in the capacitors in terms of voltage. This figure shows an example of a readout method for four vertical scanning periods (4V periods). The readout method differs depending on the operation mode.

[0064] The figure shows, from left to right, examples of operation modes: Normal mode (no binning), Fast mode (with binning), Bright mode (no binning), and Normal mode (no binning). Normal mode is an operation mode in which signals from all pixels are read out at a standard frame rate. Fast mode is an operation mode in which 1 / 4 of the binned pixels are read out at a frame rate four times the standard rate. Bright mode is an operation mode in which the exposure time of Normal mode is changed.

[0065] The reset signal RTP is output as a pulse at timings where the exposure time is 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 within one vertical scanning period, for example. By outputting the reset signal RTP during the period when the reset signal transfer signal RCP is on, the charge is reset at that timing, and the exposure time is controlled. In other words, it can be said that the reset signal RTP is controlled according to a selectable exposure time. The exposure time may be determined according to the operation mode.

[0066] The reset signal transfer signal RCP is controlled to transfer the reset signal RTP to the pixel block in accordance with the exposure time of the corresponding pixel block. In other words, the reset signal transfer signal RCP can be said to be an exposure time selector.

[0067] The global shutter signal GTX is output n times (n is the number of pixels included in the pixel block; in the illustrated example, it is output four times) at regular intervals during one vertical scanning period. The regular intervals are at a frequency that corresponds to the frame rate.

[0068] When binning is not performed, the GS transfer signal GCX is output at a frequency obtained by dividing the global shutter signal by n (n is the number of pixels included in the pixel block. In the illustrated example, the frequency is divided by a quarter), and pixel values ​​are read out. When binning is performed, the GS transfer signal GCX is output at the same frequency as the global shutter signal, and pixel values ​​are read out.

[0069] PD1-4 show an example of the amount of charge accumulated in the photodiodes PD1 to PD4. The illustrated example shows a case where uniform light is incident on the photodiodes. Therefore, the amount of charge accumulated in PD1 to PD4 increases in proportion to time. When the global shutter signal is turned on, the charge is transferred to the capacitor CMn, and the amount of charge in the photodiode is reset.

[0070] Capacitors CM1 to CM4 show an example of the amount of charge accumulated in each capacitor CMn. When the global shutter signal is turned on, the amount of charge accumulated in the photodiodes PD1 to PD4 is transferred to the capacitors CM1 to CM4.

[0071] The pixel block reset signal RT is output as a pulse multiple times within one vertical scanning period. For example, the pixel block reset signal RT is output as pulses equal to the number of pixel blocks arranged in the horizontal direction within one vertical scanning period. When the pixel block reset signal RT is output, the charge accumulated in the floating diffusion FD is reset, and the reset voltage is read out through the source follower SF.

[0072] The transfer signal TX is output as a pulse when a pixel value is read out from the capacitor CMn. The transfer signal TX is output as a pulse multiple times within one vertical scanning period. For example, the transfer signal TX is output as pulses equal to the number of pixel blocks arranged in the horizontal direction within one vertical scanning period. Note that the pixel block reset signal RT and the transfer signal TX perform the same operation regardless of the operating mode.

[0073] The pixel signal readout signals TC1 to TC4 are pulsed to select pixels to be read out within a pixel block, and may or may not be output depending on the operating mode.

[0074] Next, the flow in each operation mode will be described.

[0075] First, the flow of the normal mode will be explained. In the normal mode, binning is not performed. First, when the global shutter signal GTX and the GS transfer signal GCX are simultaneously turned on, the charges accumulated in the photodiodes PD1 to PD4 are transferred from the capacitors CM1 to CM4. Note that in the normal mode, the GS transfer signal GCX is turned on once every four times the global shutter signal GTX is turned on. The GS transfer signal GCX is turned on when the global shutter signal GTX is turned on for the first time in one vertical scanning period.

[0076] When the GS transfer signal GCX is turned on, the charge accumulated in the photodiodes PD1 to PD4 is transferred. After the voltage is reset, the signal charge in the photodiodes PD1 to PD4 increases with exposure time. During one vertical scanning period, charge is transferred from the capacitors CMn of all pixels. In the illustrated example, charge is transferred in the order of capacitors CM1, CM2, CM3, and CM4. When capacitor CM1 is empty, charge transfer from capacitor CM2 begins. When capacitor CM2 is empty, charge transfer from capacitor CM3 begins. When capacitor CM3 is empty, charge transfer from capacitor CM4 begins. In other words, the charge is transferred sequentially. When the transfer signal TX and pixel signal readout signals TC1 to TC4 are turned on, the signal voltages of the capacitors CM1 to CM4 are read out to the floating diffusion FD. The pixel signal readout signals TC1 to TC4 operate sequentially at specific intervals during one vertical scanning period. For example, pixel signal readout signal TC1 may operate for the first 1 / 4 of one vertical scanning period, pixel signal readout signal TC2 may operate for the next 1 / 4 of the period, pixel signal readout signal TC3 may operate for the next 1 / 4 of the period, and pixel signal readout signal TC4 may operate for the next 1 / 4 of the period.

[0077] Next, the flow of the Fast mode will be explained. In the Fast mode, binning is performed. First, when the global shutter signal GTX and the GS transfer signal GCX are simultaneously turned on, the charges accumulated in the photodiodes PD1 to PD4 are transferred to the capacitors CM1 to CM4. Note that in the Fast mode, the GS transfer signal GCX is turned on once every time the global shutter signal GTX is turned on once.

[0078] When the GS transfer signal GCX is turned on, the charges accumulated in the photodiodes PD1 to PD4 are transferred, and after the voltage is reset, the signal charges in the photodiodes PD1 to PD4 increase with exposure time. Fast mode differs from normal mode in that the capacitors CM1 to CM4 simultaneously transfer charges. When the transfer signal TX and the pixel signal readout signals TC1 to TC4 are turned on simultaneously, the charges accumulated in the capacitors CM1 to CM4 are simultaneously read out to the floating diffusion FD. Fast mode differs from normal mode in that the pixel signal readout signals TC1 to TC4 are turned on simultaneously.

[0079] Next, the flow of Bright mode will be explained. Binning is not performed in Bright mode. First, when the global shutter signal GTX and the GS transfer signal GCX are simultaneously turned on, the charges accumulated in the photodiodes PD1 to PD4 are transferred from the capacitors CM1 to CM4. Note that in Bright mode, as in Normal mode, the GS transfer signal GCX is turned on once every four times the global shutter signal GTX is turned on. The GS transfer signal GCX is turned on when the global shutter signal GTX is turned on for the first time in one vertical scanning period.

[0080] When the GS transfer signal GCX is turned on, the charge accumulated in the photodiodes PD1 to PD4 is transferred, and after the voltage is reset, the signal charge of the photodiodes PD1 to PD4 increases with exposure time. During one vertical scanning period, charge is transferred from the capacitors CMn of all pixels. In the illustrated example, charge is transferred to the capacitors CM1, CM2, CM3, and CM4 in that order. The reset signal RTP is pulsed at 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of one vertical scanning period. To set the exposure time according to the scene information, a pulse signal is output to the reset signal transfer signal RCP at any of the timings of the reset signal RTP pulse output. The pulse signal output from the reset signal transfer signal RCP resets the charge and controls the exposure time. The signal voltage of the capacitors CM1 to CM4 is read out to the floating diffusion FD when the transfer signal TX and pixel signal readout signals TC1 to TC4 are turned on. The pixel signal readout signals TC1 to TC4 operate sequentially at specific time intervals during one vertical scanning period. For example, the pixel signal readout signal TC1 may operate during the first 1 / 4 of one vertical scanning period, the pixel signal readout signal TC2 may operate during the next 1 / 4 of the period, the pixel signal readout signal TC3 may operate during the next 1 / 4 of the period, and the pixel signal readout signal TC4 may operate during the next 1 / 4 of the period.

[0081] Next, an example of a readout method during a horizontal scanning period will be described with reference to Figures 7 to 10. In each figure, the horizontal axis represents time, and the vertical axis represents the voltage value of each signal. PixelOutput represents the pixel output value output from transistor Tr52 in voltage. The figures show an example of a readout method during four horizontal scanning periods (4H periods). The readout method differs depending on the operating mode.

[0082] 7 is a second timing chart showing an example of pixel readout performed by the pixel readout circuit according to this embodiment. This diagram shows an example of a readout method in Normal mode and Bright mode. That is, this diagram shows an example of a readout method when binning is not performed. This diagram also shows the case where pixel signal readout signal TC1 is used as an example.

[0083] First, the pixel block reset signal RT is turned on, resetting the voltage of the floating diffusion FD and reading out the reset voltage through the source follower SF. Next, the transfer signal TX and pixel signal readout signal TC1 are turned on simultaneously, transferring signal charge from the capacitor CM1 to the floating diffusion FD and reading out the signal voltage through the source follower SF. The signal from the capacitor CM1 is sequentially read out to the floating diffusion FD by the operation of the transfer signal TX. At this time, the pixel signal readout signals TC2 to TC4 do not operate until the pixel signal readout signal TC1 has completed its operation for one horizontal scanning period. After the pixel signal readout signal TC1 has completed its operation for one horizontal scanning period, the pixel signal readout signal TC2 performs a similar operation, and this is repeated in the same manner as the pixel signal readout signal TC3 and pixel signal readout signal TC4.

[0084] 8 is a third timing chart showing an example of pixel readout performed by the pixel readout circuit according to this embodiment. This diagram shows an example of a readout method in Fast mode. That is, this diagram shows an example of a readout method when binning is performed.

[0085] First, the pixel block reset signal RT is turned on, resetting the voltage of the floating diffusion FD and reading out the reset voltage through the source follower SF. Next, the transfer signal TX and pixel signal readout signals TC1 to TC4 are turned on simultaneously, transferring signal charges from the capacitors CM1 to CM4 to the floating diffusion FD and reading out the signal voltage through the source follower SF. Here, the pixel selection signal SL is selected in the order of n, n+1, n+2, n+3, etc. for each horizontal scanning period.

[0086] 9 is a fourth timing chart showing an example of pixel readout performed by the pixel readout circuit according to this embodiment. This diagram shows an example of a readout method in which pixel block m is in Normal mode, pixel block m+1 is in Fast mode, pixel block m+2 is in Bright mode, and pixel block m+3 is in Fats mode. That is, this diagram shows an example in which a pixel block that performs binning and a pixel block that does not perform binning are adjacent in the vertical direction, and the reset operation when binning is performed and the reset operation when not performing binning are switched continuously. This diagram also shows the case where pixel signal readout signal TC1 is used as an example.

[0087] In this embodiment, the operation modes of adjacent pixel blocks may differ from each other. As shown in the figure, after a readout operation in Normal mode (i.e., without binning) is performed in pixel block m, a readout operation in Fast mode (i.e., with binning) is performed in pixel block m+1, a readout operation in Bright mode (i.e., without binning) is performed in pixel block m+2, and a readout operation in Fats mode (i.e., with binning) is performed in pixel block m+3. In this way, according to this embodiment, in the horizontal readout operation, the readout method is switched for each pixel block depending on whether binning is performed or not.

[0088] In the example shown in the figure, the pixel signal readout signal TC1 is used as an example, so in Normal mode and Bright mode (i.e., the figure shows a case where binning is not performed), the transfer signal TX and the pixel signal readout signal TC1 are simultaneously on, and the pixel signal readout signals TC2 to TC4 are not operating.

[0089] Fig. 10 is a fifth timing chart showing an example of pixel readout performed by the pixel readout circuit according to the present embodiment. In the example shown in Fig. 10, the order in which the operation modes are switched is the same as the example shown in Fig. 9, but differs from Fig. 9, which uses the pixel signal readout signal TC1 as an example, in that the pixel signal readout signal TC2 is used as an example. In the example shown in Fig. 10, because the pixel signal readout signal TC2 is used as an example, in Normal mode and Bright mode (i.e., in the figure, when binning is not performed), the transfer signal TX and pixel signal readout signal TC2 are simultaneously on, and the pixel signal readout signals TC1, TC3, and TC4 are not operating.

[0090] FIG. 11 is a block diagram showing an example of the internal configuration of an imaging device according to this embodiment. At least some of the functions of the configuration of the imaging device 3 can be implemented using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. The input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from and to the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906. All or part of the functional units provided in the imaging device 3 may be realized using hardware such as an ASIC, a PLD, or an FPGA. All or part of the functional units may be realized by a combination of software and hardware.

[0091] [Summary of the embodiment] According to the above-described embodiment, the pixel readout circuit reads pixel values ​​from a pixel block using a global shutter system. The pixel block from which the pixel readout circuit reads pixel values ​​includes n pixels (n is a natural number greater than or equal to 1) and a floating diffusion FD shared by the n pixels, and is configured to extract pixel values ​​depending on whether binning is performed. The pixel readout circuit includes a pixel signal readout switch (specifically, transistors Tr311, Tr321, Tr331, and Tr311 shown in FIG. 4 ) for each pixel, thereby transferring charge read out by a global shutter signal to the floating diffusion FD for each pixel. The pixel readout circuit also includes a GS signal transfer switch (specifically, transistor Tr20 shown in FIG. 4 ). When binning is not performed, the pixel values ​​are read out at a frequency obtained by dividing the global shutter signal, which is controlled at a frequency corresponding to the frame rate, by n. When binning is performed, the pixel values ​​are read out at the same frequency as the global shutter signal. By adopting such a configuration, it becomes possible to read pixel values ​​by a global shutter method from an image sensor that can switch between the presence and absence of binning in a local area.

[0092] It should be noted that the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects in addition to or in place of the above-described effects that would be apparent to those skilled in the art from the description of this specification. Furthermore, the present invention is not limited to these embodiments, and various modifications and substitutions can be made within the scope of the present invention. [Explanation of symbols]

[0093] 1. Imaging system 3. Imaging device 5. Video display devices 31 Lens 32 Prism 33 Sub-sensor 34 Image Sensor 35 Sensor drive board 36 Sensor power supply board 37 Scene information and image acquisition board 38 Signal Processing Board 39 Power Supply Module 51 Spatiotemporal Interpolation Processing Unit 52 Signal processing section 53 Display section

Claims

1. A pixel readout circuit that reads pixel values ​​from a pixel block that has n pixels (n is a natural number equal to or greater than 1) and a floating diffusion shared by the n pixels, and that is capable of extracting pixel values ​​according to whether or not binning is performed, by a global shutter method, a pixel signal readout switch provided for each pixel; a GS signal transfer switch that reads out pixel values ​​at a frequency obtained by dividing a global shutter signal, which is controlled at a frequency according to a frame rate, by n when binning is not performed, and that reads out pixel values ​​at the same frequency as the global shutter signal when binning is performed; A pixel readout circuit comprising:

2. further comprising a reset signal transfer switch that controls transfer of a reset signal for each pixel block; the reset signal is controlled in response to a selectable exposure time; the reset signal transfer switch transfers a reset signal to the pixel block in accordance with an exposure time of the corresponding pixel block; 2. The pixel readout circuit according to claim 1.

3. reading out pixel values ​​from a backside laminated circuit having a pixel array layer having a plurality of the pixel blocks and a global control circuit layer that performs control for reading out from the pixel blocks using a global shutter; 2. The pixel readout circuit according to claim 1.

4. a mode control circuit disposed in the pixel array layer and configured to output a control signal to the pixel signal readout switch in accordance with a readout mode; 4. The pixel readout circuit according to claim 3.

5. a vertical scanning circuit disposed in the pixel array layer, the vertical scanning circuit outputting a signal for reading out the charge read out by the GS signal transfer switch, a reset signal for resetting the charge accumulated in the floating diffusion, and a pixel selection signal for selecting a pixel to be read out, in accordance with a readout mode; 4. The pixel readout circuit according to claim 3.

6. a pixel readout circuit according to any one of claims 1 to 5; a pixel array layer having a plurality of the pixel blocks; a global control circuit layer that controls reading from the pixel block using a global shutter; An imaging element comprising:

Images