An image sensor with pixel readout at different frame rates

Abstract

An image sensor includes a pixel array that includes a plurality of pixels that accumulate charge in response to incident radiation, sense regions, transfer transistors operatively connected between the pixels and the sense regions, and pixel readout circuits, wherein each pixel readout circuit is operatively connected to a different one of the sense regions. The pixel readout circuits output pixel signals on signal lines, wherein a voltage of a pixel signal is a function of charge stored in the sense region which is operatively connected to the pixel readout circuit. A sensor controller controls the transfer transistors and the pixel readout circuits to output, at a low first frame rate, a first sequence of pixel signals to the signal lines based on a first set of the pixels and, at a high second frame rate, a second sequence of pixel signals to the signal lines based on a predefined second set of the pixels, wherein a readout of the second set of the pixels is controlled to provide less pixel signals for further processing than a readout of the first set of the pixels.

Description

[0001] 73844

[0002] 1

[0003] AN IMAGE SENSOR WITH PIXEL READOUT AT DIFFERENT FRAME RATES

[0004] The present disclosure relates to an image sensor with pixels outputting pixel signals at different frame rates.

[0005] BACKGROUND

[0006] Image sensors contain a large number of pixels arranged in a two-dimensional matrix. The number of pixels in such a pixel array determines the spatial resolution of the image sensor. Typically, a pixel array is read row by row and the time required to read all pixel rows consecutively sets an upper limit on the frame rate and temporal resolution. Binning approaches combine a number of pixels for a single readout to improve sensitivity in low light conditions at the expense of spatial resolution.

[0007] SUMMARY

[0008] The present disclosure mitigates shortcomings of image sensors with respect to applications that benefit from receiving both image streams with high temporal resolution and high spatial resolution.

[0009] For this purpose, an image sensor includes a pixel array that includes pixels that accumulate charge in response to incident radiation, sense regions, transfer transistors operatively connected between the pixels and the sense regions, and pixel readout circuits, wherein each pixel readout circuit is operatively connected to a different one of the sense regions. The pixel readout circuits output pixel signals on signal lines, wherein a voltage of a pixel signal is a function of charge stored in the sense region which is operatively connected to the pixel readout circuit. A sensor controller controls the transfer transistors and the pixel readout circuits to output, at a low first frame rate, a first sequence of pixel signals to the signal lines based on a first set of the pixels and, at a high second frame rate, a second sequence of pixel signals to the signal lines based on a predefined second set of the pixels, wherein a readout of the second set of the pixels is controlled to provide less pixel signals for further processing than a readout of the first set of the pixels.

[0010] In this way, the image sensor may provide a first data stream of high spatial resolution images at low temporal resolution and a second data stream of low spatial resolution images at high temporal resolution. The images for the first data stream and the second data stream are generated in an interlaced manner, whereby the pixel readouts for the first data stream and the second data stream can alternate and / or overlap. An application receiving the two data streams may use the first data stream and the second data stream for different tasks. For example, the application may use the first data stream to analyze the content of the images and use the second data stream for motion analysis and / or motion tracking.

[0011] BRIEF DESCRIPTION OF THE DRAWINGS 73844

[0012] 2

[0013] A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:

[0014] FIG. 1 illustrates a simplified block diagram of an image sensor providing a first data stream of high spatial resolution images at low temporal resolution and a second data stream of low spatial resolution images at high temporal resolution, in accordance with an embodiment.

[0015] FIG. 2 illustrates a simplified circuit diagram of a pixel circuit for use in the image sensor of FIG. 1.

[0016] FIG. 3 illustrates a schematic block diagram of a pixel for use in the image sensor of FIG. 1.

[0017] FIG. 4 illustrates a simplified block diagram of an image sensor with pixels for providing a first data stream of high spatial resolution images at low temporal resolution from first pixels and a second data stream of low spatial resolution images at high temporal resolution from second pixels, in accordance with an embodiment.

[0018] FIG. 5 illustrates a simplified block diagram of an image sensor with first pixels for generating a first data stream and second, different pixels for generating a second data stream, in accordance with an embodiment.

[0019] FIG. 6 illustrates a simplified circuit diagram of a pixel group with pixel circuits, in accordance with an embodiment.

[0020] FIG. 7 illustrates a simplified circuit diagram of signal lines transmitting pixel output signals of pixel groups to column signal circuits, in accordance with an embodiment.

[0021] FIG. 8 illustrates a simplified circuit diagram of signal lines transmitting pixel output signals of pixel groups to column signal circuits with multiplexers, in accordance with an embodiment.

[0022] FIG. 9A, FIG. 9B, and FIG. 9C illustrate the timing of exposures and readouts of pixel circuits for a first data stream of high spatial resolution images at low temporal resolution from first pixels and a second data stream of low spatial resolution images at high temporal resolution from second, different pixels, in accordance with an embodiment.

[0023] FIG. 10 to FIG. 13 illustrate various arrangements of pixels assigned to a first data stream and a second data stream, in accordance with embodiments.

[0024] FIG. 14 illustrates the timing of exposures and readouts of pixel circuits for a first data stream of high spatial resolution images at low temporal resolution from first pixels and a second data stream of low spatial resolution images at high temporal resolution from second pixels, in accordance with an embodiment with the second pixels being a subset of the first pixels. 3

[0025] FIG. 15 illustrates the arrangement of second pixels for a second data stream resulting from diagonal binning of first pixels for a first data stream.

[0026] FIG. 16 illustrates the arrangement of second pixels for a second data stream resulting from V2H2 binning of first pixels for a first data stream.

[0027] FIG. 17 illustrates signal lines transmitting pixel output signals of pixel groups to column signal circuits, in accordance with an embodiment.

[0028] FIG. 18 illustrates signal lines transmitting pixel output signals of pixel groups to column signal circuits with multiplexers and switches, in accordance with an embodiment.

[0029] FIG. 19 illustrates a schematic block diagram of on-chip processing stages shared by a first data stream and a second data stream, in accordance with an embodiment.

[0030] FIG. 20 illustrates organization of data frames used on-chip by a first data stream and a second data stream, in accordance with an embodiment.

[0031] FIG. 21 a is a diagram showing an example of a laminated structure of a solid-state imaging device according to an embodiment of the present disclosure.

[0032] FIG. 22 is a block diagram depicting an example of a schematic configuration of a vehicle control system.

[0033] FIG. 23 is a diagram of assistance in explaining an example of installation positions of an outside-vehicle information detecting section and an imaging section of the vehicle control system of FIG. 22.

[0034] DETAILED DESCRIPTION

[0035] Embodiments for implementing techniques of the present disclosure (also referred to as “embodiments” in the following) will be described below in detail using the drawings. The techniques of the present disclosure are not limited to the described embodiments, and various features in the embodiments are illustrative only. The same elements or elements with the same functions are denoted by the same reference signs. Duplicate descriptions are omitted.

[0036] Connected electronic elements may be electrically connected through a direct, permanent low-resistive connection, e.g., through a conductive line. The terms “electrically connected” and “signal-connected” may also include a connection through other electronic elements provided and suitable for permanent and / or temporary signal transmission and / or transmission of energy. Electronic elements can be electrically connected or signal-connected via resistors, capacitors, electronic switches such as MOSFETs, or transistor circuits such as transmission gates. Directly electrically connected electronic elements are connected through a low-resistive wiring, an ohmic contact and / or a unipolar semiconductor junction. At least one 4 electrical signal in a second electrical circuit in operational connection with a first electrical circuit changes in a predictable, intended manner to a change of an electrical signal in the first electrical circuit.

[0037] The load path of a transistor is the controlled current path through a transistor. A voltage applied to the gate of a field effect transistor controls the current flow through the load path (controlled path) between source and drain of the MOSFET by field effect.

[0038] A digital signal alternates between at least one active level and at least one inactive level. A digital signal having an active level is active. A digital signal having an inactive level is inactive. For each signal separately, the active level can be a digital high level and the inactive level a digital low level, or the active level can be the digital low level and the inactive level the digital high level.

[0039] FIG. 1 shows an image sensor 90 that provides a first data stream of high spatial resolution images at low temporal resolution and a second data stream of low spatial resolution images at high temporal resolution. The image sensor 90 is a complementary metal -oxide semiconductor (CMOS) image sensor and can include a pixel array 10, a sensor controller 80, a row decoder / driver 60, a column processing unit 20, and a digital readout unit 70. The pixel array 10 includes pixel circuits 100 organized in a two-dimensional array with columns and rows.

[0040] The pixel circuits 100 include active pixel sensors (APS) for intensity readout. When a pixel circuit 100 is exposed to radiation, the pixel circuit 100 integrates a photocurrent induced by the radiation during an exposure time (integration time) and stores the integrated charge. When a pixel circuit 100 is read, the pixel circuit 100 outputs an analog pixel signal with a voltage of the pixel signal being a function of the stored charge on a signal line 40.

[0041] The row decoder / driver 60 is in communication with each pixel circuit 100 of the pixel array 10 through one or more pixel control lines 50 and controls exposure, reset and readout of the pixel circuits 100. The pixel control lines 50 may include transfer signal lines, select signal lines and reset control lines, inter aha. The row decoder / driver 60 may control exposure, reset and readout of the pixel circuits 100 row by row, with some or all pixel circuits 100 of a same row being controlled for simultaneous reset, simultaneous exposure and simultaneous readout, and with the reset, exposure and readout of all rows being controlled according to a rolling shutter or global shutter approach.

[0042] Some or all pixel circuits 100 of a same column are connected to the same signal line 40 or to a same set of signal lines 40. Each pixel circuit 100 of the pixel array 10 is in communication with the column processing unit 20 via a signal line 40. The column processing unit 20 selectively receives the pixel signals output from selected pixel circuits 100, e.g., from some or all pixel circuits 100 in a selected row of pixel circuits 100. The column processing unit 20 converts the received analog pixel signals into digital pixel values.

[0043] The sensor controller 80 is in communication with the column processing unit 20, the row decoder / driver

[0044] 60 and, via the row decoder / driver 60, the pixel circuits 100 in the pixel array 10. The sensor controller 80 5 can provide signals to control the row decoder / driver 60 and the column processing unit 20 to perform exposure and readout according to a rolling shutter scheme or global shutter scheme.

[0045] The column processing unit 20 is in communication with the digital readout unit 70. The digital readout unit 70 receives the digital pixel values from the column processing unit 20. The digital readout unit 70 may temporarily store and / or process digital pixel values obtained in data phases and reset phases for correlated double sampling (CDS) and / or digital double sampling (DDS).

[0046] FIG. 2 shows one of the examples of a pixel circuit 100 that can be used in the image sensor 90 shown in FIG. 1. The pixel circuit 100 includes a photodetector 111 of a pixel 110, a transfer transistor 120, a sense region 130, and a pixel readout circuit 140 that may include at least a reset transistor 142, an amplifier transistor 144, and a select transistor 149.

[0047] The photodetector 111 includes a photoelectric conversion element that photoelectrically converts incident electromagnetic radiation into electric charges. The amount of electric charge generated in the photodetector 111 corresponds to the intensity of incident electromagnetic radiation. The photodetector 111 may include or consist of a photodiode which converts electromagnetic radiation incident on a detection surface into a detector current by means of the photoelectric effect. The electromagnetic radiation may include visible light, infrared radiation and / or ultraviolet radiation. The amplitude of the detector current is a function of the intensity of the incident electromagnetic radiation, wherein in the intensity range of interest the detector current may increase approximately linearly with increasing intensity of the detected electromagnetic radiation.

[0048] The sense region 130 can include a floating diffusion region and / or a capacitor electrode. In a reset phase, a potential of the sense region 130 is a function of the pixel dark current representing the noise. In a data phase, the potential of the sense region 130 is a function of the integrated detector current.

[0049] A load path of the transfer transistor 120 may be electrically connected between a cathode of the photodetector 111 and the sense region 130. The transfer transistor 120 serves as transfer element for transferring charge from the photodetector 111 to the sense region 130 in or after an integration period. A transfer signal TRG is supplied to the gate (transfer gate) of the transfer transistor 120 through a transfer signal line. The transfer signal TRG changes between an active signal level (“active transfer signal”) and an inactive signal level (“inactive transfer signal”). In response to an active transfer signal TRG, the transfer transistor 120 may transfer electrons photoelectrically converted by the photodetector 111 to the sense region 130. In the illustrated example, the active signal level is the high level.

[0050] A reset signal RST is supplied to a gate of the reset transistor 142 through a reset control line. The reset signal RST changes between an active signal level (“active reset signal”) and an inactive signal level (“inactive reset signal”). In response to an active reset signal RST, the reset transistor 142 may connect the sense region 130 to a reset potential, which may be the positive supply potential VDD. In the illustrated embodiment, the active signal level is the high level. 73844

[0051] 6

[0052] A controlled load path of the amplifier transistor 144 is electrically connected between the positive supply potential VDD and a signal line 40. The sense region 130 is connected to the gate of the amplifier transistor 144. A potential at the gate of the amplifier transistor 144 is equal to the potential of the sense region 105. When the pixel circuit 100 is selected, the amplifier transistor 144 may be in a source follower configuration with a constant current source electrically connected to the signal line 40.

[0053] In particular, the load path of the amplifier transistor 144 and a load path of the select transistor 149 may be electrically connected in series between the positive supply potential VDD and the signal line 40. A row select signal SEL is supplied to the gate of the select transistor 149 through a select signal line. The select signal SEL changes between an active signal level (“active select signal”) and an inactive signal level (“inactive select signal”). In the illustrated example, the select transistor 149 is an nFET (n channel field effect transistor) and the active signal level is the high level. The active select signal turns on the select transistor 149.

[0054] FIG. 3 shows a pixel 110 that includes a photodetector 111 and an optical element 112. Incoming radiation 113 passes the optical element 112 before entering a radiation sensitive area of the photodetector 111. Some of the optical elements 112 of the pixel array 10 may selectively transmit radiation in particular ranges of wavelengths. Some optical elements 112 may transmit radiation in the complete range of visible light. Other optical elements 112 may transmit radiation in color ranges of visible light, e.g. green light, blue light, red light.

[0055] FIG. 4 shows an image sensor 90 that includes a pixel array 10. The pixel array 10 includes a plurality of pixels 110, transfer transistor 120, sense regions 130 and pixel readout circuits 140. The pixels 110 are configured to accumulate charge in response to incident radiation. The transfer transistors 120 are operatively connected between the pixels 110 and the sense regions 130. Each pixel readout circuit 140 is operatively connected to a different one of the sense regions 130. The image sensor 90 further includes signal lines 40. The pixel readout circuits 140 are configured to output pixel signals on the signal lines 40, wherein a voltage of a pixel signal is a function of charge stored in the sense region 130 that is operatively connected to the pixel readout circuit 140. A sensor controller 80 is configured to control the transfer transistors 120 and the pixel readout circuits 140 to output, at a low first frame rate, a first sequence of pixel signals to the signal lines 40 based on a first set of the pixels 110 and, at a high second frame rate, a second sequence of pixel signals to the signal lines 40 based on a predefined second set of the pixels 110, wherein a readout of the second set of the pixels is controlled to provide less pixel signals for further processing than a readout of the first set of the pixels.

[0056] The components of the pixel array 10 may be organized in pixel circuits 100, wherein each pixel circuit 100 may include one or more, e.g., two or four pixels 110, one or more, e.g., two or four transfer transistors 120, one sense region 130, and one pixel readout circuit 140. The number of pixels 110 and the number of transfer transistors 120 may be the same, so that each pixel 110 can be selectively connected to the sense region 130 via a separate transfer transistor 120. 7

[0057] The pixel circuits 100 of the pixel array 10 may be organized in rows and columns, wherein all or some pixel circuits 100 in a same row may be controlled simultaneously, and wherein all or some pixel circuits 100 in a same column may share common signal lines 40 in a time-division multiplexed manner.

[0058] The pixel 110 may include a photodetector 111 and an optical element 112 as shown in FIG. 3.

[0059] Each transfer transistor 120 may be arranged with a controlled path electrically connected between the photodetector 111 and the sense region 130. Each transfer transistor 120 may connect the photodetector 111 directly with the sense region 130 with no further active electric component electrically connected in series between the photodetector 111 and the sense region 130. Alternatively, the controlled path of at least one more active electric component may be electrically connected in series with the transfer transistor 120 between the photodetector 111 and the sense region 130. An active transfer signal turns on the transfer transistor 120 to let pass charge accumulated on an electrode of the photodetector 111 to the sense region 130 or at least into direction of the sense region 130.

[0060] The sense region 130 may be or include a floating diffusion region and / or an electrode of another type of capacitive structure. Each sense region 130 of a pixel circuit 110 may be operatively connected to one or more pixels 110, e.g., to two pixels 110 or four pixels 110.

[0061] The pixel readout circuits 140 may be pixel circuits with four transistors for rolling shutter operation, or may include more than four transistors, e.g., for global shutter operation, pixel-internal double sampling operation, or enhanced sensitivity range operation. Each signal line 40 may be operatively connected to some or all pixel readout circuits of a column of pixel circuits 100.

[0062] When a pixel output circuit 140 is selected, the pixel output circuit 140 passes a pixel signal derived from the charge stored in the sense region 130 to a signal line 40, wherein the voltage level of the pixel signal is a function of the charge stored in the sense region 130.

[0063] The pixels 110 of all pixel circuits 100 may have the same electrical configuration and the same or different optical sensitivities. Alternatively, the first pixels and the second pixels may have the different electrical configurations and / or optical sensitivities.

[0064] The sensor controller 80 may control the transfer transistors 120 and the pixel readout circuits 140 via a row decoder / driver 60. The row decoder / driver 60 operatively connects the sensor controller 80 to the pixel circuits 100 of the pixel array 10. The row decoder / driver 60 may demultiplex control signals received from the sensor controller 80 to a plurality of rows of pixel circuits 100 or rows of pixel groups 11.

[0065] The number of pixels 110 of the first set (“first pixels”) may be equal to the total number of pixels 110. That is, the pixels 110 of the first and second set are used for both the first sequence of pixel signals (“first data stream”) and the second sequence of pixel signals (“second data stream”). The second pixels are shared between the first data stream and the second data stream. 8

[0066] Alternatively, the first data stream and the second data stream are based on different pixels. That is, each pixel is dedicated to not more than one of the data streams. The number of pixels 110 of the first set (“first pixels”) and the number of pixels 110 of the second set (“second pixels”) may complement to the total number of pixels 110.

[0067] Alternatively, both the first set (“first pixels”) and the second set (“second pixels”) may include all pixels 110 of the pixel array 10, but for the readout of the second set the charge accumulated in different pixels 110 of the same pixel circuits 100 and / or stored in sense regions 130 of different pixel circuits 100 are combined before and / or during readout. For example, the first data stream may result from a readout of individual pixels and the second data stream from a readout of superpixels, which are obtained by combining the individual pixels and / or by sub-sampling, e.g., by binning.

[0068] For each of the alternatives above, a complete readout of the second set of the pixels 110 generates less pixel signals for further processing than a readout of the first set of the pixels 110. A complete readout of the second set of the pixels 110 requires less time and can be repeated at a higher frame rate than complete readouts of the first set of the pixels 110.

[0069] The sensor controller 80 may be configured to change between a dual data stream mode and a single data stream mode, wherein in the dual data stream mode the sensor controller 80 controls the transfer transistors 120 and the pixel readout circuits 140 to output the first sequence of pixel signals to the signal lines 40 based on the first set 1101 of the pixels 110 and output the second sequence of pixel signals to the signal lines 40 based on the second set 1102 of the pixels 110, and wherein in the single data stream mode the sensor controller 80 is configured to control the transfer transistors 120 and the pixel readout circuits 140 to output a single sequence of pixel signals to the signal lines 40 based on both the first and second sets of the pixels.

[0070] In the single data stream mode, all images have the same high spatial resolution and the images are output at a frame rate that is lower than the second frame rate and higher than the first frame rate. The flexible design of the pixel array 10 allows to control the same pixel array also in a conventional way to output a single data stream derived from all pixels. The sensor controller 80 may change the data stream mode periodically or in response to a change of an internal signal, to user interaction, to a change of lighting conditions, by way of example.

[0071] The image sensor 90 may further include a column processing unit 20 that converts the pixel signals into digital pixel values, wherein for a complete readout of the second set of pixels 110 less pixel signals are converted into digital pixel values than for a complete readout of the first set of pixels 110.

[0072] For each of the alternatives, the column processing unit 20 receives less pixel signals for the second data stream than for the first data stream. The pixel signals for an image in the second data stream can be converted faster into digital values than the pixel signals required for an image in the first data stream. The digital image data for the second data stream can be obtained at a higher frame rate than the digital image data for the first data stream. 9

[0073] In FIG. 5, pixel circuits 100 with pixels 110 included in the second set 1102 are marked by dotted background. The number of pixels 110 of the second set (“second pixels”) is smaller than the total number of pixels 110 in the pixel array. The designation of the pixels 110 to the first set 1101 of pixels 110 and the second set 1102 of pixels 110 may be as follows:

[0074] According to an embodiment, each pixel 110 is included in the first set 1101 of pixels 110, and the second set 1102 includes less pixels 110 than the first set 1101 of pixels 110.

[0075] The first set 1101 of pixels 110 includes all pixels 110 of the pixel array 100. The second set 1102 of pixels 110 includes a true subset of all pixels 110 of the pixel array 100. For example, for pixel circuits 100 with more than one pixel 110, all pixels 110 may be read out for obtaining the first data stream and only one of the pixels 110 may be read out for obtaining the second data stream. Alternatively or in addition, each row of pixel circuits 100 may be read out for obtaining the first data stream and only some of the rows, e.g., each second, each fourth, each eighth 110 may be read out for obtaining the second data stream. Alternatively or in addition, each column of pixel circuits 100 may be read out for obtaining the first data stream and only some of the columns, e.g., each second, each fourth, each eighth 110 or each tenth may be read out for obtaining the second data stream.

[0076] According to another embodiment, each pixel 110 is included either in the first set 1101 or the second set 1102 of pixels 110, wherein the second set 1102 of pixels 110 includes less pixels 110 than the first set 1101 of pixels.

[0077] Each pixel circuit 100 is included in either the first set 1101 of pixels 110 or the second set 1102 of pixels 110, or a third set of pixels, wherein pixels in the third set of pixels 110 are not included in either in the first set 1101 of pixels 110 or the second set 1102 of pixels 110.

[0078] The pixel circuits 110 may be organized in pixel groups 11, wherein each pixel group 11 includes more than one pixel circuit 100. Each sense region 130 of a pixel group 11 is operatively connected to a different one of the pixel readout circuits 140 of the same pixel group 11. The pixel groups 11 are organized in a two-dimensional matrix with rows of pixel groups 11 and columns of pixel groups 11.

[0079] The total number of pixels 110 per pixel group 11 may be one or an even number, e.g. a power of two. The total number of pixels 110 per pixel group 11 may be at least four. For example, the total number of pixels 110 per pixel group 11 may be 8, 16 or 32. The number of pixels 110 of the second set (“second pixels”) may be smaller than the total number of pixels 110 per pixel group 11. For example, the number of pixels 110 of the second set 1102 of pixels 110 may be equal 1, 2, or 4. In the example shown, the number of pixel circuits 100 per pixel group 11 is four and the number of pixels 110 in the second set 1102 of pixels 110 is one.

[0080] The pixel array 10 may include a plurality of identically configured unit cells. Each unit cell may include an even number of the pixel groups 11. Corresponding pixel groups 11 in different unit cells are located in 73844

[0081] 10 the same relative location within the unit cell in the plane of the pixel array 10. In all unit cells, the configuration and orientation of corresponding pixel groups 11 in the plane of the pixel array 10 may be the same. The pixel groups 11 may have the same electrical and optical filter configuration. The orientation of the pixel groups 11 in the plane of the pixel array 10 may be the same in all unit cells or may be different. The pixel groups 11 of a unit cell may be oriented symmetrically with respect to a symmetry axis in the plane of the pixel array 10.

[0082] The number of pixels 110 of the first set 1101 of pixels 110 (“first pixels”) may be equal to the total number of pixels 110 per pixel group 11. That is, the pixels 110 of the second set 1102 (“second pixels”) are used for both the first data stream and the second data stream. The second pixels are shared between the first data stream and the second data stream. Alternatively, the number of first pixels and the number of second pixels complement to the total number of pixels 110 per pixel group 11. That is, the first data stream and the second data stream are based on different pixels. Each pixel 110 is dedicated to only one of the data streams.

[0083] The first pixels and the second pixels may have the same electrical configuration and sensitivity. Alternatively, the first pixels and the second pixels may have the different electrical configurations and / or optical sensitivities. The optical properties of the optical elements of the first pixels and the second pixels may be different. In addition to pixel groups 11 with both first and second pixels, the pixel array 10 may include pixel groups 11 that include exclusively first pixels 110.

[0084] Each signal line 40 is operatively connected to corresponding pixel readout circuits 140 in a plurality of pixel groups 11 assigned to a same column of pixel groups 11. In the example shown, each signal line 40 is operatively connected to corresponding pixel readout circuits 140 in all pixel groups 11 associated with the same column of pixel groups 11. Alternatively, each signal line 40 may be operatively connected to an integer even fraction of all pixel groups 11 associated with the same column of pixel groups 11, e.g., every 2nd, every 4th or every 8th pixel group 11.

[0085] FIG. 6 shows a pixel group 11 with four pixel circuits 110. Each pixel circuit 110 includes four pixels 110 individually connectable to a shared sense region 130, and a pixel readout circuit 140. Three pixel circuits 100 are associated with a first subset 1101 of pixels 110. One pixel circuit 100 is associated with a second subset 1101 of pixels 110.

[0086] The image sensor further includes sets of transfer signal lines 51, 52, wherein each set of transfer signal lines 51, 52 is operatively connected to a different row of pixel groups 11. Each set of transfer signal lines 51, 52 includes at least one first transfer signal line 51 operatively connected to the transfer transistors 120 operatively connected to the first set 1101 of pixels 110 and at least one second transfer signal line 52 operatively connected to the transfer transistors 120 operatively connected to the second set 1102 of pixels 73844

[0087] 11

[0088] Each first transfer signal line 51 may be exclusively connected to transfer transistors 120 of the first sets of pixels 110 of the same row of pixel groups 11. Each second transfer signal line 52 may be exclusively connected to transfer transistors 120 of the second sets of pixels 110 of the same row of pixel groups 11.

[0089] The sensor controller 80 may be configured to control the transfer transistors 120 of each row of pixel groups 11 to let overlap first integration times for the first set of pixels 110 with second integration times for the second set of pixels 110. The integration times (exposure times) for the first set 1101 of pixels 110 may be a function of current lighting conditions and may vary with time. The integration times for the second set 1102 of pixels 110 may be a function of current lighting conditions or may be selected independently from the current lighting conditions. For example, the integration times for the second set 1102 of pixels 110 may be equal to or may approximate the integration time for the first set of pixels at the brightest illumination condition.

[0090] The image sensor further includes sets of select signal lines 53, 54, wherein each set of select signal lines 53, 54 is operatively connected to a different row of pixel groups 11, wherein each set of select signal lines 53, 54 includes at least one first select signal line 53 operatively connected to the pixel readout circuits 140 operatively connected to the first set 1101 of pixels 110 and at least one second select signal line 54 operatively connected to the pixel readout circuits 140 operatively connected to the second set 1102 of pixels 110.

[0091] Each first select signal line 53 may be exclusively connected to pixel readout circuits 140 of the first sets of pixels 110 of the same row of pixel groups 11. Each second select signal line 54 may be exclusively connected to pixel readout circuits 140 of the second sets of pixels 110 of the same row of pixel groups 11. The sensor controller 80 may be configured to control the pixel readout circuits 140 of each row of pixel groups 11 to let overlap readout times for the first sets of pixels 110 in the row of pixel groups 11 with readout times for second sets of pixels 110 in the same row of pixel groups. At the level of the pixel array 10, the first and second data streams can be controlled completely independent from each other.

[0092] The signal lines 40 are grouped into sets of signal lines 40, wherein each set of signal lines 40 is operatively connected to a different column of pixel groups 11. Each set of signal lines 40 includes at least one first output signal line 41 operatively connected to the pixel readout circuits 140 of the first set 1101 of pixels 110 in the column of pixel groups 11 and at least one second output signal line 42 operatively connected to the pixel readout circuits 140 of the second set 1102 of pixels 110 in the column of pixel groups 11.

[0093] Each first output signal line 41 may be exclusively connected to pixel readout circuits 140 of the first set of pixels 110 in the same column of pixel groups 11. Each second output signal line 42 may be exclusively connected to pixel readout circuits 140 of the second set of pixels 110 in the same column of pixel groups. To each signal line 40, the same number of pixel readout circuits 140 may be operatively connected.

[0094] In the example shown, more pixel readout circuits 140 are operatively connected to a first output signal line 41 than to a second output signal line 42. 73844

[0095] 12

[0096] Two or more pixels 110 of the first set 1101 in a pixel group 11 may be operatively connected with a same first output signal line 41. Each second pixel 110 of a pixel group 11 of a column of pixel groups 11 may be operatively connected with a different second output signal line 42. The sensor controller 80 can be configured to control the pixel output circuits 140 of the second pixels in each column of pixel groups 11 to be read out faster than the pixel output circuits 140 of the first pixels in the column of pixel groups 11 . The faster readout increases the possible maximum frame rate for the second data stream.

[0097] Referring to FIG. 7, the image sensor further includes shared column signal circuits 201 operatively connected with the first and second signal lines 41, 42, wherein each shared column signal circuit 201 is configured to convert pixel signals transmitted on one of the second signal lines 42 in first conversion periods and convert pixel signals transmitted on one of the first signal lines 41 in second conversion periods.

[0098] In the first conversion periods, the shared column signal circuits 201 may convert analog pixel signals transmitted on the second signal lines 42 into digital pixel values. In the second conversion periods, the shared column signal circuits 201 may convert the analog pixel signals transmitted on the first signal lines 41 into digital pixel values. The shared analog -to-digital converters 201 convert the pixel signals from two different signal lines 40 in a time-division multiplexed way.

[0099] The image sensor may further include not-shared column signal circuits 202, wherein the not-shared column signal circuits 202 may convert pixel signals transmitted on the first signal lines 41 into digital pixel values in both the first and the second conversion periods.

[0100] The shared column signal circuits 200 may reduce the readout time for a frame(image) for the first data stream with no or only small impact on the readout time for a frame (image) of the second data stream.

[0101] FIG. 8 shows multiplexers 210, wherein each multiplexer 210 is configured to pass the pixel signal transmitted on one of the first signal lines 41 in first sampling periods and pass pixel signals transmitted on one of the second signal lines 42 in second sampling periods.

[0102] Each multiplexer 201 may be in operational connection with at least two signal lines 40. A column processing unit 20 as described above may include as many multiplexers 201 as first signal lines 41.

[0103] The image sensor further includes shared analog -to-digital converters 221. Each shared analog -to-digital converter 221 is configured to convert, in first conversion periods, pixel signals received in the first sampling periods from one of the multiplexers 210, and convert, in second conversion periods, pixel signals received from the same multiplexer 210 in the second sampling periods.

[0104] In the first conversion periods, the shared analog -to-digital converters 221 may convert analog pixel signals transmitted on the second signal lines 42 into digital pixel values. In the second conversion periods, the shared analog -to-digital converters 221 may convert analog pixel signals transmitted on the first signal lines 41 into digital pixel values. 73844

[0105] 13

[0106] The image sensor may further include not-shared analog-to-digital converters 222, wherein the not-shared analog -to-digital converters 221 may convert pixel signals transmitted on the first signal lines 41 in both the first and the second conversion periods into digital pixel values.

[0107] The sensor controller 80 may be operatively connected to the multiplexers 210 and controls the multiplexers 210 to pass the pixel signals transmitted on one of the first signal lines 41 in first sampling periods and pass pixel signals transmitted on one of the second signal lines 42 in second sampling periods.

[0108] The upper diagram in FIG. 9A shows a rolling shutter operation for an image sensor. The row address n# is increased with time t. At tO, the n-th row of pixel circuits or pixel groups starts an exposure cycle marked with line Sh. The lower diagram in FIG. 9A shows the transfer signal TRG and the reset signal RST for the n-th row, wherein the active level of the transfer signal TRG and the reset signal RST is the high level. At tO, both the reset signal RST and the transfer signal TRG become active to reset the potential at the cathode of the photodetector and the potential of the sense region. When the transfer signal TRG and the reset signal RST for the n-th row become inactive, the integration period (exposure period) starts and the photodetector accumulates charge.

[0109] The upper diagram in FIG. 9B shows a rolling readout operation for the image sensor. The row address n# is increased with time t. At tl, the n-th row of pixel circuits or pixel groups starts a readout marked with line Rd. The lower diagram in FIG. 9B shows the select signal SEL, the transfer signal TRG and the reset signal RST for the n-th row, wherein the active level of the select signal SEL, the transfer signal TRG and the reset signal RST is the high level. At tl, the active select signal connects the pixel circuit or the group of pixel circuits with signal lines. Then, the reset signal RST becomes active for a reset phase. The potential of the sense region 130 is a function of the pixel dark current representing the noise. After the reset phase, the transfer signal TRG becomes active for a data phase. In the data phase, the potential of the sense region 130 is a function of the charge accumulated by the photodetector during the integration time, which ends at the beginning or end of the data phase.

[0110] FIG. 9C shows the timing of the integration periods for pixels used for the first data stream and pixels used for the second data stream for the case that the first pixels and the second pixels are different pixels and can be read out independently from each other.

[0111] Pixels of the first subset integrate the detector current between the first shutter Shi and the first readout Rdl. Pixels of the second subset integrate the detector current between the second shutter Sh2 and the second readout Rd2. The integration time of the first set of pixels may be a function of the lighting conditions and may vary. The integration time of the second set of pixels may be independent from the lighting conditions. Since less pixels, e.g., less pixel rows are addressed, the lines indicating the second shutter Sh2 and the second readout Rd2 are steeper than the ones for the first shutter Shi and the first readout Rdl. The frame rate FR2 for the second data stream resulting from the readout of the second set of pixels is faster than the frame rate FR1 for the first data stream resulting from the readout of the first set of pixels. 73844

[0112] 14

[0113] The analog-to-digital converters of the column processing unit convert the pixel signals of the first set of pixels during the periods Strl and the pixel signals of the second set of pixels during the periods Str2. During the period Strl+Str2 the analog-to-digital converters convert pixel signals from both the first set of pixels and the second set of pixels. The analog-to-digital converters are shared between the two data streams.

[0114] FIG. 10 to FIG. 14 show various arrangements of the second set 1102 of pixels 110 for pixel circuits with four pixels 110 and one sense region 130. Pixels sensitive to red light are illustrated without a dot pattern, pixels sensitive to green light are illustrated with a sparse-dot pattern and pixels sensitive to blue light with a dense-dot pattern. The assignment between dot pattern and light sensitivity is the same in the rest of the document. The pattern for the assignment of pixels 110 to the first and second sets repeats in terms of unit cells. In the illustrated embodiment the unit cell includes 16 pixel circuits arranged in a 4x4 array. The illustrated examples show Bayer arrangements of color filters. Other examples may be based other arrangements.

[0115] In FIG. 10, the second set 1102 of pixels 110 include all pixels 110 of the pixel circuits in the comers of the unit cell. The first set of pixels 110 may include all other pixels 110 or all pixels 110 of the unit cell.

[0116] In FIG. 11 to FIG. 13 show alternative arrangements for pixels 110 for the second set 1102 of pixels 110. In each arrangement, the first set of pixels 110 may include all other pixels 110 or all pixels 110 of the unit cell.

[0117] FIG. 14 shows a timing of the integration periods for pixels used for the first data stream and pixels used for the second data stream for the case that the pixels of the second set of pixels result from combinations of the pixels of the first set of pixels.

[0118] Pixels of the first subset integrate the detector current between the first shutter Shi and the first readout Rdl. Pixels of the second subset integrate the detector current between the second shutter Sh2 and the second readout Rd2. Since less pixels, e.g., less pixel rows are addressed, the lines indicating the second shutter Sh2 and the second readout Rd2 can be steeper than the ones for the first shutter Shi and the first readout Rdl . The frame rate FR2 for the second data stream resulting from the readout of the second set of pixels is faster than the frame rate FR1 for the first data stream resulting from the readout of the first set of pixels.

[0119] The analog-to-digital converters of the column processing unit convert the pixel signals of the first set of pixels during the periods Strl and the pixel signals of the second set of pixels during the periods Str2. The sensor controller controls the readout in a way that no overlap occurs.

[0120] FIG. 15 and FIG. 16 refer to examples of image sensors, wherein each pixel 110 is included in both the first set 1101 and the second set 1102 of pixels 110. 73844

[0121] 15

[0122] For obtaining the first set of pixels 110, each pixel 110 is read individually. For obtaining the second set of pixels 110, a binning scheme combines two, four or more pixels 110 for a common and simultaneous readout. Binning may include combining, e.g. summing up or averaging the charge produced by multiple pixels 110 and / or the voltage signals produced by multiple pixel circuits into a single pixel signal. Typically, binning is used for improving sensitivity under low light conditions. Here, binning is used independently from light conditions to obtain a higher frame rate.

[0123] FIG. 15 illustrates an example for diagonal binning in pixel circuits with four pixels having the same color filter. For obtaining the second set of pixels, all pixels of a pixel circuit integrate the detector current simultaneously and the accumulated charge is summed up in the sense region.

[0124] FIG. 16 illustrates an example for V2H2 binning of four pixel circuits 100 with four pixels 110 having the same color filter. For obtaining the second set of pixels, the charge accumulated in binned pixel circuits and / or the voltage signals of binned pixel circuits may be combined.

[0125] FIG. 17 illustrates further wiring schemes for signal lines 40. The analog -to-digital converters are implemented in a way that the analog -to-digital converters are shared between the two data streams. Based on time division access, the two data streams may have dedicated analog-to-digital converters for each data stream. Alternatively or in addition, analog-to-digital converters may be shared between the data streams by time-interleaving the pixel signal. Switches and multiplexers can be used to connect the analog-to-digital converters to the respective signal lines 40. Each of the pixel groups 11 include a 2x2 array of pixel circuits, each pixel circuit including four pixels. 64 pixels are assigned 8 signal lines 40 for the first data stream. The 64 pixels may form a unit cell and a plurality of unit cells may be connected to the same set of signal lines 40 in the same way.

[0126] FIG. 18 illustrates signal lines transmitting pixel output signals of pixel groups to column signal circuits with multiplexers and switches.

[0127] FIG. 19 illustrates on-chip processing stages shared by the first data stream and the second data stream. The on-chip processing stages after analog-to-digital conversion and before sensor output are implemented in a way that they are shared between the first data stream Strl and the second data stream Str2. The on-chip processing stages switch the operation mode alternatively. In the illustrated example, the images contained in the second data stream Str2 have 1 / 4th spatial resolution of the images contained in the first data stream Strl . The frame rate of the second data stream Str2 is four times the frame rate of the first data stream Strl . FIG. 20 illustrates the organization of data frames used on-chip by the first data stream Strl and the second data stream Str2 of FIG. 19.

[0128] In FIG. 20, fs marks the frame start, fe the frame end, ph the packet header and pf the packet footer. The digital values for an image contained in the first data stream are transmitted in packets marked “main”. The digital values for the four interleaved images of the second data stream are transmitted in packets marked “sub”. 73844

[0129] 16

[0130] FIG. 21 is a diagram illustrating an example in which the image sensor 90 of FIG. 2 is formed by a stacked CMOS image sensor (CIS) having a two-layer structure with a first chip 910 (radiation receiving chip) and a second chip 920 (processing chip). The radiation receiving chip includes at least the photoelectric conversion elements. For example, the radiation receiving chip may include only the photoelectric conversion elements (photodetectors) of the pixel circuits. The image sensor assembly is formed as one sensor by bonding the radiation receiving chip and the processing chip while electrically bringing contact pads on the radiation receiving chip in contact with corresponding contact pads on the processing chip.

[0131] FIG. 22 is a block diagram depicting an example of schematic configuration of a vehicle control system as an example of a system to which the technology according to an embodiment of the present disclosure can be applied.

[0132] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. In the example depicted in FIG. 22, the vehicle control system 12000 includes a driving system control unit 12010, a body system control unit 12020, an outside-vehicle information detecting unit 12030, an in-vehicle information detecting unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound / image output section 12052, and a vehiclemounted network interface 12053 are illustrated as a functional configuration of the integrated control unit 12050.

[0133] The driving system control unit 12010 controls the operation of devices related to the driving system of the vehicle in accordance with various kinds of programs. For example, the driving system control unit 12010 functions as a control device for a driving force generating device for generating the driving force of the vehicle, such as an internal combustion engine, a driving motor, or the like, a driving force transmitting mechanism for transmitting the driving force to wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating the braking force of the vehicle, and the like.

[0134] The body system control unit 12020 controls the operation of various kinds of devices provided to a vehicle body in accordance with various kinds of programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various kinds of lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like. In this case, radio waves transmitted from a mobile device as an alternative to a key or signals of various kinds of switches can be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals, and controls a door lock device, the power window device, the lamps, or the like of the vehicle.

[0135] The outside-vehicle information detecting unit 12030 detects information about the outside of the vehicle including the vehicle control system 12000. The outside-vehicle information detecting unit 12030 can be connected with an imaging section 12031. The outside-vehicle information detecting unit 12030 makes the imaging section 12031 imaging an image of the outside of the vehicle and receives the imaged image. Based on the received image, the outside-vehicle information detecting unit 12030 may perform processing of 73844

[0136] 17 detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto.

[0137] The imaging section 12031 may be or may include an image sensor according to the embodiments of the present disclosure. The light received by the imaging section 12031 may contain visible light and / or invisible light such as infrared rays or the like.

[0138] The in-vehicle information detecting unit 12040 detects information about the inside of the vehicle and may be or may include an image sensor according to the embodiments of the present disclosure. The in-vehicle information detecting unit 12040 is, for example, connected with a driver state detecting section 12041 that detects the state of a driver. The driver state detecting section 12041, for example, includes a camera that includes the solid-stage imaging device with an image sensor according the embodiments and that is focused on the driver. Based on detection information input from the driver state detecting section 12041, the in-vehicle information detecting unit 12040 may calculate a degree of fatigue of the driver or a degree of concentration of the driver or may determine whether the driver is dozing.

[0139] The microcomputer 12051 can calculate a control target value for the driving force generating device, the steering mechanism, or the braking device based on the information about the inside or outside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in- vehicle information detecting unit 12040 and output a control command to the driving system control unit 12010. For example, the microcomputer 12051 can perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like.

[0140] In addition, the microcomputer 12051 can perform cooperative control intended for automatic driving, which makes the vehicle to travel autonomously without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the information about the outside or inside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040.

[0141] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the outside of the vehicle which information is obtained by the outsidevehicle information detecting unit 12030. For example, the microcomputer 12051 can perform cooperative control intended to prevent a glare by controlling the headlamp so as to change from a high beam to a low beam, for example, in accordance with the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detecting unit 12030.

[0142] The sound / image output section 12052 transmits an output signal of at least one of a sound or an image to an output device capable of visually or audible notifying information to an occupant of the vehicle or the outside of the vehicle. In the example of FIG. 22, an audio speaker 12061, a display section 12062, and an 73844

[0143] 18 instrument panel 12063 are illustrated as the output device. The display section 12062 may, for example, include at least one of an on-board display or a head-up display.

[0144] FIG. 23 is a diagram depicting an example of the installation position of the imaging section 12031, wherein the imaging section 12031 may include imaging sections 12101, 12102, 12103, 12104, and 12105.

[0145] The imaging sections 12101, 12102, 12103, 12104, and 12105 are, for example, disposed at positions on a front nose, side-view mirrors, a rear bumper, and a back door of the vehicle 12100 as well as a position on an upper portion of a windshield within the interior of the vehicle. The imaging section 12101 provided to the front nose and the imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle obtain mainly an image of the front of the vehicle 12100. The imaging sections 12102 and 12103 provided to the side view mirrors obtain mainly an image of the sides of the vehicle 12100. The imaging section 12104 provided to the rear bumper or the back door obtains mainly an image of the rear of the vehicle 12100. The imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle is used mainly to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like.

[0146] Incidentally, FIG. 23 depicts an example of photographing ranges of the imaging sections 12101 to 12104. An imaging range 12111 represents the imaging range of the imaging section 12101 provided to the front nose. Imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging sections 12102 and 12103 provided to the side view mirrors. An imaging range 12114 represents the imaging range of the imaging section 12104 provided to the rear bumper or the back door. A bird's-eye image of the vehicle 12100 as viewed from above is obtained by superimposing image data imaged by the imaging sections 12101 to 12104, for example.

[0147] At least one of the imaging sections 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging sections 12101 to 12104 may be a stereo camera constituted of a plurality of imaging elements, imaging element having pixels for phase difference detection or may include a ToF module including an image sensor according to the embodiments of the present disclosure.

[0148] For example, the microcomputer 12051 can determine a distance to each three-dimensional object within the imaging ranges 12111 to 12114 and a temporal change in the distance (relative speed with respect to the vehicle 12100 on the basis of the distance information obtained from the imaging sections 12101 to 12104, and thereby extract, as a preceding vehicle, a nearest three-dimensional object in particular that is present on a traveling path of the vehicle 12100 and which travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, equal to or more than 0 km / hour). Further, the microcomputer 12051 can set a following distance to be maintained in front of a preceding vehicle in advance and perform automatic brake control (including following stop control), automatic acceleration control (including following start control), or the like. It is thus possible to perform cooperative control intended for automatic driving that makes the vehicle travel autonomously without depending on the operation of the driver or the like. For example, the microcomputer 12051 can classify three-dimensional object data on three-dimensional objects into three-dimensional object data of a two-wheeled vehicle, a standard-sized vehicle, a large-sized vehicle, a pedestrian, a utility pole, and other three-dimensional objects on the basis of the distance information obtained from the imaging sections 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic avoidance of an obstacle. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that the driver of the vehicle 12100 can recognize visually and obstacles that are difficult for the driver of the vehicle 12100 to recognize visually. Then, the microcomputer 12051 determines a collision risk indicating a risk of collision with each obstacle. In a situation in which the collision risk is equal to or higher than a set value and there is thus a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display section 12062 and performs forced deceleration or avoidance steering via the driving system control unit 12010. The microcomputer 12051 can thereby assist in driving to avoid collision.

[0149] At least one of the imaging sections 12101 to 12104 may be an infrared camera that detects infrared rays. The microcomputer 12051 can, for example, recognize a pedestrian by determining whether there is a pedestrian in imaged images of the imaging sections 12101 to 12104. Such recognition of a pedestrian is, for example, performed by a procedure of extracting characteristic points in the imaged images of the imaging sections 12101 to 12104 as infrared cameras and a procedure of determining whether it is the pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object. When the microcomputer 12051 determines that there is a pedestrian in the imaged images of the imaging sections 12101 to 12104, and thus recognizes the pedestrian, the sound / image output section 12052 controls the display section 12062 so that a square contour line for emphasis is displayed so as to be superimposed on the recognized pedestrian. The sound / image output section 12052 may also control the display section 12062 so that an icon or the like representing the pedestrian is displayed at a desired position.

[0150] The example of the vehicle control system to which the technology according to an embodiment of the present disclosure is applicable has been described above. By applying a solid-state imaging device with an image sensor according the embodiments of the present disclosure, reaction time of the system can be enhanced.

[0151] Additionally, embodiments of the present technology are not limited to the above-described embodiments, but various changes can be made within the scope of the present technology without departing from the gist of the present technology.

[0152] The solid-state imaging device according to the present disclosure may be any device used for analyzing and / or processing radiation such as visible light, infrared light, ultraviolet light, and X-rays. For example, a solid-state imaging device according to the embodiments may be any electronic device in the field of traffic, the field of home appliances, the field of medical and healthcare, the field of security, the field of beauty, the field of sports, the field of agriculture, the field of image reproduction or the like. 73844

[0153] 20

[0154] Specifically, in the field of image reproduction, the solid-state imaging device according to the embodiments may be a device for capturing an image to be provided for appreciation, such as a digital camera, a smart phone, or a mobile phone device having a camera function. In the field of traffic, for example, a solid-state imaging device including a solid-state imaging device with an image sensor according to the embodiments may be integrated in an in-vehicle sensor that captures the front, rear, peripheries, an interior of the vehicle, etc. for safe driving such as automatic stop, recognition of a state of a driver, or the like, in a monitoring camera that monitors traveling vehicles and roads, or in a distance measuring sensor that measures a distance between vehicles or the like.

[0155] In the field of home appliances, the solid-state imaging device with an image sensor according to the embodiments may be integrated in any type of sensor that can be used in devices provided for home appliances such as TV receivers, refrigerators, and air conditioners to capture gestures of users and perform device operations according to the gestures. Accordingly, the solid-state imaging device with an image sensor according to the embodiments may be integrated in home appliances such as TV receivers, refrigerators, and air conditioners and / or in devices controlling the home appliances. Furthermore, in the field of medical and healthcare, the solid-state imaging device with an image sensor according to the embodiments may be integrated in any type of sensor, e.g., a solid-state image device, provided for use in medical and healthcare, such as an endoscope or a device that performs angiography by receiving infrared light.

[0156] In the field of security, the solid-state imaging device with an image sensor according to the embodiments can be integrated in a device provided for use in security, such as a monitoring camera for crime prevention or a camera for person authentication use. Furthermore, in the field of beauty, a solid-state imaging device according to the embodiments can be used in a device provided for use in beauty, such as a skin measuring instrument that captures skin or a microscope that captures a probe. In the field of sports, a solid-state imaging device according to the embodiments can be integrated in a device provided for use in sports, such as an action camera or a wearable camera for sport use or the like. Furthermore, in the field of agriculture, the solid-state imaging device can be used in a device provided for use in agriculture, such as a camera for monitoring the condition of fields and crops.

[0157] The present technology can also be configured as described below: [1] An image sensor, including: a pixel array (10) including pixels (110) configured to accumulate charge in response to incident radiation, sense regions (130), transfer transistors (120) operatively connected between the pixels (110) and the sense regions (130), and pixel readout circuits (140), each pixel readout circuit (140) being operatively connected to a different one of the sense regions (130); signal lines (40), wherein the pixel readout circuits (140) are configured to output pixel signals on the signal lines (40), and wherein a voltage of a pixel signal is a function of charge stored in the sense region (130) operatively connected to the pixel readout circuit (140); and a sensor controller (80) configured to control the transfer transistors (120) and the pixel readout circuits (140) to output, at a low first frame rate, a first sequence of pixel signals to the signal lines (40) based on a first set (1101) of the pixels (110) and, at a high second frame rate, a second sequence of pixel signals to the signal lines (40) based on a predefined second set (1102) of the pixels (110), wherein a complete readout 73844

[0158] 21 of the second set of the pixels (110) is controlled to generate less pixel signals for further processing than a readout of the first set (1101) of the pixels (110).

[0159] [2] The image sensor according to [1], further including: a column processing unit (20) configured to convert the pixel signals into digital pixel values, wherein for a complete readout of the second set (1102) of pixels (110) less pixel signals are converted into digital pixel values than for a complete readout of the first set (1101) of pixels (110).

[0160] [3] The image sensor according to any of [1] and [2], wherein each pixel (110) is included in the first set (1101), and wherein the second set (1102) includes less pixels (110) than the first set (1101).

[0161] [4] The image sensor according to any of [1] to [3], wherein each pixel (110) is included either in the first set (1101) or the second set (1102) of pixels (110), and wherein the second set (1102) of pixels (110) includes less pixels (110) than the first set (1101).

[0162] [5] The image sensor according to any of [1] to [4], further including: sets of transfer signal lines (51, 52), wherein each set of transfer signal lines (51, 52) is operatively connected to a different row of pixel groups (11), wherein each set of transfer signal lines (51, 52) includes at least one first transfer signal line (51) operatively connected to the transfer transistors (120) operatively connected to the first set (1101) of pixels and at least one second transfer signal line (52) operatively connected to the transfer transistors (120) operatively connected to the second set (1102) of pixels.

[0163] [6] The image sensor according to any of [1] to [5], further including: sets of select signal lines (53, 54), wherein each set of select signal lines (53, 54) is operatively connected to a different row of pixel groups (11), wherein each set of select signal lines (53, 54) includes at least one first select signal line (53) operatively connected to the pixel readout circuits (140) operatively connected to the first set (1101) of pixels and at least one second select signal line (54) operatively connected to the pixel readout circuits (140) operatively connected to the second set (1102) of pixels.

[0164] [7] The image sensor according to any of [1] to [6], wherein the signal lines (40) are grouped into sets of signal lines (40), wherein each set of signal lines (40) is operatively connected to a different column of pixel groups (11), wherein each set of signal lines (40) includes at least one first output signal line (41) operatively connected to the pixel readout circuits (140) of the first set of pixels (110) in the column of pixel groups (11) and at least one second output signal line (42) operatively connected to the pixel readout circuits (140) of the second set of pixels (110) in the column of pixel groups (11).

[0165] [8] The image sensor according to [7], wherein more pixel readout circuits (140) are operatively connected to a first output signal line (41) than to a second output signal line (42).

[0166] [9] The image sensor according to any of [7] and [8], further including: shared column signal circuits (201) operatively connected with the first and second signal lines (41, 42), wherein each shared column signal circuit (201) is configured to convert pixel signals transmitted on one of the second signal lines (42) in first 73844

[0167] 22 conversion periods and convert pixel signals transmitted on one of the first signal lines (41) in second conversion periods.

[0168]

[0010] The image sensor according to any of [7] to [9], further including: multiplexers (210), wherein each multiplexer (210) is configured to pass the pixel signal transmitted on one of the first signal lines (41) in first sampling periods and pass pixel signals transmitted on one of the second signal lines (42) in second sampling periods.

[0169]

[0011] The image sensor according to

[0010] , further including: shared analog-to-digital converters (221), wherein each shared analog-to-digital converter (221) is configured to convert, in first conversion periods, pixel signals received in the first sampling periods from one of the multiplexers (210), and convert, in second conversion periods, pixel signals received from the multiplexer (210) in the second sampling periods.

[0170]

[0012] The image sensor according to any of

[0010] and

[0011] , wherein the sensor controller (80) is operatively connected to the multiplexers (210) and is configured to control the multiplexers (210) to pass the pixel signals transmitted on one of the first signal lines (41) in first sampling periods and pass pixel signals transmitted on one of the second signal lines (42) in second sampling periods.

[0171]

[0013] The image sensor according to any of [1] to

[0012] , wherein each pixel (110) is included in both the first set (1101) and the second set (1102) of pixels (110).

[0172]

[0014] The image sensor according to any of [1] to

[0013] , wherein the sensor controller (80) is configured to change between a dual data stream mode and a single data stream mode, wherein in the dual data stream mode the sensor controller (80) is configured to control the transfer transistors (120) and the pixel readout circuits (140) to output the first sequence of pixel signals to the signal lines (40) based on the first set of the pixels and output the second sequence of pixel signals to the signal lines (40) based on the second set of the pixels (110), and wherein in the single data stream mode the sensor controller (80) is configured to control the transfer transistors (120) and the pixel readout circuits (140) to output a single sequence of pixel signals to the signal lines (40) based on both the first and second sets of the pixels.

[0173]

[0015] The image sensor according to any of [1] to

[0014] , wherein the pixel array (10) includes a plurality of identically configured unit cells (12), each unit cell (12) including an even number of pixel groups (11).

Claims

7384423CLAIMS1. An image sensor, comprising: a pixel array comprising pixels configured to accumulate charge in response to incident radiation, sense regions, transfer transistors operatively connected between the pixels and the sense regions, and pixel readout circuits, each pixel readout circuit being operatively connected to a different one of the sense regions; signal lines, wherein the pixel readout circuits are configured to output pixel signals on the signal lines, and wherein a voltage of a pixel signal is a function of charge stored in the sense region operatively connected to the pixel readout circuit; and a sensor controller configured to control the transfer transistors and the pixel readout circuits to output, at a low first frame rate, a first sequence of pixel signals to the signal lines based on a first set of the pixels and, at a high second frame rate, a second sequence of pixel signals to the signal lines based on a predefined second set of the pixels, wherein a complete readout of the second set of the pixels is controlled to generate less pixel signals for further processing than a readout of the first set of the pixels.

2. The image sensor according to claim 1, further comprising: a column processing unit configured to convert the pixel signals into digital pixel values, wherein for a complete readout of the second set of pixels less pixel signals are converted into digital pixel values than for a complete readout of the first set of pixels.

3. The image sensor according to claim 1, wherein each pixel is included in the first set, and wherein the second set includes less pixels than the first set.

4. The image sensor according to claim 1, wherein each pixel is included either in the first set or the second set of pixels, and wherein the second set of pixels includes less pixels than the first set.

5. The image sensor according to claim 1, further comprising: sets of transfer signal lines, wherein each set of transfer signal lines is operatively connected to a different row of pixel groups, wherein each set of transfer signal lines comprises at least one first transfer signal line operatively connected to the transfer transistors operatively connected to the first set of pixels and at least one second transfer signal line operatively connected to the transfer transistors operatively connected to the second set of pixels.

6. The image sensor according to claim 1, further comprising: sets of select signal lines, wherein each set of select signal lines is operatively connected to a different row of pixel groups, wherein each set of select signal lines comprises at least one first select signal line operatively connected to the pixel readout circuits operatively connected to the first7384424 set of pixels and at least one second select signal line operatively connected to the pixel readout circuits operatively connected to the second set of pixels.

7. The image sensor according to claim 1, wherein the signal lines are grouped into sets of signal lines, wherein each set of signal lines is operatively connected to a different column of pixel groups, wherein each set of signal lines comprises at least one first output signal line operatively connected to the pixel readout circuits of the first set of pixels in the column of pixel groups and at least one second output signal line operatively connected to the pixel readout circuits of the second set of pixels in the column of pixel groups.

8. The image sensor according to claim 7, wherein more pixel readout circuits are operatively connected to a first output signal line than to a second output signal line.

9. The image sensor according to claim 7, further comprising: shared column signal circuits operatively connected with the first and second signal lines, wherein each shared column signal circuit is configured to convert pixel signals transmitted on one of the second signal lines in first conversion periods and convert pixel signals transmitted on one of the first signal lines in second conversion periods.

10. The image sensor according to claim 7, further comprising: multiplexers, wherein each multiplexer is configured to pass the pixel signal transmitted on one of the first signal lines in first sampling periods and pass pixel signals transmitted on one of the second signal lines in second sampling periods.

11. The image sensor according to claim 10, further comprising: shared analog-to-digital converters, wherein each shared analog-to-digital converter is configured to convert, in first conversion periods, pixel signals received in the first sampling periods from one of the multiplexers, and convert, in second conversion periods, pixel signals received from the multiplexer in the second sampling periods.

12. The image sensor according to claim 10, wherein the sensor controller is operatively connected to the multiplexers and is configured to control the multiplexers to pass the pixel signals transmitted on one of the first signal lines in first sampling periods and pass pixel signals transmitted on one of the second signal lines in second sampling periods.

13. The image sensor according to claim 1, wherein each pixel is included in both the first set and the second set of pixels.

14. The image sensor according to claim 1,wherein the sensor controller is configured to change between a dual data stream mode and a single data stream mode, wherein in the dual data stream mode the sensor controller is configured to control the transfer transistors and the pixel readout circuits to output the first sequence of pixel signals to the signal lines based on the first set of the pixels and output the second sequence of pixel signals to the signal lines based on the second set of the pixels, and wherein in the single data stream mode the sensor controller is configured to control the transfer transistors and the pixel readout circuits to output a single sequence of pixel signals to the signal lines based on both the first and second sets of the pixels.

15. The image sensor according to claim 1, wherein the pixel array comprises a plurality of identically configured unit cells, each unit cell comprising an even number of pixel groups.

Images