Sensing System

Abstract

This system for capturing image data measures the distance to an object without having a distance measurement sensor added thereto.　In this sensing system, a light emission unit emits invisible light in synchronization with a prescribed light emission control signal. An invisible light pixel subjects reflected invisible light to photoelectric conversion and thereby generates a pulse signal as an invisible light pulse signal. A visible light pixel subjects visible light to photoelectric conversion and thereby generates a pulse signal as a visible light pulse signal. A counting unit carries out processing for counting the number of visible light pulse signal pulses and processing for counting the number of invisible light pulse signal pulses in synchronization with the light emission control signal.

Description

[Technical field]

[0001] The present technology relates to a sensing system, and more particularly to a sensing system that counts the number of pulses generated by a pixel in response to the incidence of a photon. [Background technology]

[0002] In recent years, a device called SPAD (Single Photon Avalanche Diode) has been developed and researched to capture very weak optical signals and realize optical communication, distance measurement, photon counting, etc. This SPAD is an avalanche photodiode with high sensitivity that can detect a single photon. For example, a solid-state imaging element has been proposed in which pixels that generate pulse signals using SPADs and a counter that counts the number of these pulse signals within an exposure period are arranged (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 150785 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technology, a high-sensitivity SPAD is used to detect weak light, thereby improving image quality when capturing images in a dark environment. However, the above-mentioned solid-state imaging element cannot measure the distance to an object in the captured image. If a distance measuring sensor using infrared rays or a laser is added to measure the distance, the power consumption and cost of the system will increase, which is not desirable.

[0005] This technology was developed in light of these circumstances, and aims to measure the distance to an object in a system that captures image data without adding a distance measuring sensor. [Means for solving the problem]

[0006] The present technology has been made to solve the above-mentioned problems, and a first aspect thereof is a sensing system including a light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal, invisible light pixels that perform photoelectric conversion of reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal, visible light pixels that perform photoelectric conversion of visible light to generate a pulse signal as a visible light pulse signal, and a counting unit that performs a process of counting the number of the visible light pulse signals and a process of counting the number of the invisible light pulse signals in synchronization with the light-emission control signal, thereby providing an effect of performing image capture of image data and distance measurement.

[0007] In addition, in this first aspect, the visible light pixel may include first, second and third visible light pixels which photoelectrically convert different visible light, the invisible light pixel may include first, second, third and fourth invisible light pixels which are associated with enable signals having different phase differences with respect to the light emission control signal, the first, second, third and fourth invisible light pixels may be arranged adjacent to each other, and the first, second and third visible light pixels may be arranged in the vicinity of the first invisible light pixel, thereby providing an effect that the number of pulses of each of the first, second and third visible light pixels and the first invisible light pixel are counted.

[0008] In addition, in the first aspect, the counting unit may include a counter that counts the number of the visible light pulse signals of each of the first, second and third visible light pixels and the number of the invisible light pulse signals in a predetermined order, thereby providing an effect that the counter is shared by four pixels.

[0009] In addition, in this first aspect, the counting unit may include a first counter that counts the number of the visible light pulse signals of the first visible light pixel, a second counter that counts the number of the visible light pulse signals of the second visible light pixel, a third counter that counts the number of the visible light pulse signals of the third visible light pixel, and a fourth counter that counts the number of the invisible light pulse signals in synchronization with the light emission control signal, thereby providing an effect that the number of pulses of each of the four pixels is counted in parallel.

[0010] In addition, in this first aspect, the visible light pixel may include first, second and third visible light pixels which photoelectrically convert the same visible light, the invisible light pixel may include first, second, third and fourth invisible light pixels which are associated with enable signals having different phase differences with respect to the light emission control signal, and the first, second and third visible light pixels may be disposed in the vicinity of the first invisible light pixel, thereby providing an effect that the number of pulses of each of the first, second and third visible light pixels and the first invisible light pixel are counted.

[0011] In addition, in this first aspect, the counting unit may include a selector that sequentially selects the visible light pulse signals of the first, second and third visible light pixels as input signals, a first counter that counts the number of the input signals, and a second counter that counts the number of the invisible light pulse signals in synchronization with the light emission control signal, thereby providing an effect that the number of pulses is counted in parallel for each of the visible light pixels and the invisible light pixels.

[0012] In addition, in this first aspect, the counting unit may include a logical OR gate that outputs a logical OR of the invisible light pulse signals of the first, second and third visible light pixels, a selector that selects as an input signal any one of the invisible light pulse signals of the first, second and third visible light pixels, the logical OR and the visible light pulse signal, and a counter that counts the number of the input signals, thereby providing an effect of pixel addition of four pixels.

[0013] In addition, in the first aspect, the visible light pixels include R (Red) pixels, G (Green) pixels, and B (Blue) pixels, and the invisible light pixels are arranged at the positions of the G pixels in a Bayer array, thereby providing an effect of simplifying demosaic processing.

[0014] In addition, in the first aspect, the invisible light pixels may include a plurality of invisible light pixels associated with enable signals having different phase differences with respect to the light emission control signal, and the plurality of invisible light pixels may be arranged in a predetermined direction, thereby providing an effect of increasing the number of invisible light pixels in the predetermined direction.

[0015] In addition, in the first aspect, the visible light pixels may be inserted between each of the plurality of invisible light pixels, thereby providing an effect of reducing the number of pixels to be interpolated.

[0016] In addition, in the first aspect, the visible light pixel may include first, second, third and fourth visible light pixels arranged adjacent to each other, the invisible light pixel may include first, second, third and fourth invisible light pixels arranged adjacent to each other, and the first, second, third and fourth visible light pixels may photoelectrically convert visible light different from each other, thereby providing an effect of increasing the number of ranging points.

[0017] In addition, in the first aspect, the counting unit may include a plurality of counters that count the number of the invisible light pulse signals in synchronization with enable signals having different phase differences with respect to the light emission control signal, thereby providing an effect that the number of pulses is counted in parallel for a plurality of phases.

[0018] In the first aspect, the counting unit may include a selector that selects one of the visible light pulse signals of the first, second, and third visible light pixels as an input signal, and a counter that counts the number of the input signals, thereby providing an effect that the counter is shared by a plurality of pixels.

[0019] In addition, in this first aspect, the counting unit may include a first counter that counts the invisible light pulse signals in synchronization with a first enable signal whose phase difference with respect to the light emission control signal is set to 0 degrees or 180 degrees, and a second counter that counts the invisible light pulse signals in synchronization with a second enable signal whose phase difference with respect to the light emission control signal is set to 90 degrees or 270 degrees, thereby providing the effect of reducing the number of counters.

[0020] In addition, in this first aspect, the counting unit may include a logic circuit that outputs a logical sum of two or more of the invisible light pulse signals of the first, second, third and fourth invisible light pixels, a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical sum and outputs it as an input signal, a fifth counter that counts the number of the input signals, a sixth counter that counts the number of the invisible light pulse signals of the second invisible light pixel, a seventh counter that counts the number of the invisible light pulse signals of the third invisible light pixel, and an eighth counter that counts the number of the invisible light pulse signals of the fourth invisible light pixel. This brings about an effect that the number of pulses is counted in parallel for the first to fourth invisible light pixels.

[0021] In addition, in this first aspect, the counting unit may include a logic circuit that outputs a logical product of a logical sum of the invisible light pulse signals of each of the first, second, third and fourth invisible light pixels and each of first and second enable signals having different phase differences with respect to the light emission synchronization signal, a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical product and outputs it as an input signal, a fifth counter that counts the number of the input signals, a sixth counter that counts the number of the invisible light pulse signals of the second invisible light pixel, a seventh counter that counts the number of the invisible light pulse signals of the third invisible light pixel, and an eighth counter that counts the number of the invisible light pulse signals of the fourth invisible light pixel. This brings about the effect of adding up a plurality of pixels.

[0022] In addition, in this first aspect, the counting unit may include a logic circuit that outputs a logical product of a logical sum of the invisible light pulse signals of each of the first, second, third and fourth invisible light pixels and each of first and second enable signals having different phase differences with respect to the light emission synchronization signal, a selector that selects and outputs as an input signal either the invisible light pulse signal of the first invisible light pixel or the logical product corresponding to the first enable signal, a switch that outputs the logical product corresponding to the first enable signal in accordance with a predetermined control signal, a fifth counter that counts the number of the input signals, and a sixth counter that counts based on the logical product output from the second switch, thereby providing the effect of reducing the number of counters.

[0023] In addition, in the first aspect, the visible light pixel may include first, second, third and fourth visible light pixels arranged adjacent to each other, the invisible light pixel may include first, second, third and fourth invisible light pixels arranged adjacent to each other, and the first, second, third and fourth visible light pixels may photoelectrically convert the same visible light, thereby providing an effect of counting the number of pulses of the first, second, third and fourth visible light pixels of the same color.

[0024] In addition, in this first aspect, the first and second visible light pixels receive one of a pair of pupil-divided incident light beams, and the third and fourth visible light pixels receive the other of the pair of pupil-divided incident light beams, and the counter unit includes a first logical OR gate that outputs a logical OR of the visible light pulse signals of the first and second visible light pixels as a first logical OR, and a first logical OR gate that selects either the first logical OR or the visible light pulse signal of the first and second visible light pixels to output the first input light beam. The focus detection device may further include a first selector for outputting a first input signal as a first input signal, a second OR gate for outputting a logical OR of the visible light pulse signals of the third and fourth visible light pixels as a second logical OR, a second selector for selecting either the second logical OR or the visible light pulse signals of the third and fourth visible light pixels and outputting the selected signal as a second input signal, a first counter for counting the number of the first input signals, and a second counter for counting the number of the second input signals. This provides the effect of detecting the focus by an image plane phase difference method.

[0025] In addition, in this first aspect, the counter unit may further include a third logic gate that outputs a logical sum of the first logical sum and the second logical sum to the first selector as a third logical sum, and the first selector may select any one of the third logical sum, the first logical sum, and the visible light pulse signal of each of the first and second visible light pixels. This provides the effect of adding up a plurality of pixels. [Brief description of the drawings]

[0026] [Figure 1] 1 is a block diagram showing a configuration example of a sensing system according to a first embodiment of the present technology. [Diagram 2] 1 is a diagram showing an example of a layered structure of a solid-state imaging element according to a first embodiment of the present technology. [Diagram 3] 1 is a block diagram showing a configuration example of a solid-state imaging element according to a first embodiment of the present technology. [Figure 4]1 is an example of a plan view of a pixel array unit according to a first embodiment of the present technology. [Diagram 5] 1 is a block diagram showing an example of a configuration of a pixel block according to a first embodiment of the present technology. [Figure 6] 1 is a circuit diagram showing a configuration example of a pixel according to a first embodiment of the present technology. [Figure 7] 1 is a circuit diagram showing a configuration example of a circuit block according to a first embodiment of the present technology. [Figure 8] 4 is a diagram for explaining an operation of a counter according to the first embodiment of the present technology. FIG. [Figure 9] 4 is a timing chart showing an example of an operation in a distance measurement mode of the solid-state imaging element according to the first embodiment of the present technology. [Figure 10] 4 is a timing chart showing an example of an operation in an imaging mode of the solid-state imaging element according to the first embodiment of the present technology. [Figure 11] 1 is an example of an overall view of a sensing system according to a first embodiment of the present technology. [Figure 12] 4 is a flowchart showing an example of an operation of the sensing system according to the first embodiment of the present technology. [Figure 13] 11 is a block diagram showing an example of a configuration of a pixel block in a modified example of the first embodiment of the present technology. FIG. [Figure 14] 11 is a circuit diagram showing a configuration example of a circuit block in a modified example of the first embodiment of the present technology. FIG. [Figure 15] 13 is a diagram for explaining an operation of a counter in a modified example of the first embodiment of the present technology. FIG. [Figure 16] 13 is an example of a plan view of a pixel array unit according to a second embodiment of the present technology. [Figure 17] 13 is a block diagram showing an example of the configuration of a pixel block according to a second embodiment of the present technology. FIG. [Figure 18] FIG. 11 is a circuit diagram showing a configuration example of a circuit block according to a second embodiment of the present technology. [Figure 19]13 is a circuit diagram showing a configuration example of a circuit block in a modified example of the second embodiment of the present technology. FIG. [Figure 20] 13 is an example of a plan view of a pixel array unit according to a third embodiment of the present technology. [Figure 21] 13 is an example of a plan view of a pixel array unit according to a fourth embodiment of the present technology. [Figure 22] 13 is an example of a plan view of a pixel array unit in a modified example of the fourth embodiment of the present technology. [Diagram 23] 13 is an example of a plan view of a pixel array unit according to a fifth embodiment of the present technology. [Figure 24] FIG. 13 is a block diagram showing an example configuration of a pixel block in which visible light pixels are arranged according to a fifth embodiment of the present technology. [Diagram 25] FIG. 13 is a block diagram showing an example configuration of a pixel block in which IR (Infra Red) pixels are arranged according to a fifth embodiment of the present technology. [Figure 26] FIG. 13 is a circuit diagram showing a configuration example of a circuit block according to a fifth embodiment of the present technology. [Figure 27] 13 is a diagram for explaining an operation of a counter according to the fifth embodiment of the present technology. FIG. [Figure 28] FIG. 23 is a block diagram showing an example configuration of a pixel block in which visible light pixels are arranged in a first modified example of the fifth embodiment of the present technology. [Figure 29] FIG. 23 is a block diagram showing an example configuration of a pixel block in which IR pixels are arranged according to a second modified example of the fifth embodiment of the present technology. [Diagram 30] 23 is a timing chart showing an example of an operation in a distance measurement mode of a solid-state imaging element in a second modified example of the fifth embodiment of the present technology. [Diagram 31] 13 is an example of a plan view of a pixel array unit in a third modified example of the fifth embodiment of the present technology. [Diagram 32] FIG. 23 is a circuit diagram showing a configuration example of a circuit block in a third modified example of the fifth embodiment of the present technology. [Diagram 33]23 is a diagram for explaining an operation of a pixel driving section in a third modified example of the fifth embodiment of the present technology. FIG. [Diagram 34] FIG. 23 is a circuit diagram showing a configuration example of a circuit block in a fourth modified example of the fifth embodiment of the present technology. [Diagram 35] FIG. 23 is a diagram for explaining an operation of a counter in a fourth modified example of the fifth embodiment of the present technology. [Diagram 36] FIG. 13 is a block diagram showing an example configuration of a pixel block in which IR pixels are arranged, in a fifth modified example of the fifth embodiment of the present technology. [Figure 37] FIG. 13 is a circuit diagram showing a configuration example of a circuit block in a fifth modified example of the fifth embodiment of the present technology. [Figure 38] FIG. 13 is a diagram for explaining an operation of a counter in a fifth modified example of the fifth embodiment of the present technology. [Figure 39] 23 is an example of a plan view of a pixel array unit according to a sixth embodiment of the present technology. [Diagram 40] FIG. 23 is a block diagram showing an example configuration of a pixel block in which visible light pixels are arranged according to a sixth embodiment of the present technology. [Diagram 41] FIG. 23 is a circuit diagram showing a configuration example of a circuit block according to a sixth embodiment of the present technology. [Diagram 42] FIG. 23 is a circuit diagram showing a configuration example of a circuit block in a modified example of the sixth embodiment of the present technology. [Diagram 43] 23 is an example of a plan view of a pixel array unit according to a seventh embodiment of the present technology. [Diagram 44] 23 is an example of a plan view of a pixel array unit according to an eighth embodiment of the present technology. [Diagram 45] 1 is a block diagram showing a schematic configuration example of a vehicle control system; [Figure 46] FIG. 4 is an explanatory diagram showing an example of an installation position of an imaging unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, modes for carrying out the present technology (hereinafter, referred to as embodiments) will be described. The description will be made in the following order. 1. First embodiment (example of counting the number of pulses of visible light pixels and IR pixels) 2. Second embodiment (example of counting the number of pulses of three visible light pixels and an IR pixel of the same color) 3. Third embodiment (example of inserting visible light pixels into the positions of G pixels in a Bayer array and counting the number of pulses) 4. Fourth embodiment (example in which IR pixels are arranged in a predetermined direction and the number of pulses of visible light pixels and IR pixels are counted) 5. Fifth embodiment (example of counting the number of pulses of four visible light pixels and four IR pixels in a Bayer array) 6. Sixth embodiment (example of counting the number of pulses of four visible light pixels and four IR pixels in a quadra array) 7. Seventh embodiment (example of counting the number of pulses of 16 visible light pixels and 16 IR pixels in a Bayer array) 8. Eighth embodiment (example of counting the number of pulses of 16 visible light pixels and 16 IR pixels in a quadra array) 9. Application examples for mobile objects

[0028] <1. First embodiment> [Example of sensing system configuration] 1 is a block diagram showing a configuration example of a sensing system 100 according to a first embodiment of the present technology. The sensing system 100 is for capturing image data and measuring distances. The sensing system 100 includes a light emitting unit 110, a driver 120, a controller 130, a solid-state imaging element 200, a processor 140, and an application processor 150.

[0029] Each of the elements in the sensing system 100 may be arranged in one electronic device or may be distributed across multiple devices. In the case of distributing the elements across multiple devices, for example, the light emitting unit 110, the driver 120, the controller 130, the solid-state imaging element 200, and the processor 140 are arranged in an imaging device, and the application processor 150 is arranged in an image processing device.

[0030] The light emitting unit 110 emits light and irradiates irradiation light in accordance with a light emission control signal LCLK from the driver 120. For example, invisible light (near-infrared light, etc.) is used as the irradiation light.

[0031] The driver 120 generates a predetermined periodic signal as a light emission control signal LCLK under the control of the controller 130 , and supplies the signal to the light emitting unit 110 .

[0032] The controller 130 operates the driver 120 and the processor 140 in synchronization. Here, the sensing system has a plurality of modes set, including a distance measurement mode for measuring the distance to an object and an imaging mode for capturing image data. In the distance measurement mode, the controller 130 causes the driver 120 to generate a light emission control signal LCLK, and causes the processor 140 to generate a signal identical to the light emission control signal LCLK as a light emission control signal LCLK'. On the other hand, in the imaging mode, the controller 130 stops the driver 120 and causes the processor 140 to generate a vertical synchronization signal VSYNC.

[0033] Here, the frequency of the vertical synchronization signal VSYNC is, for example, 30 Hertz (Hz) or 60 Hertz (Hz). On the other hand, the frequency of the light emission control signal LCLK is higher than that of the vertical synchronization signal VSYNC, for example, 10 to 20 Megahertz (MHz).

[0034] The processor 140 controls the solid-state imaging element 200 and the application processor 150. In the distance measurement mode, the processor 140 generates a light emission control signal LCLK' and supplies it to the solid-state imaging element 200, and receives a depth map from the solid-state imaging element 200. On the other hand, in the imaging mode, the processor 140 generates a vertical synchronization signal VSYNC and supplies it to the solid-state imaging element 200, and receives image data from the solid-state imaging element 200. Then, the processor 140 supplies the depth map and image data to the application processor 150.

[0035] The application processor 150 performs predetermined processing such as image recognition processing based on image data and a depth map.

[0036] The solid-state imaging element 200 generates image data or a depth map by photoelectric conversion. In the distance measurement mode, the solid-state imaging element 200 photoelectrically converts reflected light relative to the irradiated light in synchronization with the light emission control signal LCLK' to generate a depth map. On the other hand, in the imaging mode, the solid-state imaging element 200 photoelectrically converts incident light in synchronization with the vertical synchronization signal VSYNC to generate image data. The solid-state imaging element 200 supplies the image data and the depth map to the processor 140.

[0037] Note that the solid-state imaging device 200 may have some or all of the functions of the processor 140 and the application processor 150 .

[0038] [Example of the configuration of a solid-state imaging element] 2 is a diagram showing an example of a stacked structure of a solid-state imaging element 200 in the first embodiment of the present technology. This solid-state imaging element 200 includes a circuit chip 202 and a pixel chip 201 stacked on the circuit chip 202. These chips are electrically connected through a connection portion such as a via. In addition to the via, the chips can also be connected by Cu-Cu bonding or bumps. They can also be connected by other methods (such as magnetic coupling). Although two chips are stacked, three or more layers can also be stacked.

[0039] 3 is a block diagram showing an example of a configuration of a solid-state imaging element 200 according to the first embodiment of the present technology. The solid-state imaging element 200 includes a pixel driving section 210, a vertical scanning circuit 220, a pixel array section 230, a column buffer 240, a signal processing circuit 250, and an output section 260. In the pixel array section 230, a plurality of pixels are arranged in a two-dimensional lattice pattern.

[0040] The pixel driving section 210 drives the pixels in the pixel array section 230 in synchronization with the light emission control signal LCLK', and causes the pixels to count the number of pulses.

[0041] The vertical scanning circuit 220 sequentially selects rows of pixels in synchronization with a vertical synchronization signal VSYNC, and outputs the count values ​​to a column buffer 240 .

[0042] The column buffer 240 holds the count value for each pixel.

[0043] The signal processing circuit 250 performs a predetermined signal processing on the data in which the count values ​​are arranged. For example, in the distance measurement mode, the signal processing circuit 250 obtains the distances for a plurality of distance measurement points based on the count values, and generates a depth map in which the distance data are arranged. In the imaging mode, the signal processing circuit 250 generates image data in which the count values ​​for each pixel are arranged as pixel data, and performs various image processing on the image data. Then, the signal processing circuit 250 supplies the depth map and image data to the processor 140.

[0044] 4 is an example of a plan view of the pixel array section 230 according to the first embodiment of the present technology. The pixel array section 230 is divided into a plurality of pixel blocks such as pixel blocks 301 to 304. Each pixel block has four pixels arranged in two rows and two columns.

[0045] In the figure, for example, an R (Red) pixel 315, a G (Green) pixel 310, a B (Blue) pixel 316, and an IR pixel 321 are arranged in an upper left pixel block 301. An R pixel, a G pixel, a B pixel, and an IR pixel 322 are arranged in an upper right pixel block 302. An R pixel, a G pixel, a B pixel, and an IR pixel 323 are arranged in a lower left pixel block 303. An R pixel, a G pixel, a B pixel, and an IR pixel 324 are arranged in a lower right pixel block 304.

[0046] Furthermore, in the upper left pixel block 301, the IR pixel 321 is arranged at the bottom right, and in the upper right pixel block 302, the IR pixel 322 is arranged at the bottom left. In the lower left pixel block 303, the IR pixel 323 is arranged at the top right, and in the lower right pixel block 304, the IR pixel 324 is arranged at the top left. With this arrangement, the IR pixels 321 to 324 are arranged adjacent to each other in 2 rows and 2 columns.

[0047] The R pixel 315 photoelectrically converts red visible light to generate a pulse signal. The G pixel 310 photoelectrically converts green visible light to generate a pulse signal. The B pixel 316 photoelectrically converts blue visible light to generate a pulse signal.

[0048] The IR pixels 321 to 324 photoelectrically convert reflected light of irradiated light (i.e., infrared light) to generate a pulse signal. Of these, the IR pixel 321 is a pixel that counts the number of its pulse signals in synchronization with an enable signal that has a phase difference of 0 degrees with respect to the light emission control signal LCLK. The IR pixel 322 is a pixel that counts the number of its pulse signals in synchronization with an enable signal that has a phase difference of 90 degrees with respect to the light emission control signal LCLK. The IR pixels 323 and 324 are pixels that count the number of its pulse signals in synchronization with enable signals that have phase differences of 180 degrees and 270 degrees with respect to the light emission control signal LCLK.

[0049] Each of the pixel blocks 301 to 304 further includes circuits and elements such as a counter, in addition to pixels such as a G pixel 310. In the figure, circuits and elements other than the pixels are omitted.

[0050] [Pixel block configuration example] 5 is a block diagram showing an example of a configuration of a pixel block 301 according to the first embodiment of the present technology. The pixel block 301 includes an IR pixel 321, an R pixel 315, a G pixel 310, and a B pixel 316, as well as a counting unit 330 and a switch 351. In the counting unit 330, a circuit block 370 and a counter 341 are arranged.

[0051] The IR pixel 321 photoelectrically converts reflected light of irradiated light (i.e., infrared light) to generate a pulse signal Pir. The R pixel 315 photoelectrically converts red visible light to generate a pulse signal Pr. The G pixel 310 photoelectrically converts green visible light to generate a pulse signal Pg. The B pixel 316 photoelectrically converts blue visible light to generate a pulse signal Pb. These four pixels output the generated pulse signals to the circuit block 370.

[0052] The circuit block 370 controls the output destination of the pulse signals Pir, Pr, Pg, and Pb. In the imaging mode, the circuit block 370 sequentially selects the pulse signals Pir, Pr, Pg, and Pb, and outputs them to the counter 341 as an input signal CIN. On the other hand, in the distance measurement mode, the circuit block 370 outputs the pulse signal Pir to the counter 341 as an input signal CIN in synchronization with an enable signal EN1 from the pixel driving unit 210. In the circuit block 370, the switching of the pulse signal is performed in accordance with a control signal CTRL from the pixel driving unit 210.

[0053] The counter 341 counts the number of input signals CIN that have been input. The counter 341 outputs the count value as CNT to the switch 351. A reset signal RST from the vertical scanning circuit 220 is also input to the counter 341. The count value of the counter 341 is initialized by this reset signal RST. Note that the pixel driving unit 210 can also supply the reset signal RST instead of the vertical scanning circuit 220.

[0054] The switch 351 outputs the count value CNT to the column buffer 240 via the vertical signal line 309 in accordance with a selection signal SEL from the vertical scanning circuit 220 .

[0055] The pixel blocks 302 to 304 are each configured similarly to the pixel block 301. However, an enable signal EN2 is supplied to the pixel block 302. Pixel blocks 303 and 304 are supplied with enable signals EN3 and EN4.

[0056] Here, the enable signal EN1 is the same signal as the light emission control signal LCLK. The enable signal EN2 is a signal whose phase is shifted by 90 degrees from the light emission control signal LCLK. The enable signal EN3 is a signal whose phase is shifted by 180 degrees from the light emission control signal LCLK. The enable signal EN4 is a signal whose phase is shifted by 270 degrees from the light emission control signal LCLK. In other words, the enable signals EN1 to EN4 are signals whose phase differences with the light emission control signal LCLK are 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0057] [Pixel configuration example] 6 is a circuit diagram showing an example of a configuration of a G pixel 310 according to the first embodiment of the present technology. The G pixel 310 includes a SPAD 311, a resistor 312, and an inverter 313.

[0058] The SPAD 311 generates a photocurrent by photoelectric conversion and avalanche amplifies the photocurrent. The resistor 312 and the SPAD 311 are connected in series between a power supply terminal and a ground terminal.

[0059] The inverter 313 inverts the potential at the connection point between the resistor 312 and the SPAD 311 and outputs the inverted potential to the circuit block 370 as a pulse signal Pg.

[0060] Also, for example, the SPAD 311 is provided on the pixel chip 201, and the resistor 312 and inverter 313 and subsequent circuits (such as the circuit block 370) are provided on the circuit chip 202. It is also possible to provide the entire G pixel 310 on the pixel chip 201.

[0061] The circuit configurations of the R pixel 315, the B pixel 316, and the IR pixels 321 to 324 are similar to that of the G pixel 310.

[0062] [Example of circuit block configuration] 7 is a circuit diagram showing an example of a configuration of a circuit block 370 according to the first embodiment of the present technology. The circuit block 370 includes an AND (logical product) gate 381 and a selector 391.

[0063] The AND (logical product) gate 381 obtains a logical product of the enable signal EN 1 from the pixel driving unit 210 and the pulse signal Pir from the IR pixel 321 , and outputs the result to the selector 391 .

[0064] The selector 391 selects one of the logical product from the AND gate 381 and the pulse signals Pir, Pr, Pg, and Pb in accordance with a control signal CTRL from the pixel driving unit 210. The selector 391 outputs the selected signal to the counter 341 as an input signal CIN.

[0065] [Example of operation of solid-state imaging device] 8 is a diagram for explaining the operation of the counters in the first embodiment of the present technology. The counter 341 in pixel block 301 is counter #1, the counter in pixel block 302 is counter #2, the counter in pixel block 303 is counter #3, and the counter in pixel block 304 is counter #4.

[0066] In the distance measurement mode, counter #1 counts the number of pulses (in other words, the number of photons) in synchronization with enable signal EN1 with a phase difference of 0 degrees. Counter #2 counts the number of pulses in synchronization with enable signal EN2 with a phase difference of 90 degrees. Counter #3 counts the number of pulses in synchronization with enable signal EN3 with a phase difference of 180 degrees. Counter #4 counts the number of pulses in synchronization with enable signal EN3 with a phase difference of 270 degrees.

[0067] The signal processing circuit 250 calculates the distance based on the count values ​​CNT1 to CNT4 of the counters #1 to #4, for example, using the following formula. d=(c / 4πf)×tan -1 ×{(CNT2-CNT4) / (CNT1-CNT3)}…Formula 1 In the above formula, d is the distance, for example, in meters (m). c is the speed of light, for example, in meters per second (m / s). tan -1 is the inverse function of the tangent function. The value of (CNT2-CNT4) / (CNT1-CNT3) indicates the phase difference between the irradiated light and the reflected light. π indicates the ratio of the circumference of a circle to its diameter. Furthermore, f is the frequency of the irradiated light, and its unit is, for example, megahertz (MHz).

[0068] A distance measurement method that calculates distance based on the time of flight of light in this way is called a ToF (Time of Flight) method. Note that the solid-state imaging element 200 measures distance using four enable signals with different phases, but is not limited to this configuration. For example, the solid-state imaging element 200 can also measure distance using two enable signals with different phases. In this case, for example, an IR pixel 321 corresponding to 0 degrees and an IR pixel 323 corresponding to 180 degrees are arranged, and the distance is calculated from their count values.

[0069] On the other hand, in the imaging mode, the counters #1 to #4 count the number of pulses of the IR pixels, R pixels, G pixels, and B pixels in sequence in synchronization with the vertical synchronization signal VSYNC. The signal processing circuit 250 processes the count value of each pixel as a pixel signal of that pixel.

[0070] Here, it is assumed that the R, G, and B pixels are not provided with an IR cut filter that blocks infrared light. In this configuration, the R, G, and B pixels receive infrared light in addition to visible light. For this reason, the signal processing circuit 250 uses the count value of the IR pixel to separate the IR component from the pixel signals (i.e., the count value) of the R, G, and B pixels to generate image data.

[0071] It should be noted that an IR cut filter can be provided for each of the R, G, and B pixels, in which case separation of the IR component becomes unnecessary in the imaging mode.

[0072] In addition, the solid-state imaging element 200 can also capture an IR image in which only pixel signals of IR pixels are arranged in the imaging mode. In this case, for example, the imaging mode includes an IR imaging mode for capturing an IR image and an RGB imaging mode for capturing an RGB image in which only pixel signals of R pixels, G pixels, and B pixels are arranged. When the IR imaging mode is set, the counters #1 to #4 output pixel signals of the IR pixels, and when the RGB imaging mode is set, the counters #1 to #4 output pixel signals of the R, G, and B pixels.

[0073] 9 is a timing chart showing an example of the operation of the solid-state imaging element 200 in the distance measurement mode according to the first embodiment of the present technology. It is assumed that the distance measurement mode is set at timing T0. The processor 140 stops supplying the vertical synchronization signal VSYNC. The vertical scanning circuit 220 supplies a reset signal RST to each pixel block to initialize the count value.

[0074] Also, at timing T1, the driver 120 starts supplying the light emission control signal LCLK, and the light emitting unit 110 emits light in synchronization with the signal. Also, at timing T1, the pixel driving unit 210 starts supplying the enable signal EN1, which has a phase difference of 0 degrees with respect to the light emission control signal LCLK. Then, at timing T2, the pixel driving unit 210 starts supplying the enable signal EN2, which has a phase difference of 90 degrees. At timing T3, the pixel driving unit 210 starts supplying the enable signal EN3, which has a phase difference of 180 degrees. At timing T4, the pixel driving unit 210 starts supplying the enable signal EN4, which has a phase difference of 270 degrees.

[0075] After a certain period of time has elapsed, the vertical scanning circuit 220 outputs the count values ​​in response to a selection signal SEL. The signal processing circuit 250 calculates the distance for each pixel block using Equation 1 based on the count values.

[0076] 10 is a timing chart showing an example of the operation of the imaging mode of the solid-state imaging element 200 according to the first embodiment of the present technology. It is assumed that the imaging mode is set at timing T10. The processor 140 starts supplying the vertical synchronization signal VSYNC at timing T11 or later.

[0077] In addition, the driver 120 stops supplying the light emission control signal LCLK, and the pixel driving unit 210 stops supplying the enable signals EN1 to EN4. The vertical scanning circuit 220 supplies a reset signal RST to each of the pixel blocks to initialize the count values. Then, during the exposure period from timing T12 to T13 synchronized with the vertical synchronization signal VSYNC, the vertical scanning circuit 220 stops supplying the reset signal RST. During this period, each counter such as the counter 341 counts the number of pulses, and the vertical scanning circuit 220 outputs the count values ​​in response to the selection signal SEL. The signal processing circuit 250 performs processing such as IR separation on the count values ​​to generate image data.

[0078] 11 is an example of an overall diagram of the sensing system 100 according to the first embodiment of the present technology. In the pixel block 301, the circuit block 370 and the counter 341 are disposed in the counting unit 330.

[0079] The light-emitting unit 110 emits invisible light (such as infrared light) as irradiated light in synchronization with a light-emission control signal LCLK having a higher frequency than the vertical synchronization signal VSYNC. The IR pixel 321 photoelectrically converts reflected light of the irradiated light to generate a pulse signal Pir. The R pixel 315, the G pixel 310, and the B pixel 316 photoelectrically convert red, green, and blue visible light to generate pulse signals Pr, Pg, and Pb.

[0080] The light emitting unit 110 can also irradiate invisible light (such as ultraviolet light) other than infrared light. Also, the pixel array unit 230 can have pixels arranged therein that receive visible light (such as white light) other than red, green, and blue.

[0081] The IR pixel 321 is an example of an invisible light pixel as defined in the claims. The R pixel 315, the G pixel 310, and the B pixel 316 are an example of a visible light pixel as defined in the claims.

[0082] In the distance measurement mode, the counter 330 counts the number of pulse signals Pir in synchronization with the enable signal. On the other hand, in the imaging mode, the counter 330 counts the number of pulse signals Pir, Pr, Pg, and Pb in synchronization with the vertical synchronization signal VSYNC. Since only one counter 341 is provided in the pixel block 301, the counter 341 counts each of the pulse signals Pir, Pr, Pg, and Pb in a predetermined order.

[0083] By the above-mentioned control, the solid-state imaging element 200 can perform distance measurement by the ToF method in addition to capturing image data. Also, since the solid-state imaging element 200 itself can perform distance measurement, there is no need to add a distance measurement sensor that uses infrared rays or a laser. Therefore, the power consumption and cost of the sensing system 100 can be reduced compared to the case where a distance measurement sensor is added separately.

[0084] 12 is a flowchart showing an example of the operation of the sensing system 100 in the first embodiment of the present technology. This operation is started, for example, when an application for performing distance measurement and imaging is executed.

[0085] The sensing system 100 transitions to a distance measurement mode, and the light emitting unit 110 emits irradiation light in synchronization with the light emission control signal LCLK (step S901). The counter 341 counts the number of pulses in synchronization with the light emission control signal LCLK (step S902). The signal processing circuit 250 then measures the distance based on the count value and generates a depth map (step S903).

[0086] Next, the sensing system 100 transitions to an imaging mode, and the counter 341 in the solid-state imaging element 200 counts the number of pulses during an exposure period synchronized with a vertical synchronization signal (step S904). The signal processing circuit 250 performs image processing such as face recognition based on the image data in which the count values ​​are arranged (step S905). After step S905, the sensing system 100 ends its operation.

[0087] Although the solid-state imaging device 200 performs imaging (step S904) after distance measurement (step S903), distance measurement may be performed after imaging. Also, distance measurement and imaging may be performed simultaneously.

[0088] Thus, according to the first embodiment of the present technology, the counting unit 330 counts the number of pulses of R, G, and B pixels, and counts the number of pulses of IR pixels in synchronization with the light emission control signal, so that distance measurement can be performed while capturing image data.

[0089] [Variations] In the above-described first embodiment, four pixels in a pixel block share one counter 341, but in this configuration, the four pixels cannot perform counting in parallel. The solid-state imaging device 200 of this modified example of the first embodiment differs from the first embodiment in that a counter is provided for each pixel.

[0090] 13 is a block diagram showing an example of a configuration of a pixel block 301 in a modification of the first embodiment of the present technology. The pixel block 301 in the modification of the first embodiment differs from the first embodiment in that it further includes counters 342, 343, and 344, and switches 352, 353, and 354. The counters 342, 343, and 344 are arranged in a counting unit 330.

[0091] The counter 341 in the first embodiment outputs a count value as CNTir to the switch 351. The counter 342 counts the number of pulse signals Pr and outputs the count value as CNTr to the switch 352. The counters 343 and 344 count the number of pulse signals Pg and Pb and output the count values ​​as CNTg and CNTb to the switches 353 and 354. The counters 341, 342, 343 and 344 are initialized by reset signals RSTir, RSTr, RSTg and RSTb.

[0092] The counters 341 to 344 are an example of the first to fourth counters set forth in the claims.

[0093] The switch 351 in the first embodiment outputs the count value CNTir to the column buffer 240 via the vertical signal line 309-(k+1) in accordance with the selection signal SEL(n+1). The switch 352 outputs the count value CNTr to the column buffer 240 via the vertical signal line 309-(k+1) in accordance with the selection signal SELn.

[0094] The switch 353 outputs the count value CNTg to the column buffer 240 via the vertical signal line 309-k in accordance with the selection signal SELn. The switch 354 outputs the count value CNTb to the column buffer 240 via the vertical signal line 309-k in accordance with the selection signal SEL(n+1).

[0095] The pixel blocks 302 to 304 have the same configuration as the pixel block 301 .

[0096] 14 is a circuit diagram showing a configuration example of a circuit block 370 in a modification of the first embodiment of the present technology. The circuit block 370 in the modification of the first embodiment differs from the first embodiment in that the pulse signals Pr, Pg, and Pb are not input to a selector 391. The selector 391 in the modification of the first embodiment selects either the logical product from the AND gate 381 or the pulse signal Pir in accordance with a control signal CTRL.

[0097] 15 is a diagram for explaining the operation of the counter in the modified example of the first embodiment of the present technology. The counters 341 to 344 are counters #1 to #4. In the distance measurement mode, the counter #1 counts the number of pulses in synchronization with the enable signal EN1 with a phase difference of 0 degrees. The counters #2 to #4 stop counting operations.

[0098] In the imaging mode, counters #1 to #4 count the number of pulses of each of the IR pixels, R pixels, G pixels, and B pixels in synchronization with the vertical synchronization signal VSYNC. These counts are performed in parallel. As described above, since a counter is provided for each pixel, the number of pulses can be counted in parallel for four pixels. This allows faster counting compared to the case where one counter is shared by four pixels.

[0099] In this manner, in the modified example of the first embodiment of the present technology, the counters 341 to 344 of the counting unit 330 count the number of pulses in parallel, so that the time required for counting can be reduced compared to the case where one counter is shared by four pixels.

[0100] <2. Second embodiment> In the first embodiment described above, four pixels in a pixel block share one counter 341, but this configuration does not allow parallel counting for each of the visible light pixels and the IR pixels. The solid-state imaging device 200 of the second embodiment differs from the first embodiment in that a counter is provided for each of the visible light pixels and the IR pixels.

[0101] 16 is an example of a plan view of a pixel array section 230 in the second embodiment of the present technology. In the pixel array section 230 in the second embodiment, an IR pixel 321 and R pixels 315-1, 315-2, and 315-3 are arranged in a pixel block 301. An IR pixel 322 and three G pixels are arranged in a pixel block 302. An IR pixel 323 and three G pixels are arranged in a pixel block 303. An IR pixel 324 and three B pixels are arranged in a pixel block 304.

[0102] 17 is a block diagram showing an example of a configuration of a pixel block 301 according to a second embodiment of the present technology. The pixel block 301 according to the second embodiment differs from the pixel block 301 according to the first embodiment in that it further includes a counter 342 and a switch 352. The counter 342 and the switch 352 are disposed in the counting unit 330.

[0103] The circuit block 370 of the second embodiment outputs an input signal CIN1 to a counter 341 and outputs an input signal CIN2 to a counter 342.

[0104] The counter 341 in the second embodiment counts the input signal CIN1 and outputs the count value as CNT1 to the switch 351. The counter 342 counts the input signal CIN2 and outputs the count value as CNT2 to the switch 352. Note that the counters 341 and 342 are examples of the first and second counters described in the claims.

[0105] The switch 351 outputs the count value CNT1 to the column buffer 240 via the vertical signal line 309-2 in accordance with the selection signal SEL. The switch 352 outputs the count value CNT2 to the column buffer 240 via the vertical signal line 309-1 in accordance with the selection signal SEL.

[0106] The pixel blocks 302 to 304 have the same configuration as the pixel block 301 .

[0107] 18 is a circuit diagram showing a configuration example of a circuit block 370 according to a second embodiment of the present technology. The circuit block 370 according to the second embodiment differs from the first embodiment in that it further includes a selector 392. In addition, a pulse signal from a visible light pixel is not input to the selector 391 according to the second embodiment. The selector 391 selects either the logical product from the AND gate 381 or the pulse signal Pir in accordance with a control signal CTRL1, and outputs the selected signal to the counter 341 as an input signal CIN1.

[0108] The selector 392 selects one of the pulse signals Pr1, Pr2, and Pr3 from the R pixels 315-1, 315-2, and 315-3 in accordance with a control signal CTRL2, and outputs the selected signal as an input signal CIN2 to the counter 342. As the selector 392, for example, a multiplexer is used.

[0109] 16 to 18, a counter is provided for each of the visible light pixels and the IR pixels, so that the counting unit 330 can count the number of pulses of the visible light pixels in parallel with the number of pulses of the IR pixels 321. This allows faster counting compared to when the visible light pixels and the IR pixels 321 share one counter.

[0110] In this way, according to the second embodiment of the present technology, counter 341 counts the number of pulses of IR pixels, and counter 342 counts the number of pulses of visible light pixels, so that the time required for counting can be shortened compared to the case where those pixels share one counter.

[0111] [Variations] In the second embodiment described above, the number of pulses of each of the three visible light pixels is counted, but with this configuration, it is difficult to shorten the time required for counting. The solid-state imaging device 200 of the modified example of the second embodiment differs from the second embodiment in that pixel addition is performed.

[0112] A plan view of a pixel array section 230 in the modification of the second embodiment is similar to that of the second embodiment. In the modification of the second embodiment, one counter is arranged for each pixel block.

[0113] 19 is a circuit diagram showing a configuration example of a circuit block 370 in a modified example of the second embodiment of the present technology. The circuit block 370 in the modified example of the second embodiment differs from the second embodiment in that it includes an OR (logical sum) gate 371 instead of a selector 392.

[0114] The OR gate 371 obtains the logical sum of the pulse signals Pr1, Pr2, and Pr3, and outputs the result to the selector 391. This OR gate 371 can perform pixel addition for the R pixels 315-1, 315-2, and 315-3.

[0115] A selector 391 according to the modified example of the second embodiment receives the logical product from the AND gate 381, the pulse signal Pir, the pulse signals Pr1, Pr2, and Pr3, and the logical sum from the OR gate 371. The selector 391 selects one of these signals in accordance with a control signal CTRL, and outputs the selected signal to the counter 341 as an input signal CIN.

[0116] In addition, in the modified example of the second embodiment, the imaging modes include an addition mode in which pixel addition is performed and a non-addition mode in which pixel addition is not performed. In the addition mode, the selector 391 selects the logical sum from the OR gate 371 and the pulse signal Pir in sequence, and in the non-addition mode, the selector 391 selects the pulse signals Pir, Pr1, Pr2, and Pr3 in sequence. By pixel addition, the number of pulses can be counted at high speed.

[0117] In this way, according to the modification of the second embodiment of the present technology, in the addition mode, the selector 391 selects the logical sum from the OR gate 371, so that pixel addition can be performed for three pixels. This makes it possible to shorten the time required to count the number of pulses.

[0118] <3. Third embodiment> In the first embodiment described above, the IR pixels 321 to 324 are arranged adjacent to each other, but this arrangement may complicate the demosaic processing. The solid-state imaging device 200 of the third embodiment differs from the first embodiment in that the IR pixels are arranged at the positions of the G pixels in the Bayer arrangement.

[0119] FIG. 20 is an example of a plan view of a pixel array section in the third embodiment of the present technology. In the pixel array section 230 of the third embodiment, the IR pixel 321 is arranged in the lower left or upper right (lower left in the figure) in the pixel block 301. The IR pixels 322, 323, and 324 are also arranged in the lower left or upper right (lower left in the figure) in the corresponding pixel block. In other words, the IR pixel is arranged at the position of the G pixel in the Bayer array. With this arrangement, when a downstream circuit (such as the signal processing circuit 250) performs demosaic processing, the IR pixel can be interpolated using the signal of the G pixel in the same block. This can simplify the demosaic processing.

[0120] In this way, in the third embodiment of the present technology, IR pixels are arranged at the positions of G pixels in the Bayer array in each pixel block, so that demosaic processing can be simplified.

[0121] <4. Fourth embodiment> In the first embodiment described above, the IR pixels 321, 322, 323, and 324 are arranged in a two-dimensional lattice, but this arrangement may result in a shortage of IR pixels in a predetermined direction (such as the horizontal direction). The solid-state imaging element 200 of the fourth embodiment differs from the first embodiment in that the IR pixels 322, 323, and 324 are arranged only in a predetermined direction.

[0122] 21 is an example of a plan view of a pixel array section 230 according to a fourth embodiment of the present technology. In the pixel array section 230 according to the fourth embodiment, IR pixels such as IR pixels 322, 323, and 324 are arranged adjacent to each other in a predetermined direction (horizontal direction). In the lines in which the IR pixels are arranged, the R, G, and B pixels are arranged, for example, in a Bayer array.

[0123] As shown in the figure, by arranging IR pixels in a direction such as the horizontal direction, the number of IR pixels in that arrangement direction can be increased compared to when the IR pixels are arranged in a two-dimensional grid, thereby improving the distance measurement accuracy in that direction.

[0124] In the imaging mode of the fourth embodiment, the counter 330 does not count the number of pulses of the IR pixels, but only counts the number of pulses of the R, G, and B pixels. The signal processing circuit 250 then interpolates the line in the image data in which the IR pixels are arranged, using pixel signals around the line.

[0125] In this way, in the fourth embodiment of the present technology, since the IR pixels 321, 322, 323, and 324 are arranged in a predetermined direction, the number of IR pixels in the arrangement direction is greater than in the case where the IR pixels are arranged in a two-dimensional lattice shape, thereby improving the distance measurement accuracy in the direction in which the IR pixels are arranged.

[0126] [Variations] In the above-described fourth embodiment, the IR pixels 321, 322, 323, and 324 are arranged adjacent to each other in a predetermined direction, but this configuration increases the number of pixels to be interpolated, which may result in a decrease in image quality. The solid-state imaging device 200 according to the modification of the fourth embodiment differs from the fourth embodiment in that visible light pixels are inserted between the IR pixels.

[0127] 22 is an example of a plan view of a pixel array section 230 in a modification of the fourth embodiment of the present technology. The pixel array section 230 in the modification of the fourth embodiment is divided into a plurality of pixel blocks such as pixel blocks 301 to 307.

[0128] The pixel blocks 301 to 307 are arranged in a predetermined direction (such as the horizontal direction). In the pixel block 301, 16 pixels are arranged in 4 rows and 4 columns. Of these, G pixels are arranged in the upper left 2 rows and 2 columns and the lower right 2 rows and 2 columns. Three R pixels and an IR pixel are arranged in the upper right 2 rows and 2 columns. The IR pixel is disposed to the lower right of these four pixels. B pixels are arranged in the lower left 2 rows and 2 columns.

[0129] In the pixel block 302, 16 pixels are arranged in 4 rows and 4 columns. Of these, G pixels are arranged in the upper left 2 rows and 2 columns and the lower right 2 rows and 2 columns. R pixels are arranged in the upper right 2 rows and 2 columns, and B pixels are arranged in the lower left 2 rows and 2 columns. This type of arrangement is called a quadra arrangement. In a quadra arrangement, the signal processing circuit 250 can improve sensitivity in a dark place or the like by pixel addition of four adjacent visible light pixels.

[0130] The arrangement of pixel blocks 303, 305, and 307 is similar to that of pixel block 301. The arrangement of pixel blocks 304 and 306 is similar to that of pixel block 302.

[0131] Furthermore, the number of pulses of the IR pixels in pixel block 301 is counted in synchronization with enable signal EN1 having a phase difference of 0 degrees, and the number of pulses of the IR pixels in pixel block 303 is counted in synchronization with enable signal EN2 having a phase difference of 90 degrees. The number of pulses of the IR pixels in pixel block 305 is counted in synchronization with enable signal EN3 having a phase difference of 180 degrees, and the number of pulses of the IR pixels in pixel block 307 is counted in synchronization with enable signal EN4 having a phase difference of 270 degrees.

[0132] The arrangement shown in the figure inserts visible light pixels between IR pixels, which reduces the number of pixels to be interpolated in the imaging mode compared to when IR pixels are arranged adjacent to each other, improving image quality.

[0133] In this way, in the modification of the fourth embodiment of the present technology, since the IR pixels are arranged in a predetermined direction and the visible light pixels are inserted between them, the number of pixels to be interpolated can be reduced compared to the case where the IR pixels are arranged adjacent to each other, thereby improving the image quality of the image data.

[0134] <5. Fifth embodiment> In the first embodiment described above, R, G, and B pixels and IR pixels are arranged in each of the pixel blocks 301 to 304, but in this configuration, the minimum unit of distance measurement is 16 pixels, and there is a risk of a shortage of distance measurement points. The solid-state imaging element 200 of the fifth embodiment differs from the first embodiment in that the number of distance measurement points is increased.

[0135] 23 is an example of a plan view of a pixel array section 230 according to the fifth embodiment of the present technology. The pixel array section 230 according to the fifth embodiment is divided into a plurality of pixel blocks such as pixel blocks 301 to 304.

[0136] In the upper left pixel block 301, IR pixels 321 to 324 are arranged in 2 rows and 2 columns. In the upper right pixel block 302, an R pixel 315, a G pixel 310-1, a G pixel 310-2, and a B pixel 316 are arranged in a Bayer array. The array of the lower left pixel block 303 is similar to that of the pixel block 302. The array of the lower right pixel block 304 is similar to that of the pixel block 301.

[0137] With the arrangement shown in the figure, the downstream signal processing circuit 250 can obtain the distances for two pixel blocks in a 16 row x 16 column area. In other words, the number of distance measurement points is doubled compared to the first embodiment.

[0138] 24 is a block diagram showing an example configuration of a pixel block 302 in which visible light pixels are arranged according to the fifth embodiment of the present technology. The pixel block 302 includes an R pixel 315, a G pixel 310-1, a G pixel 310-2, and a B pixel 316, counters 341 to 344, and switches 351 to 354. The counters 341 to 344 are disposed in a counting unit 330.

[0139] The counter 341 of the fifth embodiment counts the number of pulse signals Pr from the R pixels 315 and outputs a count value CNTr to the switch 351. The counter 342 of the fifth embodiment counts the number of pulse signals Pg1 from the G pixels 310-1 and outputs a count value CNTg1 to the switch 352. The counter 343 of the fifth embodiment counts the number of pulse signals Pg2 from the G pixels 310-2 and outputs a count value CNTg2 to the switch 353. The counter 344 of the fifth embodiment counts the number of pulse signals Pb from the B pixels 316 and outputs a count value CNTb to the switch 354.

[0140] Moreover, the counters 341, 342, 343 and 344 are initialized by reset signals RSTr, RSTg1, RSTg2 and RSTb.

[0141] The switch 351 of the fifth embodiment outputs the count value CNTr to the column buffer 240 via the vertical signal line 309-k in accordance with the selection signal SELn. The switch 352 of the fifth embodiment outputs the count value CNTg1 to the column buffer 240 via the vertical signal line 309-(k+1) in accordance with the selection signal SELn. The switch 353 of the fifth embodiment outputs the count value CNTg2 to the column buffer 240 via the vertical signal line 309-k in accordance with the selection signal SEL(n+1). The switch 354 of the fifth embodiment outputs the count value CNTb to the column buffer 240 via the vertical signal line 309-(k+1) in accordance with the selection signal SEL(n+1).

[0142] 25 is a block diagram showing an example of a configuration of a pixel block 301 in which IR pixels are arranged according to the fifth embodiment of the present technology. The pixel block 301 includes IR pixels 321 to 324, a circuit block 370, counters 345 to 348, and switches 355 to 358. The counters 345 to 348 are disposed in a counting unit 330.

[0143] In the distance measurement mode, the circuit block 370 of the fifth embodiment supplies the logical sum of pulse signals as the input signal CINir1 to the counter 345 in synchronization with the enable signal EN1 having a phase difference of 0 degrees. Also, in the distance measurement mode, the circuit block 370 supplies the logical sum of pulse signals as the input signals CINir2 to CINir4 to the counters 346 to 348 in synchronization with the enable signals EN2 to EN4 having phase differences of 90 degrees, 180 degrees, and 270 degrees.

[0144] The counters 345 to 348 count the number of input signals CINir1 to CINir4. These counters output the count values ​​to the switches 355 to 358 as CNTir1 to CNTir4.

[0145] The configuration of the switches 355 to 358 is similar to that of the switches 351 to 354 in the pixel block 302 in which visible light pixels are arranged.

[0146] 26 is a circuit diagram showing a configuration example of a circuit block 370 according to the fifth embodiment of the present technology. The circuit block 370 according to the fifth embodiment includes OR gates 371 to 374 and AND gates 381 to 384.

[0147] The OR gate 371 in the fifth embodiment outputs the logical sum of the pulse signals Pin1 to Pin4 to the AND gate 381. The OR gates 372 to 374 output the logical sum of the pulse signals Pin1 to Pin4 to the AND gates 382 to 384.

[0148] An AND gate 381 in the fifth embodiment calculates the logical product of the enable signal EN1 and the output of the OR gate 371, and outputs the result as an input signal CINir1 to the counter 345. AND gates 382 to 384 calculate the logical product of the enable signals EN2 to EN4 and the outputs of the OR gates 372 to 374, and output the result as input signals CINir2 to CINir4 to the counters 346 to 348.

[0149] 27 is a diagram for explaining the operation of the counters in the fifth embodiment of the present technology. Counters 341 to 344 in pixel block 302 in which R, G, and B pixels are arranged are counters #1 to #4, and counters 345 to 348 in pixel block 301 in which IR pixels are arranged are counters #5 to #8.

[0150] In the distance measurement mode, the counters #5 to #8 count the number of pulses in synchronization with the enable signals EN1 to EN4 having phase differences of 0 degrees, 90 degrees, 180 degrees, and 270 degrees. The counters #1 to #4 stop counting operations.

[0151] On the other hand, in the imaging mode, the counters #1 to #4 count the number of pulses of the R pixel 315, the G pixel 310-1, the G pixel 310-2, and the B pixel 316 in synchronization with the vertical synchronization signal VSYNC. The counters #5 to #8 stop their counting operations.

[0152] As shown in the examples of FIGS. 24 to 27, a counter is provided for each pixel, so that counting can be performed at higher speeds than in the first embodiment in which one counter is shared by four pixels.

[0153] In this way, according to the fifth embodiment of the present technology, the signal processing circuit 250 can increase the number of distance measurement points by calculating the distance between two pixel blocks in a 16 row x 16 column area. Also, since a counter is provided for each pixel, the time required for counting can be reduced compared to the case where one counter is shared by four pixels.

[0154] [First Modification] In the above-mentioned fifth embodiment, a counter is provided for each pixel, but this configuration makes it difficult to miniaturize the pixels. The solid-state imaging device 200 of the first modified example of the fifth embodiment differs from the fifth embodiment in that a counter is shared by multiple visible light pixels.

[0155] 28 is a block diagram showing an example of a configuration of a pixel block 302 in which visible light pixels are arranged in a first modified example of the fifth embodiment of the present technology. The pixel block 302 of the first modified example of the fifth embodiment differs from the fifth embodiment in that it does not include the selectors 342 to 344 and the switches 352 to 354. In addition, the pixel block 302 further includes a selector 393.

[0156] The selector 393 selects one of the pulse signals Pr, Pg1, Pg2, and Pb in accordance with a control signal CTRL, and outputs the selected signal to the counter 341. The counter 341 in the modification of the fifth embodiment outputs the count value to the switch 351 as CNT.

[0157] As shown in the figure, the selector 393 selects one of the pulse signals of each of the four pixels, so that the counter 341 can be shared by the four pixels. Reducing the number of counters facilitates increasing the number of pixels.

[0158] In this way, according to the first modified example of the fifth embodiment of the present technology, the selector 393 selects one of the pulse signals of each of the four pixels, so that the four pixels can share one counter 341. This makes it easy to achieve a large number of pixels.

[0159] [Second modified example] In the above-described fifth embodiment, the distance is calculated using enable signals EN1 to EN4 with fixed phases in pixel block 301 in which IR pixels are arranged, but this configuration may result in a shortage of ranging points. The solid-state imaging device 200 of the second modified example of the fifth embodiment differs from the fifth embodiment in that the phase of the enable signal is switched to increase the number of ranging points.

[0160] 29 is a block diagram showing an example of a configuration of a pixel block 301 in which IR pixels are arranged in a second modified example of the fifth embodiment of the present technology. In the second modified example of the fifth embodiment, the counters 345 and 348 count the number of pulses in synchronization with an enable signal that switches the phase difference from 0 degrees to 180 degrees. The counters 346 and 347 count the number of pulses in synchronization with an enable signal that switches the phase difference from 90 degrees to 270 degrees.

[0161] The counters 345 and 348 are an example of a first counter recited in the claims, and the counters 346 and 347 are an example of a second counter recited in the claims.

[0162] 30 is a timing chart showing an example of an operation of the solid-state imaging device 200 in the distance measurement mode in the second modified example of the fifth embodiment of the present technology. It is assumed that the distance measurement mode is set at timing T0. The processor 140 stops supplying the vertical synchronization signal VSYNC. The vertical scanning circuit 220 supplies a reset signal RSTir to the counters 345 to 348 to initialize the count values.

[0163] Also, at timing T1, driver 120 starts supplying a light emission control signal LCLK, and light emitting unit 110 emits light in synchronization with that signal. Also, at timing T1, pixel driving unit 210 starts supplying enable signals EN1 and EN4, which have a phase difference of 0 degrees with respect to light emission control signal LCLK. Then, at timing T2, pixel driving unit 210 starts supplying enable signals EN2 and EN3, which have a phase difference of 90 degrees.

[0164] After a certain period of time has elapsed, the vertical scanning circuit 220 outputs the count values ​​in response to a selection signal, and the signal processing circuit 250 holds the count values.

[0165] Next, at timing T3, vertical scanning circuit 220 supplies reset signal RSTir to counters 345 to 348 to initialize the count values. At timing T4, pixel driving unit 210 starts supplying enable signals EN1 and EN4 that have a phase difference of 180 degrees with respect to light emission control signal LCLK. Then, at timing T5, pixel driving unit 210 starts supplying enable signals EN2 and EN3 that have a phase difference of 270 degrees.

[0166] After a certain period of time has elapsed, the vertical scanning circuit 220 outputs the count value in response to a selection signal. The signal processing circuit 250 calculates the distance for each pixel block based on the held count value and the output count value.

[0167] As illustrated in the figure, the pixel driving unit 210 supplies an enable signal that sequentially sets the phase difference to a plurality of set values ​​(such as 90 degrees and 270 degrees). By switching the phase difference in this way, two ranging points can be provided in the pixel block 301, doubling the number of ranging points compared to the fifth embodiment.

[0168] In this way, according to the second modified example of the fifth embodiment of the present technology, the pixel driving section 210 switches the phase difference of the enable signal, so that it is possible to increase the number of distance measurement points.

[0169] [Third Variation] In the fifth embodiment described above, the counter counts the number of pulses in units of four pixels in the distance measurement mode. However, in this configuration, as the number of pixels to be counted increases, the maximum value of the count value increases, and the data size of the count value increases. The solid-state imaging device 200 of the third modified example of the fifth embodiment differs from the fifth embodiment in that the number of pixels to be counted is switched between four pixels and two pixels, making the data size variable.

[0170] 31 is an example of a plan view of a pixel array unit in a third modified example of the fifth embodiment of the present technology. In this third modified example of the fifth embodiment, in a pixel block 301, the number of pulses is counted in synchronization with an enable signal EN1 of 0 degrees. In addition, in a pixel block 303, the number of pulses is counted in synchronization with an enable signal EN2 of 90 degrees. In addition, in each of the other two pixel blocks, the number of pulses is counted in synchronization with enable signals EN2 of 180 degrees and 270 degrees. In the figure, the pixel block corresponding to 180 degrees and the pixel block corresponding to 270 degrees are omitted.

[0171] 32 is a circuit diagram showing a configuration example of a circuit block 370 in a third modified example of the fifth embodiment of the present technology. The circuit block 370 in the third modified example of the fifth embodiment differs from the fifth embodiment in that it does not include an OR gate 374 and AND gates 383 and 384, but further includes a selector 391.

[0172] The OR gate 371 of the third modified example of the fifth embodiment outputs the logical sum of the pulse signals Pir1 and Pir2 to the AND gate 381. The OR gate 372 of the third modified example of the fifth embodiment outputs the logical sum of the pulse signals Pir3 and Pir4 to the AND gate 382.

[0173] The AND gate 381 of the third modified example of the fifth embodiment outputs the logical product of the signal from the OR gate 371 and the enable signal EN1a to the OR gate 373. The AND gate 382 of the third modified example of the fifth embodiment outputs the logical product of the signal from the OR gate 372 and the enable signal EN1b to the OR gate 373.

[0174] The OR gate 373 in the third modified example of the fifth embodiment outputs the logical sum of the signals from the AND gates 381 and 382 to the selector 391 .

[0175] The selector 391 of the third modified example of the fifth embodiment selects either the pulse signal Pir1 or the signal from the OR gate 373 in accordance with the control signal CTRL, and outputs the selected signal to the counter 345 as the input signal CINir1.

[0176] Moreover, the pulse signals Pir2 to Pir4 are supplied directly to counters 346 to 348 as input signals CINir2 to CINir4.

[0177] A logical circuit made up of OR gates 371 to 373 and AND gates 381 and 382 shown in the figure outputs the logical sum of two or more of the pulse signals of the four pixels.

[0178] The counters 345 to 348 are an example of the fifth to eighth counters set forth in the claims.

[0179] 33 is a diagram for explaining the operation of the pixel driving unit 210 in the third modified example of the fifth embodiment of the present technology. The control in the figure corresponds to the pixel block 301. In the third modified example of the fifth embodiment, the distance measurement mode is set to either a four-pixel addition mode or a two-pixel addition mode. The four-pixel addition mode is a mode in which the number of pixels targeted for counting the pulse signal is four, and the two-pixel addition mode is a mode in which the number of pixels targeted for counting the pulse signal is two.

[0180] In the four-pixel addition mode, the pixel driving unit 210 supplies a signal with a phase difference of 0 degrees as the enable signals EN1a and EN1b. In the two-pixel addition mode, the pixel driving unit 210 supplies a signal with a phase difference of 0 degrees as one of the enable signals EN1a and EN1b. The other of the enable signals EN1a and EN1b is not supplied. In the imaging mode, no enable signal is supplied. Note that the control of blocks other than the pixel block 301 is similar to that of the pixel block 301 illustrated in the same figure, except that the phase difference is set to 90 degrees, for example.

[0181] Furthermore, in the distance measurement mode, the pixel driving unit 210 sets the control signal CTRL to "0" to cause the selector 391 to select the signal from the OR gate 373. On the other hand, in the imaging mode, the pixel driving unit 210 sets the control signal CTRL to "1" to cause the selector 391 to select the pulse signal Pir1.

[0182] 32 and 33, circuit block 370 outputs the logical sum of the pulse signals of a set number (four or two) of the four pixels in pixel block 301, and counter 345 counts the logical sum. This makes it possible to switch the number of pixels to be counted between four and two, and change the data size of the count value.

[0183] It should be noted that pixel driving section 210 switches the number of pixels to be counted between four pixels and two pixels, but is not limited to this configuration, and can also be switched to, for example, one pixel or three pixels.

[0184] In this way, according to the third modified example of the fifth embodiment of the present technology, the counter 345 counts the logical sum of pulse signals of a set number of pixels among the four pixels in the pixel block 301, so that the data size of the count value can be changed.

[0185] [Fourth Variation] In the third modified example of the fifth embodiment described above, counting was performed in pixel block 301 in synchronization with enable signal EN1 of 0 degrees, but with this configuration, four pixel blocks are required to obtain one ranging point, which may result in a shortage of ranging points. The solid-state imaging element 200 of the fourth modified example of the fifth embodiment differs from the third modified example of the fifth embodiment in that the phase difference of the enable signal is switched.

[0186] 34 is a circuit diagram showing a configuration example of a circuit block 370 in a fourth modified example of the fifth embodiment of the present technology. The circuit block 370 in the fourth modified example of the fifth embodiment differs from the third modified example of the fifth embodiment in that the OR gates 372 to 374 and the AND gates 383 and 384 are not provided. In addition, the pixel block 370 further includes a selector 391.

[0187] The OR gate 371 in the fourth modified example of the fifth embodiment outputs the logical sum of the pulse signals Pir1 to Pir4 to the AND gates 381 and 382.

[0188] The AND gate 381 of the fourth modification of the fifth embodiment outputs the logical product of the signal from the OR gate 371 and the enable signal EN1 to the selector 391. The AND gate 382 of the fourth modification of the fifth embodiment outputs the logical product of the signal from the OR gate 371 and the enable signal EN2 to the selector 391. In addition, the phase difference of the enable signal EN1 is switched from 0 degrees to 180 degrees. The phase difference of the enable signal EN2 is switched from 90 degrees to 270 degrees.

[0189] A selector 391 in the fourth modification of the fifth embodiment selects one of the pulse signal Pir1, the signal from the AND gate 381, and the signal from the AND gate 382 in accordance with a control signal CTRL, and outputs the selected signal to the counter 345 as an input signal CINir1.

[0190] A logical circuit made up of an OR gate 371 and AND gates 381 and 382 shown in the figure outputs the logical sum of the pulse signals of the four pixels and the logical products of the enable signals EN1 and EN2.

[0191] 35 is a diagram for explaining the operation of the counters in the fourth modified example of the fifth embodiment of the present technology. Counters 341 to 344 in a pixel block 302 in which R, G, and B pixels are arranged are counters #1 to #4, and counters 345 to 348 in a pixel block 301 in which IR pixels are arranged are counters #5 to #8.

[0192] In the distance measurement mode, counter #5 counts the number of pulses in synchronization with enable signal EN1 having a phase difference of 0 degrees or 180 degrees, and then counts the number of pulses in synchronization with enable signal EN2 having a phase difference of 90 degrees or 270 degrees. Counters other than counter #5 stop counting.

[0193] On the other hand, in the imaging mode, counters #1 to #4 count the number of pulses of R pixels, G pixels, and B pixels in sequence in synchronization with the vertical synchronization signal VSYNC, while counters #5 to #8 count the number of pulses of IR pixels 321 to 324, respectively.

[0194] It is also possible to use two separate imaging modes: an IR imaging mode for capturing an IR image, and an imaging mode for capturing an RGB image in which R, G, and B pixels are arranged. In this case, only counters #5 to #8 count the number of pulses when capturing an IR image, and only counters #1 to #4 count the number of pulses when capturing an RGB image.

[0195] As shown in the figure, counter #5 (counter 345) counts the number of pulses in synchronization with enable signals of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, so that the signal processing circuit 250 can measure the distance for each pixel block in which IR pixels are arranged. This makes it possible to increase the number of ranging points compared to the third modified example of the fifth embodiment, which requires four pixel blocks to obtain one ranging point.

[0196] Thus, according to the fourth variant of the fifth embodiment of the present technology, the counter 345 counts the number of pulses in synchronization with the enable signals of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, thereby making it possible to measure the distance for each pixel block.

[0197] [Fifth Variation] In the above-mentioned fifth embodiment, a counter is arranged for each pixel, but this configuration makes it difficult to miniaturize the pixels. The solid-state imaging device 200 of the fifth modification of the fifth embodiment differs from the fifth embodiment in that the phase difference of the enable signal is switched to reduce the number of counters.

[0198] 36 is a block diagram showing an example of a configuration of a pixel block 301 in which IR pixels are arranged in a fifth modified example of the fifth embodiment of the present technology. The pixel block 301 of the fifth embodiment differs from the fifth embodiment in that it does not include the counters 347 and 348 and the switches 357 and 358.

[0199] 37 is a circuit diagram showing a configuration example of a circuit block 370 in a fifth modified example of the fifth embodiment of the present technology. The circuit block 370 in the fifth modified example of the fifth embodiment of the present technology differs from the fifth embodiment in that it does not include OR gates 372 to 374 and AND gates 383 and 384, and further includes a selector 391 and a switch 395.

[0200] The OR gate 371 of the fifth modified example of the fifth embodiment outputs the logical sum of the pulse signals Pir1 to Pir4 to the AND gates 381 and 382.

[0201] The AND gate 381 of the fifth modification of the fifth embodiment outputs the logical product of the signal from the OR gate 371 and the enable signal EN1 to the selector 391. The AND gate 382 of the fifth modification of the fifth embodiment outputs the logical product of the signal from the OR gate 371 and the enable signal EN2 to the switch 395. In addition, the phase difference of the enable signal EN1 is switched from 0 degrees to 180 degrees. The phase difference of the enable signal EN2 is switched from 90 degrees to 270 degrees.

[0202] A logic circuit consisting of the above-mentioned OR gate 371 and AND gates 381 and 382 outputs the logical sum of the pulse signals of the four pixels and the logical products of the enable signals EN1 and EN2.

[0203] A selector 391 in the fifth modification of the fifth embodiment selects either the pulse signal Pir1 or the signal from the AND gate 381 in accordance with a control signal CTRL1, and outputs the selected signal to the counter 345 as an input signal CINir1.

[0204] The switch 395 outputs the signal from the AND gate 382 to the counter 346 as the input signal CINir2 in accordance with the control signal CTRL2.

[0205] 38 is a diagram for explaining the operation of the counters in the fifth modified example of the fifth embodiment of the present technology. Counters 341 to 344 in pixel block 302 in which R, G, and B pixels are arranged are counters #1 to #4, and counters 345 and 346 in pixel block 301 in which IR pixels are arranged are counters #5 and #6.

[0206] In the distance measurement mode, counter #5 counts the number of pulses in synchronization with enable signal EN1 having a phase difference of 0 degrees or 180 degrees, and counter #6 counts the number of pulses in synchronization with enable signal EN2 having a phase difference of 90 degrees or 270 degrees. Counters #1 to #4 stop counting operations.

[0207] On the other hand, in imaging mode, counters #1 to #4 count the number of pulses of the R pixel 315, the G pixel 310-1, the G pixel 310-2, and the B pixel 316 in synchronization with the vertical synchronization signal VSYNC. Counter #5 counts the number of pulses of the IR pixel 321. Counter #6 stops counting operation.

[0208] As illustrated in FIGS. 36 to 38, the pixel driving unit 210 switches the phase difference between the enable signals EN1 and EN2, so that the number of counters in the pixel block 301 can be reduced to two.

[0209] It is also possible to use two separate imaging modes: an IR imaging mode for capturing an IR image and an imaging mode for capturing an RGB image. In this case, only counter #5 counts the number of pulses when capturing an IR image, and only counters #1 to #4 count the number of pulses when capturing an RGB image.

[0210] In this way, according to the fifth variant of the fifth embodiment of the present technology, the pixel driving unit 210 switches the phase difference between the enable signals EN1 and EN2, so that the counters in the pixel block 301 can be reduced to two, the counters 345 and 346.

[0211] <6. Sixth embodiment> In the first embodiment described above, the visible light pixels are arranged in a Bayer array, but it is also possible to provide visible light pixels that receive a pair of pupil-split incident light beams, and use the pixel signals for phase-difference AF (Auto Focus). The solid-state imaging device 200 of the sixth embodiment differs from the fifth embodiment in that the visible light pixels receive a pair of pupil-split incident light beams.

[0212] 39 is an example of a plan view of a pixel array section 230 in the sixth embodiment of the present technology. In the pixel array section 230 in the sixth embodiment, R pixels 315-1 to 315-4 are arranged in 2 rows and 2 columns in the upper right pixel block 302. Four B pixels are arranged in 2 rows and 2 columns in the lower left pixel block 303. Four G pixels are arranged in 2 rows and 2 columns in the lower right pixel block 304. The arrangement illustrated in the figure corresponds to an arrangement in which the four G pixels are replaced with IR pixels 321 to 324 in a quadra arrangement.

[0213] Furthermore, the R pixels 315-1 and 315-2 receive one of a pair of pupil-split incident light beams, and the R pixels 315-3 and 315-4 receive the other of the incident light beams. By using pixel signals from these pixels, a downstream circuit (such as the signal processing circuit 250) can realize AF using an image plane phase difference method. Only a portion of all the R pixels in the pixel array unit 230 are used for AF. Also, signals from G pixels or B pixels can be used for AF instead of R pixels.

[0214] 40 is a block diagram showing an example of a configuration of a pixel block 302 in which visible light pixels are arranged according to a sixth embodiment of the present technology. The pixel block 302 according to the sixth embodiment differs from the fifth embodiment in that it does not include the counters 343 and 344 and the switches 353 and 354, but further includes a circuit block 400.

[0215] The pixel block in which visible light pixels not used for AF are arranged has the same configuration as in the fifth embodiment, and a counter is provided for each pixel.

[0216] The circuit block 400 calculates the logical sum of the pulse signals Pr1 and Pr2 from the R pixels 315-1 and 315-2 and the logical sum of the pulse signals Pr3 and Pr4 from the R pixels 315-3 and 315-4. The circuit block 400 outputs the pulse signal Pr1 or Pr2, or the logical sum thereof, as an input signal CINr1 to the counter 341. The circuit block 400 also outputs the pulse signal Pr3 or Pr4, or the logical sum thereof, as an input signal CINr2 to the counter 342.

[0217] The counter 341 of the sixth embodiment counts the number of input signals CINr1 and outputs the count value as CNTr1 to the switch 351. The counter 342 of the sixth embodiment counts the number of input signals CINr2 and outputs the count value as CNTr2 to the switch 352. The counters 341 and 342 are initialized by reset signals RSTr1 and RSTr2.

[0218] The switch 351 of the sixth embodiment outputs the count value CNTr1 to the column buffer 240 via the vertical signal line 309-1 in accordance with the selection signal SEL. The switch 352 of the sixth embodiment outputs the count value CNTr2 to the column buffer 240 via the vertical signal line 309-2 in accordance with the selection signal SEL.

[0219] 41 is a circuit diagram showing an example of a configuration of a circuit block 400 according to the sixth embodiment of the present technology. The circuit block 400 includes OR gates 411 and 412, and selectors 421 and 422.

[0220] The OR gate 411 outputs the logical sum of the pulse signals Pr1 and Pr2 to the selector 421. The OR gate 412 outputs the logical sum of the pulse signals Pr3 and Pr4 to the selector 422. The OR gates 411 and 412 are examples of the first and second logical sum gates set forth in the claims.

[0221] The selector 421 outputs one of the pulse signal Pr1, the pulse signal Pr2, and the output of the OR gate 411 as an input signal CINr1 to the counter 341 in accordance with the control signal CTRL. The selector 422 outputs one of the pulse signal Pr3, the pulse signal Pr4, and the output of the OR gate 412 as an input signal CINr2 to the counter 342 in accordance with the control signal CTRL. The selectors 421 and 422 are examples of the first and second selectors described in the claims.

[0222] The selectors 421 and 422 select the logical sum when AF is performed, and sequentially select pulse signals other than the logical sum when AF is not performed. When the logical sum is selected, the signal processing circuit 250 detects the focus by the image plane phase difference method based on the output waveforms of the pair of R pixels.

[0223] As illustrated in FIGS. 39 to 41, the visible light pixel receives a pair of pupil-divided incident light beams, and the signal processing circuit 250 can perform AF using the image plane phase difference method.

[0224] In the sixth embodiment, the second modified example of the fifth embodiment can be applied to a pixel block in which visible light pixels not used for AF are arranged.

[0225] Furthermore, in the sixth embodiment, the selectors 421 and 422 can also output the logical sum of two visible light pixels (such as the R pixels 315-1 and 315-2) as a value obtained by adding the two pixels.

[0226] In this way, according to the sixth embodiment of the present technology, the visible light pixel receives a pair of pupil-split incident light beams, and the pixel signal can be used to perform AF using the image plane phase difference method.

[0227] [Variations] In the sixth embodiment described above, the signal processing circuit 250 performs AF using pixel signals from the R pixels 315-1 to 315-4, etc., but cannot perform addition of four pixels. The solid-state imaging device 200 of this modified example of the sixth embodiment differs from the sixth embodiment in that it performs pixel addition for four pixels.

[0228] 42 is a circuit diagram showing a configuration example of a circuit block 400 in a modification of the sixth embodiment of the present technology. The circuit block 400 in the modification of the sixth embodiment differs from the sixth embodiment in that it further includes an OR gate 413.

[0229] The OR gate 413 outputs the logical sum of the signals from the OR gates 411 and 412 to the selector 421. The OR gate 413 is an example of the third logical sum gate set forth in the claims.

[0230] Moreover, the selector 421 of the modification of the sixth embodiment selects one of the signal from the OR gate 413, the pulse signal Pir1, the pulse signal Pir2, and the output of the OR gate 411 in accordance with a control signal CTRL1. The selector 422 of the modification of the sixth embodiment selects a signal in accordance with a control signal CTRL2.

[0231] In addition, in the modification of the sixth embodiment, the imaging modes include an addition mode in which pixel addition is performed, and a non-addition mode in which pixel addition is not performed.

[0232] With the configuration shown in the figure, the selector 421 can output the signal from the OR gate 413 as a value obtained by adding four pixels in the addition mode. Also, the selectors 421 and 422 can output the signals from the OR gates 411 and 412 as a value obtained by adding two pixels in the addition mode. This makes it possible to add two or four pixels in the addition mode.

[0233] In this way, according to the modified example of the sixth embodiment of the present technology, the OR gate outputs the logical sum of the pulse signals of the four pixels to the selector 421, so that in the addition mode, the selector 421 can output the value obtained by adding up the four pixels.

[0234] <7. Seventh embodiment> In the sixth embodiment described above, IR pixels and visible light pixels are arranged adjacent to each other in a pixel block of 2 rows by 2 columns, but IR pixels etc. can also be arranged in an area wider than 2 rows by 2 columns. The solid-state imaging element 200 of the seventh embodiment differs from the sixth embodiment in that IR pixels etc. are arranged in an area wider than 2 rows by 2 columns.

[0235] FIG. 43 is an example of a plan view of the pixel array unit 230 in the seventh embodiment of the present technology. In the pixel array unit 230 in the seventh embodiment, 16 IR pixels such as IR pixels 321 to 324 are arranged adjacent to each other in 4 rows and 4 columns. When calculating the distance for every 2 rows and 2 columns, four ranging points are obtained within the region. The signal processing circuit 250 can reduce noise in the depth map by calculating the average or sum of the measured values ​​of these four ranging points. The counter can also count the number of pulses for more phases than the four phases of 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0236] Furthermore, the R, G and B pixels are arranged adjacently in a Bayer array of 4 rows and 4 columns.

[0237] In the seventh embodiment, the configuration of the counter and other components in the region where IR pixels are arranged is the same as that of the pixel block where IR pixels are arranged in the fifth embodiment. The configuration of the counter and other components in the region where visible light pixels are arranged is the same as that of the pixel block where visible light pixels are arranged in the fifth embodiment.

[0238] Note that any of the second to fifth modifications of the fifth embodiment, the sixth embodiment, and the modification of the sixth embodiment can be applied to the seventh embodiment.

[0239] In this way, according to the seventh embodiment of the present technology, since IR pixels are arranged in a 4 row x 4 column region, four ranging points can be obtained for each region. By calculating the average of these ranging points, noise can be reduced.

[0240] <8. Eighth embodiment> In the seventh embodiment described above, the visible light pixels are arranged in a 4-row x 4-column region in a Bayer array, but this configuration may result in insufficient pixel sensitivity. The solid-state imaging device 200 of the eighth embodiment differs from the seventh embodiment in that the visible light pixels are arranged in a Quadra array.

[0241] 44 is an example of a plan view of a pixel array section 230 according to the eighth embodiment of the present technology. The pixel array section 230 according to the eighth embodiment differs from the seventh embodiment in that the visible light pixels are arranged in a quadra array.

[0242] In the quadra array, the signal processing circuit 250 can improve sensitivity in a dark place or the like by performing pixel addition on four adjacent visible light pixels.

[0243] Note that any of the second to fifth modifications of the fifth embodiment, the sixth embodiment, and the modification of the sixth embodiment can be applied to the seventh embodiment.

[0244] In this way, in the eighth embodiment of the present technology, since the visible light pixels are arranged in a quadra array, the signal processing circuit 250 can improve sensitivity by performing pixel addition on four adjacent pixels.

[0245] <Applications to moving objects> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, etc.

[0246] FIG. 45 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving object control system to which the technology according to the present disclosure can be applied.

[0247] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 45, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also, as functional configurations of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (interface) 12053 are illustrated.

[0248] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0249] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 12020. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.

[0250] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for a person, a car, an obstacle, a sign, or characters on a road surface, based on the received image.

[0251] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0252] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing.

[0253] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the outside-of-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.

[0254] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0255] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside-vehicle information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0256] The audio / video output unit 12052 transmits at least one output signal of audio and video to an output device capable of visually or audibly notifying information to passengers in the vehicle or the outside of the vehicle. In the example of Fig. 45, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as the output device. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

[0257] FIG. 46 is a diagram showing an example of the installation position of the imaging unit 12031.

[0258] In FIG. 46, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0259] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the upper part of the windshield inside the vehicle cabin is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

[0260] 46 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.

[0261] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of a plurality of imaging elements, or may be an imaging element having pixels for detecting a phase difference.

[0262] For example, the microcomputer 12051 can extract, as a preceding vehicle, a three-dimensional object that is the closest three-dimensional object on the travel path of the vehicle 12100 and travels at a predetermined speed (for example, 0 km / h or more) in approximately the same direction as the vehicle 12100, by calculating the distance to each three-dimensional object in the imaging ranges 12111 to 12114 and the change over time of this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. Furthermore, the microcomputer 12051 can set a vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving that travels autonomously without relying on the driver's operation.

[0263] For example, the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0264] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

[0265] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the imaging unit 12031 among the configurations described above. Specifically, the solid-state imaging element 200 in FIG. 3 can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, distance measurement can be performed without adding a sensor, thereby reducing the power consumption and cost of the vehicle control system.

[0266] The above-mentioned embodiment shows an example for realizing the present technology, and the matters in the embodiment and the matters specified in the claims correspond to each other. Similarly, the matters specified in the claims and the matters in the embodiment of the present technology having the same name correspond to each other. However, the present technology is not limited to the embodiment, and can be realized by applying various modifications to the embodiment within the scope of the gist of the technology.

[0267] The processing procedures described in the above embodiments may be regarded as a method having a series of these procedures, or as a program for causing a computer to execute the series of procedures or a recording medium for storing the program. For example, a CD (Compact Disc), an MD (MiniDisc), a DVD (Digital Versatile Disc), a memory card, a Blu-ray (registered trademark) Disc, etc. may be used as this recording medium.

[0268] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0269] The present technology can also be configured as follows. (1) a light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; A sensing system comprising: (2) the visible light pixels include first, second, and third visible light pixels that photoelectrically convert different visible light beams from each other; the invisible light pixels include first, second, third and fourth invisible light pixels associated with enable signals having mutually different phase differences with respect to the light emission control signal, the first, second, third and fourth invisible light pixels are arranged adjacent to each other; The first, second and third visible light pixels are disposed adjacent to the first invisible light pixel. The sensing system described in (1) above. (3) The counting unit includes a counter that performs a process of counting the number of the visible light pulse signals of each of the first, second, and third visible light pixels and a process of counting the number of the invisible light pulse signals in a predetermined order. The sensing system described in (2) above. (4) The counting unit is a first counter that counts the number of the visible light pulse signals of the first visible light pixel; a second counter that counts the number of the visible light pulse signals of the second visible light pixel; a third counter that counts the number of the visible light pulse signals of the third visible light pixel; a fourth counter that counts the number of the invisible light pulse signals in synchronization with the light emission control signal; The sensing system according to (2) above, comprising: (5) the visible light pixels include first, second and third visible light pixels that photoelectrically convert the same visible light; the invisible light pixels include first, second, third and fourth invisible light pixels associated with enable signals having mutually different phase differences with respect to the light emission control signal, The first, second and third visible light pixels are disposed adjacent to the first invisible light pixel. The sensing system described in (1) above. (6) The counting unit is a selector that sequentially selects the visible light pulse signals of the first, second, and third visible light pixels as input signals; a first counter that counts the number of the input signals; a second counter that counts the number of the invisible light pulse signals in synchronization with the light emission control signal; The sensing system according to (5) above, comprising: (7) The counting unit is a logical OR gate that outputs a logical OR of the invisible light pulse signals of the first, second, and third visible light pixels; a selector that selects, as an input signal, any one of the invisible light pulse signal, the logical sum, and the visible light pulse signal of each of the first, second, and third visible light pixels; A counter that counts the number of the input signals; The sensing system according to (5) above, comprising: (8) the visible light pixels include an R (Red) pixel, a G (Green) pixel, and a B (Blue) pixel; The invisible light pixels are arranged at the positions of the G pixels in the Bayer array. The sensing system described in (1) above. (9) The invisible light pixel includes a plurality of invisible light pixels associated with enable signals having mutually different phase differences with respect to the light emission control signal, The plurality of invisible light pixels are arranged in a predetermined direction. The sensing system described in (1) above. (10) The visible light pixel is inserted between each of the plurality of invisible light pixels. The sensing system according to (9) above. (11) The visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; The first, second, third and fourth visible light pixels photoelectrically convert different visible light rays. The sensing system described in (1) above. (12) The counting unit includes a plurality of counters that count the number of the invisible light pulse signals in synchronization with enable signals having different phase differences with respect to the light emission control signal. The sensing system according to (11) above. (13) The counting unit comprises: a selector that selects one of the visible light pulse signals of the first, second, and third visible light pixels as an input signal; A counter that counts the number of the input signals; The sensing system according to (11) above, comprising: (14) The counting unit comprises: a first counter that counts the invisible light pulse signals in synchronization with a first enable signal, the phase difference of which with respect to the light emission control signal is set to 0 degrees or 180 degrees; a second counter that counts the invisible light pulse signals in synchronization with a second enable signal, the phase difference of which with respect to the light emission control signal is set to 90 degrees or 270 degrees; The sensing system according to (11) above, comprising: (15) The counting unit comprises: a logic circuit that outputs a logical sum of two or more of the invisible light pulse signals of the first, second, third and fourth invisible light pixels; a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical sum and outputs it as an input signal; a fifth counter that counts the number of the input signals; a sixth counter that counts the number of the invisible light pulse signals of the second invisible light pixel; a seventh counter that counts the number of the invisible light pulse signals of the third invisible light pixel; and The sensing system according to (11), further comprising: an eighth counter that counts the number of the invisible light pulse signals of the fourth invisible light pixel. (16) The counting unit comprises: a logic circuit that outputs a logical product of a logical sum of the invisible light pulse signals of each of the first, second, third and fourth invisible light pixels and a logical product of each of first and second enable signals having mutually different phase differences with respect to the light emission synchronization signal; a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical product and outputs it as an input signal; a fifth counter that counts the number of the input signals; a sixth counter that counts the number of the invisible light pulse signals of the second invisible light pixel; a seventh counter that counts the number of the invisible light pulse signals of the third invisible light pixel; and The sensing system according to (11), further comprising: an eighth counter that counts the number of the invisible light pulse signals of the fourth invisible light pixel. (17) The counting unit comprises: a logic circuit that outputs a logical product of a logical sum of the invisible light pulse signals of each of the first, second, third and fourth invisible light pixels and a logical product of each of first and second enable signals having mutually different phase differences with respect to the light emission synchronization signal; a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical product corresponding to the first enable signal and outputs the selected signal as an input signal; a switch that outputs the logical product corresponding to the first enable signal in accordance with a predetermined control signal; a fifth counter that counts the number of the input signals; a sixth counter that counts based on the logical product output from the second switch; The sensing system according to (11) above, comprising: (18) The visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; The first, second, third and fourth visible light pixels photoelectrically convert the same visible light. The sensing system described in (1) above. (19) The first and second visible light pixels receive one of a pair of pupil-split incident lights, the third and fourth visible light pixels receive the other of a pair of pupil-divided incident lights; The counting unit is a first logical OR gate that outputs a logical OR of the visible light pulse signals of the first and second visible light pixels as a first logical OR; a first selector that selects either the first logical sum or the visible light pulse signals of the first and second visible light pixels and outputs the selected signal as a first input signal; a second OR gate that outputs a second OR of the visible light pulse signals of the third and fourth visible light pixels; a second selector that selects either the second logical sum or the visible light pulse signals of the third and fourth visible light pixels and outputs the selected signal as a second input signal; a first counter that counts the number of the first input signals; a second counter that counts the number of the second input signals; The sensing system according to (18) above, (20) The counter unit further includes a third logic gate that outputs a logical sum of the first logical sum and the second logical sum as a third logical sum to the first selector; The first selector selects one of the third logical sum, the first logical sum, and the visible light pulse signal of each of the first and second visible light pixels. The sensing system described in (19). (21) The visible light pixels include first, second and third visible light pixels that photoelectrically convert different visible light beams, the invisible light pixels include first, second, third and fourth invisible light pixels associated with enable signals having mutually different phase differences with respect to the light emission control signal, the first, second and third visible light pixels are arranged in a first region of 4 rows by 4 columns in a Bayer array; The first, second, third and fourth invisible light pixels are arranged in a second region of 4 rows x 4 columns. The sensing system described in (1) above. (22) The visible light pixels include first, second and third visible light pixels that photoelectrically convert different visible light beams, the invisible light pixels include first, second, third and fourth invisible light pixels associated with enable signals having mutually different phase differences with respect to the light emission control signal, The first visible light pixels are arranged in a first region of 2 rows and 2 columns; The second visible light pixels are arranged in a second region of 2 rows and 2 columns; The third visible light pixels are arranged in a third region of 2 rows and 2 columns; The first, second, third and fourth invisible light pixels are arranged in a fourth region of 4 rows x 4 columns. The sensing system described in (1) above. [Explanation of symbols]

[0270] 100 Sensing System 110 Light emitting part 120 Driver 130 Controller 140 processors 150 Application Processors 200 Solid-state image sensor 201 Pixel Chip 202 Circuit Chip 210 Pixel driving unit 220 Vertical scanning circuit 230 Pixel array section 240 Column Buffer 250 Signal Processing Circuit 260 Output section 301~307 pixel block 310, 310-1, 310-2 G pixels 311 SPAD 312 Resistance 313 Inverter 315, 315-1, 315-2, 315-3, 315-4 R pixels 316 B pixels 321~324 IR pixels 330 Counting Department 341~348 Counter 351~358, 395 Switch 370, 400 circuit blocks 371~374, 411~413 OR gate 381~384 AND gate 391~393, 421, 422 Selector 12031 Imaging unit

Claims

1. a light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emitting control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; a signal processing circuit that calculates distance from the count value of the invisible light pulse signal by an iToF (indirect Time of Flight) method; Equipped with the invisible light pixels include first and second invisible light pixels, The counting unit is a first AND gate that outputs a logical product of the first enable signal of first and second enable signals having mutually different phase differences with respect to the light emission control signal and the invisible light pulse signal of the first invisible light pixel; a first selector that selects, as an input signal, either the logical product from the first logical product gate or the visible light pulse signal of the visible light pixel; a first counter that counts the number of the input signals from the first selector; a second AND gate that outputs a logical product of the second enable signal and the invisible light pulse signal of the second invisible light pixel; a second selector that selects, as an input signal, either the logical product from the second logical product gate or the visible light pulse signal of the visible light pixel; a second counter that counts the number of the input signals from the second selector; Equipped Sensing system.

2. The invisible light pixel further includes a third and a fourth invisible light pixel, The counting unit is a third AND gate that outputs a logical product of the third enable signal among first, second, third and fourth enable signals having mutually different phase differences with respect to the light emission control signal and the invisible light pulse signal of the third invisible light pixel; a third selector that selects, as an input signal, either the logical product from the third logical product gate or the visible light pulse signal of the visible light pixel; a third counter that counts the number of the input signals from the third selector; a fourth AND gate that outputs a logical product of the fourth enable signal and the invisible light pulse signal of the fourth invisible light pixel; a fourth selector that selects, as an input signal, either the logical product from the fourth logical product gate or the visible light pulse signal of the visible light pixel; a fourth counter that counts the number of the input signals from the fourth selector; Further equipped The sensing system of claim 1 .

3. the visible light pixels include first, second and third visible light pixels that photoelectrically convert different visible light beams from each other; the first, second, third and fourth invisible light pixels are arranged adjacent to each other; The first, second and third visible light pixels are disposed adjacent to the first invisible light pixel. The sensing system according to claim 2 .

4. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; a signal processing circuit that calculates a distance from the count value of the invisible light pulse signal by the iToF method; Equipped with the invisible light pixels include first and second invisible light pixels, The counting unit is a first counter that counts a signal of a logical product of the first enable signal and the invisible light pulse signal of the first invisible light pixel, the first enable signal being one of first and second enable signals having mutually different phase differences with respect to the light emission control signal; a second counter that counts a signal of a logical product of the second enable signal and the invisible light pulse signal of the second invisible light pixel; a third counter that counts the number of the visible light pulse signals of the visible light pixels; Equipped Sensing system.

5. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; Equipped with the visible light pixels include first, second and third visible light pixels that photoelectrically convert the same visible light; the invisible light pixels include first, second, third and fourth invisible light pixels associated with enable signals having mutually different phase differences with respect to the light emission control signal, The first, second and third visible light pixels are disposed adjacent to the first invisible light pixel. Sensing system.

6. The counting unit is a selector that sequentially selects the visible light pulse signals of the first, second, and third visible light pixels as input signals; a first counter for counting the number of said input signals; a second counter that counts the number of the invisible light pulse signals in synchronization with the light emission control signal; The sensing system of claim 5 .

7. The counting unit is a logical OR gate that outputs a logical OR of the visible light pulse signals of the first, second, and third visible light pixels; a selector that selects, as an input signal, any one of the logical sum of the visible light pulse signal and the logical product of the invisible light pulse signal and the enable signal of each of the first, second, and third visible light pixels; A counter that counts the number of the input signals; The sensing system of claim 5 .

8. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; a signal processing circuit that calculates a distance from the count value of the invisible light pulse signal by the iToF method; Equipped with the visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; the first, second, third, and fourth visible light pixels photoelectrically convert different visible light beams from one another; The counting unit is a first counter that counts the number of signals indicating a logical product between a logical sum of the invisible light pulse signals of the first, second, third and fourth invisible light pixels and the first enable signal among first, second, third and fourth enable signals having mutually different phase differences with respect to the light emission control signal; a second counter that counts the number of signals indicating a logical product of the logical sum of the invisible light pulse signals of each of the first, second, third and fourth invisible light pixels and the second enable signal; a third counter that counts the number of signals indicating a logical product of the logical sum of the invisible light pulse signals of each of the first, second, third and fourth invisible light pixels and the third enable signal; a fourth counter that counts the number of signals indicating a logical product of the logical sum of the invisible light pulse signals of each of the first, second, third and fourth invisible light pixels and the fourth enable signal; Equipped Sensing system.

9. The counting unit is a selector that selects one of the visible light pulse signals of the first, second, third and fourth visible light pixels as an input signal; a fifth counter for counting the number of said input signals; The sensing system of claim 8 further comprising:

10. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; Equipped with the visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; the first, second, third, and fourth visible light pixels photoelectrically convert different visible light beams from one another; The counting unit is a first counter that counts the invisible light pulse signals in synchronization with a first enable signal, the phase difference of which with respect to the light emission control signal is set to 0 degrees or 180 degrees; a second counter that counts the invisible light pulse signals in synchronization with a second enable signal, the phase difference of which with respect to the light emission control signal is set to 90 degrees or 270 degrees; A sensing system comprising:

11. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; Equipped with the visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; the first, second, third, and fourth visible light pixels photoelectrically convert different visible light beams from one another; The counting unit is a logic circuit that outputs a logical sum of two or more of the invisible light pulse signals of the first, second, third and fourth invisible light pixels; a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical sum and outputs the selected signal as an input signal; a fifth counter that counts the number of the input signals; a sixth counter that counts the number of the invisible light pulse signals of the second invisible light pixel; a seventh counter that counts the number of the invisible light pulse signals of the third invisible light pixel; an eighth counter that counts the number of the invisible light pulse signals of the fourth invisible light pixel.

12. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; Equipped with the visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; the first, second, third, and fourth visible light pixels photoelectrically convert different visible light beams from one another; The counting unit is a logic circuit that outputs a logical product of a logical sum of the invisible light pulse signals of the first, second, third and fourth invisible light pixels and first and second enable signals having mutually different phase differences with respect to the light emission control signal; a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical product and outputs the selected signal as an input signal; a fifth counter that counts the number of the input signals; a sixth counter that counts the number of the invisible light pulse signals of the second invisible light pixel; a seventh counter that counts the number of the invisible light pulse signals of the third invisible light pixel; an eighth counter that counts the number of the invisible light pulse signals of the fourth invisible light pixel.

13. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; Equipped with the visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; the first, second, third, and fourth visible light pixels photoelectrically convert different visible light beams from one another; The counting unit is a logic circuit that outputs a logical product of a logical sum of the invisible light pulse signals of the first, second, third and fourth invisible light pixels and first and second enable signals having mutually different phase differences with respect to the light emission control signal; a selector that selects either the invisible light pulse signal of the first invisible light pixel or the logical product corresponding to the first enable signal and outputs the selected signal as an input signal; a switch that outputs the logical product corresponding to the first enable signal in accordance with a predetermined control signal; a fifth counter that counts the number of the input signals; a sixth counter that counts based on the logical product output from the switch; A sensing system comprising:

14. A light-emitting unit that irradiates invisible light in synchronization with a predetermined light-emission control signal; an invisible light pixel that photoelectrically converts reflected light of the invisible light to generate a pulse signal as an invisible light pulse signal; A visible light pixel that photoelectrically converts visible light to generate a pulse signal as a visible light pulse signal; a counting unit that counts the number of the visible light pulse signals and counts the number of the invisible light pulse signals in synchronization with the light emission control signal; Equipped with the visible light pixels include first, second, third and fourth visible light pixels arranged adjacent to each other; the invisible light pixels include first, second, third and fourth invisible light pixels arranged adjacent to each other; the first, second, third and fourth visible light pixels photoelectrically convert the same visible light; the first and second visible light pixels receive one of a pair of pupil-divided incident lights; the third and fourth visible light pixels receive the other of a pair of pupil-divided incident lights; The counting unit is a first logical sum gate that outputs a logical sum of the visible light pulse signals of the first and second visible light pixels as a first logical sum; a first selector that selects either the first logical sum or the visible light pulse signals of the first and second visible light pixels, and outputs the selected signal as a first input signal; a second OR gate that outputs a second OR of the visible light pulse signals of the third and fourth visible light pixels; a second selector that selects either the second logical sum or the visible light pulse signals of the third and fourth visible light pixels and outputs the selected signal as a second input signal; a first counter for counting the number of the first input signals; a second counter that counts the number of the second input signals; A sensing system comprising:

15. the counting unit further includes a third logic gate that outputs a logical sum of the first logical sum and the second logical sum as a third logical sum to the first selector; The first selector selects one of the third logical sum, the first logical sum, and the visible light pulse signal of each of the first and second visible light pixels. The sensing system of claim 14.

Images