Imaging device and information processing system

Abstract

An imaging device according to an aspect of the present disclosure comprises: a plurality of pixels arranged in a matrix; and a control unit that performs TDI control for the plurality of pixels. Each of the pixels has an optical pulse response unit that generates an optical pulse in response to light entry, and a counter unit. The counter unit includes a rewriting circuit that rewrites an initial value, and an addition circuit that adds information based on the optical pulse to the initial value. The control unit causes information stored in a counter unit to be written into a counter unit included in a pixel of the next stage in the column direction as an initial value, and then causes the information based on the optical pulse acquired from the optical pulse response unit to be added to the initial value.

Description

Technical Field

[0001] The present disclosure relates to an imaging device and an information processing system that perform TDI (Time Delay Integration) processing.

Background Art

[0002] Conventionally, TDI sensors have been used in the fields of FA (Factory Automation), aerial photography, and medicine. This TDI sensor is a sensor that performs TDI processing for integrating the charge amount while shifting the time according to the moving speed of the subject. For example, a TDI sensor using CMOS (Complementary Metal Oxide Semiconductor) is described in Patent Document 1 below.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] [[ID=3且2]]By the way, in a conventional TDI sensor, since AD conversion processing and correlated double sampling (CDS) processing are performed, the line rate is limited by the AD conversion time and the CDS processing time, and the upper limit of the line rate is limited to about several 100 kHz. In addition, since a frame memory is required for CDS processing, it is difficult to reduce the chip cost. Therefore, it is desirable to provide an imaging device and an information processing system that do not require AD conversion processing and CDS processing.

[0005] An imaging device relating to the first aspect of this disclosure comprises a plurality of pixels arranged in a matrix and a control unit that performs TDI control on the plurality of pixels. Each pixel has an optical pulse response unit that generates an optical pulse in response to incident light and a counter unit. The counter unit includes a rewrite circuit that rewrites an initial value and an adder circuit that adds information corresponding to the optical pulse to the initial value. The control unit causes the information held in the counter unit to be written as an initial value to the counter unit included in the next row of pixels in the column direction, and then adds the information corresponding to the optical pulse obtained from the optical pulse response unit to the initial value.

[0006] The information processing system relating to the second aspect of this disclosure comprises an imaging device that acquires image data by imaging, and a processing unit that processes the image data obtained by the imaging device. In the information processing system relating to the second aspect of this disclosure, the imaging device has a plurality of pixels arranged in a matrix, a control unit that performs TDI control on the plurality of pixels, and an output unit that outputs the image data obtained by TDI control to the processing unit. Each pixel has an optical pulse response unit that generates an optical pulse in response to incident light, and a counter unit. The counter unit includes a rewriting circuit that rewrites an initial value and an adding circuit that adds information corresponding to the optical pulse to the initial value. The control unit causes the information held in the counter unit to be written as an initial value to the counter unit included in the next row of pixels in the column direction, and then causes the information corresponding to the optical pulse obtained from the optical pulse response unit to be added to the initial value.

[0007] In the imaging apparatus relating to the first aspect of this disclosure and the information processing system relating to the second aspect of this disclosure, at each pixel, the information held in the counter unit is written as an initial value to the counter unit included in the next pixel in the column direction, and then information corresponding to the optical pulse obtained from the optical pulse response unit is added to the initial value. As a result, the information held in the counter unit after the addition is written as an initial value to the counter unit included in the next pixel in the column direction. Consequently, addition can be performed without performing AD conversion. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing.

[0008] An imaging device relating to a third aspect of the present disclosure comprises a plurality of pixels arranged in a matrix and a control unit that performs TDI control on the plurality of pixels. Each pixel has an optical pulse response unit that generates an optical pulse in response to incident light, a switch unit, and a counter unit. The switch unit connects one of a number of input terminals equal to the number of pixels in the column line to one output terminal. The counter unit integrates information corresponding to the optical pulses obtained from all optical pulse response units in the column line via the switch unit. The control unit causes the switch unit to sequentially select all optical pulse response units in the column line one by one, thereby causing the counter unit to integrate information corresponding to the optical pulses obtained from all optical pulse response units.

[0009] The information processing system relating to the fourth aspect of this disclosure comprises an imaging device that acquires image data by imaging, and a processing unit that processes the image data obtained by the imaging device. In the information processing system relating to the fourth aspect of this disclosure, the imaging device has a plurality of pixels arranged in a matrix, a control unit that performs TDI control on the plurality of pixels, and an output unit that outputs the image data obtained by TDI control to the processing unit. Each pixel has an optical pulse response unit that generates an optical pulse in response to incident light, a switch unit, and a counter unit. The switch unit connects one of a number of input terminals equal to the number of pixels in the column line to one output terminal. The counter unit integrates information corresponding to optical pulses obtained from all optical pulse response units in the column line via the switch unit. The control unit causes the switch unit to sequentially select all optical pulse response units in the column line one by one, thereby causing the counter unit to integrate information corresponding to optical pulses obtained from all optical pulse response units.

[0010] In the imaging apparatus relating to the third aspect of this disclosure and the information processing system relating to the fourth aspect of this disclosure, a switch unit sequentially selects all optical pulse response units included in the row line one by one, and the counter unit performs integration of information corresponding to the optical pulses obtained from all optical pulse response units. This allows addition processing to be performed without AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing.

[0011] An imaging device relating to a fifth aspect of this disclosure includes a plurality of pixels arranged in a matrix, and a generation unit that generates image data by performing TDI processing using information obtained from a plurality of sets of pixel lines arranged in the row direction from among the plurality of pixels. Each pixel has an optical pulse response unit that generates an optical pulse in response to incident light, and a counter unit that integrates information corresponding to the optical pulse obtained from the optical pulse response unit.

[0012] The information processing system relating to the sixth aspect of this disclosure comprises an imaging device that acquires image data by imaging, and a processing unit that processes the image data obtained by the imaging device. In the information processing system relating to the sixth aspect of this disclosure, the imaging device comprises a plurality of pixels arranged in a matrix, and a generation unit that generates image data by performing TDI processing using information obtained from a plurality of sets of pixel lines arranged in the row direction from among the plurality of pixels. Each pixel has an optical pulse response unit that generates an optical pulse in response to incident light, and a counter unit that integrates information corresponding to the optical pulse obtained from the optical pulse response unit.

[0013] In the imaging apparatus relating to the fifth aspect of this disclosure and the information processing system relating to the sixth aspect of this disclosure, information corresponding to the optical pulse obtained from the optical pulse response unit is integrated at each pixel, and image data is generated by performing TDI processing using information (integrated values) obtained from multiple sets of pixel lines arranged in the row direction from among multiple pixels. This allows TDI processing to be performed without performing AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows an example of a functional block of an imaging device according to one embodiment of the present disclosure. [Figure 2] This figure shows a schematic example of the configuration of a solid-state image sensor. [Figure 3] This figure shows an example of the circuit configuration of a pixel in Figure 1. [Figure 4] This figure shows a schematic example of the counter unit configuration shown in Figure 3. [Figure 5] Figure 1 shows how the solid-state image sensor is formed on a substrate. [Figure 6] Figure 1 is a timing diagram illustrating an example of a control method for a solid-state image sensor. [Figure 7] This figure shows a modified example of the schematic configuration of the counter section shown in Figure 3. [Figure 8] This figure shows a modified example of the schematic configuration of the counter section shown in Figure 3. [Figure 9] Figure 1 shows how the solid-state image sensor is formed on a multilayer substrate. [Figure 10] This figure shows a modified example of the pixel circuit configuration in Figure 1. [Figure 11] This figure shows a modified example of the pixel circuit configuration in Figure 1. [Figure 12] This figure shows a modified example of the pixel circuit configuration in Figure 1. [Figure 13] This figure shows a modified example of the pixel circuit configuration in Figure 1. [Figure 14]It is a timing diagram showing an example of a method for controlling a solid-state imaging device having the pixels of FIG. 13. [Figure 15] It is a diagram showing an example of the circuit configuration of the detection unit of FIG. 13. [Figure 16] It is a timing diagram showing an example of a method for controlling the detection unit of FIG. 13. [Figure 17] It is a diagram showing a modified example of the circuit configuration of the pixel of FIG. 1. [Figure 18] It is a timing diagram showing an example of a method for controlling a solid-state imaging device having the pixels of FIG. 17. [Figure 19] It is a diagram showing an example of a circuit for inputting a control signal to the detection units of FIGS. 3, 12, and 13. [Figure 20] It is a diagram showing an example of a circuit for inputting a control signal to the detection units of FIGS. 3, 12, and 13. [Figure 21] It is a diagram showing a modified example of the circuit configuration of the pixel of FIG. 1. [Figure 22] It is a timing diagram showing an example of a method for controlling a solid-state imaging device having the pixels of FIG. 21. [Figure 23] It is a diagram showing the state of counting when low-intensity light is incident. [Figure 24] It is a diagram showing the state of counting when medium-intensity light is incident. [Figure 25] It is a diagram showing the state of counting when high-intensity light is incident. [Figure 26] It is a diagram showing a modified example of the circuit configuration of the pixel of FIG. 1. [Figure 27] It is a diagram for explaining the data pattern in a solid-state imaging device having the pixels of FIG. 26. [Figure 28] It is a diagram for explaining the switch switching in a solid-state imaging device having the pixels of FIG. 26. [Figure 29] It is a diagram for explaining the data output in a solid-state imaging device having the pixels of FIG. 26. [Figure 30] It is a diagram showing a modified example of the circuit configuration of the pixel of FIG. 1. [Figure 31]This diagram illustrates the switching mechanism in a solid-state image sensor equipped with pixels, as shown in Figure 30. [Figure 32] This figure illustrates the data output in a solid-state image sensor equipped with the pixels shown in Figure 30. [Figure 33] This figure shows a modified example of the schematic configuration of the solid-state image sensor shown in Figure 1. [Figure 34] This figure shows a modified example of the schematic configuration of the solid-state image sensor shown in Figure 1. [Figure 35] This figure shows a modified example of the pixel circuit configuration in Figure 1. [Figure 36] Figure 35 is a timing diagram illustrating an example of a control method for a solid-state image sensor equipped with pixels. [Figure 37] Figures 3, 4, 7, 8, 10, 12, 13, 17, 21, 26, 30, and 35 show schematic examples of the configuration of the optical pulse response section. [Figure 38] Figures 3, 4, 7, 8, 10, 12, 13, 17, 21, 26, 30, and 35 show schematic examples of the configuration of the optical pulse response section. [Figure 39] This figure shows an example of factory automation equipped with the imaging device shown in Figure 1. [Figure 40] This figure shows an example of a biological sample analysis device equipped with the imaging device shown in Figure 1. [Modes for carrying out the invention]

[0015] The forms for implementing this disclosure will be described in detail below with reference to the drawings. The explanation will be given in the following order. 1. Embodiments (Figures 1 to 6) 2. Variations (Figures 7-38) 3. Application Examples (Figures 39, 40)

[0016] <1. Embodiment> [composition] Figure 1 shows an example of the functional block of an imaging device 100 according to one embodiment of the present disclosure. The imaging device 100 is a device that acquires image data by imaging, as shown in Figure 1, and comprises an optical system 110, a solid-state image sensor 120, a control unit 130, and a communication unit 140.

[0017] The optical system 110 focuses incident light and guides it to the solid-state image sensor 120. The solid-state image sensor 120 acquires image data through imaging, and outputs the image data obtained through imaging to the outside via the communication unit 140. The solid-state image sensor 120 has a plurality of pixels 10, which will be described later. The plurality of pixels 10 are arranged in two dimensions in the effective pixel area. The communication unit 140 is an interface for communicating with external devices and outputs the image data obtained by the solid-state image sensor 120 to the external devices.

[0018] The control unit 130 controls the solid-state image sensor 120 to acquire image data through imaging. The control unit 130 performs TDI (Time Delay Integration) control on the solid-state image sensor 120 (multiple pixels 10 described later). For example, the control unit 130 supplies the solid-state image sensor 120 with a vertical synchronization signal indicating the imaging timing and control signals φ1, φ2, and φ3 for controlling the detection unit 12 described later. For example, the control unit 130 simultaneously selects multiple pixels 10 (row lines) arranged in a row direction, and outputs the multiple pixel data obtained from the selected row lines to the next row line. At this time, the control unit 130 sequentially selects multiple row lines at a predetermined period. The selection period of the multiple row lines corresponds to the imaging period in the solid-state image sensor 120. If the imaging device 100 is equipped with a measurement unit that measures the velocity of the object to be imaged, the control unit 130 may select multiple pixels 10 (row lines) at a period based on the velocity data obtained by the measurement unit. The control unit 130 may, for example, set the size of the pixel selection period so that the object to be imaged moves by the pixel pitch in the column direction during one period of pixel selection.

[0019] Figure 2 shows a schematic example of the configuration of the solid-state image sensor 120. The solid-state image sensor 120 is equipped with a TDI sensor and acquires high-sensitivity, low-noise image data. The solid-state image sensor 120 has, for example, a pixel array section 121 and an interface section 122, as shown in Figure 2. The pixel array section 121 has a plurality of pixels 10 that perform photoelectric conversion. The plurality of pixels 10 are arranged in a matrix in the effective pixel area. In the solid-state image sensor 120, it is assumed that the object to be imaged moves at a constant speed in the column direction (up and down direction on the page of Figure 2) relative to the pixel array section 121. The interface section 122 acquires, for example, the sum of multiple pixel data obtained from a plurality of pixels 10 arranged in a column direction (column line) for each column line.

[0020] Figure 3 shows an example of the circuit configuration of pixel 10. Pixel 10 includes, for example, an optical pulse response unit 11 and a detection unit 12, as shown in Figure 3. The optical pulse response unit 11 generates optical pulses in response to incident light. The optical pulse response unit 11 includes, for example, a light receiving unit 11b and a quenching unit 11a.

[0021] The light-receiving section 11b includes an avalanche photodiode (APD). In a Geiger-mode APD, when a voltage above the breakdown voltage is applied between the terminals, the avalanche phenomenon occurs with the incidence of a single photon. An APD that multiplies a single photon by the avalanche phenomenon is called a single-photon avalanche diode (SPAD). In each pixel 10, the optical pulse response section 11 includes, for example, an SPAD. The quench section 11a has the function of stopping the avalanche phenomenon (quenching) by lowering the voltage applied to the light-receiving section 11b to the breakdown voltage. The quench section 11a further has the function of enabling the light-receiving section 11b to detect photons again by making the voltage applied to the light-receiving section 11b a bias voltage above the breakdown voltage. The quench section 11a includes, for example, a MOS transistor. The quench section 11a may also be, for example, a resistor.

[0022] One end of the quench section 11a (e.g., the source of a MOS transistor) is connected to a power line Vdd to which a fixed voltage VDD is applied. On the other hand, the other end of the quench section 11a (e.g., the drain of a MOS transistor) is connected to one end of the light-receiving section 11b (e.g., the cathode of an APD) via a signal line L1. The other end of the light-receiving section 11b (e.g., the cathode of an APD) is connected to a power line Vss to which a reference voltage Van is applied, for example. The values ​​of the fixed voltage VDD and the reference voltage Van are set so that a voltage greater than or equal to the breakdown voltage is applied to the light-receiving section 11b. The signal line L1 is connected to the other end of the quench section 11a (e.g., the drain of a MOS transistor) and one end of the light-receiving section 11b (e.g., the cathode of an APD).

[0023] In each pixel 10, the detection unit 12 includes a counter unit 12b. In each pixel 10 included in the second row and subsequent rows, the detection unit 12 includes a switch unit 12a, a counter unit 12b, a switch unit 12c, a latch unit 12d, and a switch unit 12e. In each pixel 10 included in the first row, the detection unit 12 includes a switch unit 12a, a counter unit 12b, a switch unit 12c, and a latch unit 12d. In addition, in each pixel 10 included in the first row, the detection unit 12 may further include a switch unit 12e.

[0024] One end of the switch unit 12a is connected to the signal line L1. The other end of the switch unit 12a is connected to the input terminal of the counter unit 12b (the input terminal of the adder circuit described later). The switch unit 12a controls the connection between the signal line L1 and the input terminal of the counter unit 12b. The counter unit 12b is, for example, a k-bit (k≧1) digital counter as shown in Figure 4. The digital counter is a digital circuit that includes a latch circuit and the like. The counter unit 12b includes a circuit (rewrite circuit) that rewrites the initial value based on a signal (rewrite signal) input from the latch unit 12d of the pixel 10 included in the row line of the previous stage (the previous row), and a circuit (adder circuit) that adds the input signal (addition signal) from the optical pulse response unit 11 to the initial value.

[0025] One end of the switch unit 12c is connected to the output terminal (output terminal of the adder circuit) of the counter unit 12b. The other end of the switch unit 12c is connected to the input terminal of the latch unit 12d. The switch unit 12c controls the connection between the output terminal of the counter unit 12b and the input terminal of the latch unit 12d. The latch unit 12d is a digital circuit capable of holding k bits (k≧1) of information, as shown in Figure 4, for example. The output terminal of the latch unit 12d is connected to one end of the switch unit 12e of the pixel 10 included in the next row line. The latch unit 12d outputs the information it holds to the counter unit 12b (input terminal of the rewrite circuit) of the pixel 10 included in the next row line via the switch unit 12e of the pixel 10 included in the next row line. The latch unit 12d writes the information it holds to the counter unit 12b of the pixel 10 included in the next row line as an initial value. One end of the switch unit 12e is connected to the output terminal of the latch unit 12d of the pixel 10 included in the row line of the previous stage (the row before the previous one). The other end of the switch unit 12e is connected to the input terminal (input terminal of the rewrite circuit) of the counter unit 12b.

[0026] The solid-state image sensor 120 is formed on a single semiconductor substrate 20, for example, as shown in Figure 5. At this time, an effective pixel region is formed on the surface of the semiconductor substrate 20, where multiple pixels 10 are arranged in a matrix. Around the effective pixel region on the surface of the semiconductor substrate 20, for example, an interface portion 122 is mounted.

[0027] [Operation] Next, the operation of the imaging device 100 will be described. Figure 6 is a timing diagram showing an example of a control method for the solid-state image sensor 120. The control unit 130 turns on the switch unit 12a of each pixel 10 by setting the control signal φ1 to High. As a result, photons are input from the light-receiving unit 11b, where the avalanche phenomenon is occurring, to the counter unit 12b. Subsequently, when a predetermined exposure period has elapsed, the control unit 130 turns off the switch unit 12a of each pixel 10 by setting the control signal φ1 to Low. As a result, the input of photons from the light-receiving unit 11b to the counter unit 12b stops. At this time, in the counter unit 12b, information corresponding to the number of input photons is added to the initial value, and the resulting information (retained information) is retained.

[0028] Next, the control unit 130 turns on the switch unit 12c of each pixel 10 by setting the control signal φ2 to High. As a result, the counter unit 12b outputs the information held in the counter unit 12b (held information) to the latch unit 12d via the switch unit 12c. The latch unit 12d holds the information (held information) input from the counter unit 12b. Subsequently, the control unit 130 turns off the switch unit 12c of each pixel 10 by setting the control signal φ2 to Low. Next, the control unit 130 turns on the switch unit 12e of each pixel 10 by setting the control signal φ3 to High. As a result, the latch unit 12d outputs the information held in the latch unit 12d (held information) to the counter unit 12b via the switch unit 12e. The counter unit 12b holds the information (held information) input from the latch unit 12d as an initial value.

[0029] In this way, the pixel data of the previous row line is transferred to the next row line, and the pixel data of the previous row line and the pixel data of the next row line are added together. In this manner, the control unit 130 writes the information held in the counter unit 12b as an initial value to the counter unit 12b included in the next row of pixels 10 in the column direction, and then adds the information corresponding to the optical pulse obtained from the optical pulse response unit 11 to the initial value. The control unit 130 also writes the information input from the counter unit 12b, held in the latch unit 12d, as an initial value to the counter unit 12b included in the next row of pixels 10 in the column direction.

[0030] The control unit 130 repeatedly performs this operation, causing the interface unit 122 to output the sum of the multiple pixel data obtained from multiple pixels 10 (column lines) arranged in a row direction, for each column line.

[0031] [effect] Next, the effects of the imaging device 100 will be explained.

[0032] In this embodiment, the information held in the counter unit 12b for each pixel 10 is written as an initial value to the counter unit 12b included in the next row line, and the information obtained from the optical pulse response unit 11 in response to light incidence is added to the initial value of the counter unit 12b. As a result, the information held in the counter unit 12b after the addition is written as an initial value to the counter unit 12b included in the next row line. Consequently, addition can be performed without AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. Therefore, an imaging device 100 that does not require AD conversion processing or CDS processing can be provided.

[0033] In this embodiment, the information input from the counter unit 12b, which is held in the latch unit 12d, is written as an initial value to the counter unit 12b included in the next row of pixels 10 in the column direction. By providing the latch unit 12d between the two counter units 12b in this way, image data transfer can be achieved with simple control. <2. Variant> Next, a modified example of the solid-state image sensor 120 according to the above embodiment will be described.

[0034] [Differentiation A] In the counter section 12b, the update frequency of the counter for the higher bits is low. In the switch section 12c, the switch connected to the higher bits of the counter section 12b also operates at a low frequency. In the latch section 12d, the latch connected to the higher bits of the counter section 12b also operates at a low frequency. In the switch section 12e, the switch connected to the higher bits of the counter section 12b also operates at a low frequency.

[0035] Therefore, in the above embodiment, the higher bits of the counter unit 12b and the parts of the switch unit 12c, latch unit 12d, and switch unit 12e connected to the higher bits of the counter unit 12b may be formed outside the formation area of ​​the pixel array unit 121 (outside array area R2) of the solid-state image sensor 120, for example, as shown in Figure 7. In this case, the bits other than the higher bits of the counter unit 12b and the parts of the switch unit 12c, latch unit 12d, and switch unit 12e connected to the bits other than the higher bits of the counter unit 12b may be formed in the formation area of ​​the pixel array unit 121 (array area R1) of the solid-state image sensor 120, for example, as shown in Figure 7. This is because the operating frequency of the higher bits of the counter unit 12b is low, so even if there is a delay in the signal controlling the higher bits of the counter unit 12b, that delay may not be a problem. Furthermore, by forming the higher bits of the counter unit 12b etc. in the outside array area R2 in this way, the area of ​​the array area R1 can be reduced.

[0036] [Variation B] In multiple pixels 10 (column lines) arranged in a column direction, the counter value of the counter unit 12b increases as you move toward the later stages of the column line. Therefore, the counter value of the counter unit 12b may saturate toward the later stages of the column line.

[0037] Therefore, in the above embodiment and its modified form, the counter unit 12b, switch unit 12c, latch unit 12d, and switch unit 12e may be configured such that the number of bits is larger in the later stages of the column line compared to the first stage of the column line, as shown in Figure 8. By doing so, it is possible to prevent saturation of the counter unit 12b while keeping the circuit size down, compared to the case where the number of bits is increased for the entire column line.

[0038] [Differentiation C] In the above embodiment and its modified form, the solid-state image sensor 120 may be formed dispersed on a plurality of stacked semiconductor substrates. The solid-state image sensor 120 may be formed dispersed on two stacked semiconductor substrates 21 and 22, for example, as shown in Figure 9. In this case, the portion of each pixel 10 other than the light-receiving portion 11b (hereinafter referred to as "circuit 21a") may be formed on the first semiconductor substrate 21. Also, the portion of each pixel 10 including the light-receiving portion 11b (hereinafter referred to as "circuit 22a") may be formed on the second semiconductor substrate 22. Circuit 21a and circuit 22a may be electrically connected, for example, by joining a pad electrode 21b connected to circuit 21a and a pad electrode 22b connected to circuit 22a to each other.

[0039] [Differentiation D] In the above embodiment and its modified form, the optical pulse response unit 11 of each pixel 10 may have, for example, a plurality of sub-pixels 11-1 and an adder 11-2 connected to the output of the plurality of sub-pixels 11-1, as shown in Figure 10.

[0040] Each sub-pixel 11-1 has a quenching unit 11a, a light-receiving unit 11b, and a pulse generation unit 11c. The pulse generation unit 11c is connected to the signal line L1, and photons output from the light-receiving unit 11b are input to the pulse generation unit 11c. The pulse generation unit 11c generates pulses in response to the photons input from the light-receiving unit 11b and outputs them to the summing unit 11-2. The summing unit 11-2 counts the pulses input from multiple sub-pixels 11-1 and outputs a number of pulses corresponding to the count value obtained to the detection unit 12. In this way, a highly sensitive image can be obtained by using multiple sub-pixels 11-1 and the summing unit 11-2.

[0041] [Differentiation Example E] In the modified example D described above, the pulse generation unit 11c may be omitted, for example, as shown in Figure 11. In this case, the adder 11-2 counts the photons input from each sub-pixel 11-1 and outputs a number of pulses corresponding to the count value obtained to the detection unit 12. Even in this case, a highly sensitive image can be obtained by using the pixel data obtained by the adder 11-2.

[0042] [Modification F] In the row lines of the above embodiment and its modified form, for example, as shown in Figure 12, the latch unit 12d and the switch unit 12c may be omitted for each pixel 10 in odd-numbered lines. Even in this case, the control unit 130 can control the solid-state image sensor 12 at the timing shown in Figure 6, for example, to provide an imaging device 100 that does not require AD conversion processing or CDS processing.

[0043] [Differentiation G] In the above embodiment and its modified form, for example, as shown in Figure 13, a detection unit 12A may be provided in all pixels 10 in which the latch unit 12d and the switch unit 12c are omitted. In this case, the control unit 130 can provide an imaging device 100 that does not require AD conversion processing or CDS processing by inputting control signals φ2_0, φ2_1, φ2_2, and φ2_3 to the solid-state image sensor 12 at the timings shown in Figure 14, for example.

[0044] Figure 15 shows an example of the circuit configuration of the detection unit 12A. The detection unit 12A is composed of a plurality of 1-bit counters connected in series, a multiplexer (hereinafter referred to as the "first multiplexer") that controls the input of control signals (SHIFT_PULSE, PC_PULSE_i (i=A,B…)) to the plurality of 1-bit counters, and a NOT gate.

[0045] A 1-bit counter is configured, for example, with a D flip-flop consisting of multiple (two) D latches and a multiplexer (hereinafter referred to as the "second multiplexer") connected to the data input terminal of the D flip-flop. In other words, the detection unit 12A has a D flip-flop for each bit.

[0046] The output of the first multiplexer is input as the clock for the D latch preceding the D flip-flop. The inverted output of the first multiplexer (the output of the NOT gate) is input as the clock for the D latch following the D flip-flop. The first multiplexer receives input from the output of the D flip-flop in the same stage (same row) as the first multiplexer, and from the output of the D flip-flop included in the row line of the preceding stage (previous row). The control unit 130 outputs a control signal (SHIFT_PULSE, PC_PULSE_i (i=A,B…)) to the detection unit 12A at the timing shown in Figure 16, for example. At this time, it can be seen that the D flip-flop provided for each 1-bit counter also serves as the latch. This makes it possible to suppress an increase in circuit size.

[0047] [Modified form H] In the above embodiment and its modified form, each pixel 10 may have a plurality of detection units 12. Each pixel 10 may have, for example, two detection units 12 and an adder 16 that adds the output values ​​of the two detection units 12, as shown in Figure 17.

[0048] At this time, the two detection units 12 are connected to a common optical pulse response unit 11, and in multiple pixels 10 included in a row line, one detection unit 12 is connected in a row, and the other detection unit 12 is also connected in a row. While the control unit 130 causes the information held in one detection unit 12 (hereinafter referred to as "detection unit 12L") to be written as an initial value to the detection unit 12L included in the next row of pixels 10 in the row direction, the control unit 130 causes the information held in the other detection unit 12 (hereinafter referred to as "detection unit 12R") to be written as an initial value to the detection unit 12R included in the next row of pixels 10 in the row direction.

[0049] The control unit 130 performs exposure in a pre-stage and post-stage in one line cycle, for example, as shown in Figure 18. The control unit 130 sequentially outputs control signals φ2 and φ3 to one of the detection units 12 in a plurality of pixels 10 included in the row line, for example, after the pre-stage exposure (during the post-stage exposure). The control unit 130 further outputs control signals φ2' and φ3' to the other detection unit 12 in a plurality of pixels 10 included in the row line, for example, after the post-stage exposure (during the pre-stage exposure). By doing this, the unexposed time can be significantly reduced compared to the case where exposure is performed only in the pre-stage in one line cycle, as in the above embodiment. As a result, a more sensitive imaging device 100 can be provided.

[0050] [Modification I] In the above embodiment and its modifications, in order to improve the waveform distortion of the control signals (e.g., control signals φ1, φ2, φ3) output from the control unit 130 to each detection unit 12 due to wiring resistance, etc., a repeater 31 may be provided at a predetermined location in the wiring, for example, as shown in Figure 19. The repeater 31 is inserted into the wiring through which the control signals input from the control unit 130 to the plurality of pixels 10 propagate. Alternatively, a shift register 32 may be provided at a predetermined location in the wiring instead of the repeater 31, for example, as shown in Figure 20. By providing the repeater 31 or the shift register 32 in this way, control pulses that are independent of the distance from the control unit 130 to each detection unit 12 can be input to the detection unit 12. As a result, the exposure time at each pixel 10 can be made uniform.

[0051] [Modification J] In the above embodiment and its modified form, each detection unit 12 may have an OF determination unit 12h that determines whether or not the counter unit 12b is saturated, as shown in Figure 21, for example. The OF determination unit 12h determines that the counter unit 12b is not saturated when the count value of the counter unit 12b is below a predetermined threshold, and outputs, for example, a Low-level FLAG (=0). The OF determination unit 12h determines that the counter unit 12b is saturated when the count value of the counter unit 12b exceeds a predetermined threshold, and outputs, for example, a High-level FLAG (=1).

[0052] In each pixel 10 included in the row lines from the second row onward, each detection unit 12 includes, for example, a switch unit 12a, 12c, 12e, 12g, 12i, a multiplexer 12f, a counter unit 12b, an OF determination unit 12h, and a latch unit 12d, 12j, as shown in Figure 21. In each pixel 10 included in the row line of the first row, the detection unit 12 includes, for example, a switch unit 12a, 12c, 12i, a multiplexer 12f, a counter unit 12b, an OF determination unit 12h, and a latch unit 12d, 12j, as shown in Figure 21.

[0053] One end of the switch unit 12a is connected to the signal line L1. The other end of the switch unit 12a is connected to the input terminal of the multiplexer 12f. The switch unit 12a controls the connection between the signal line L1 and the input terminal of the multiplexer 12f. The input terminal of the multiplexer 12f is connected to the other end of the switch unit 12a and to the wiring to which the clock CLK is input. The output terminal of the multiplexer 12f is connected to the input terminal of the counter unit 12b (the input terminal of the adder circuit).

[0054] One end of the switch unit 12i is connected to the output terminal of the OF determination unit 12h (the output terminal of the adder circuit described later). The other end of the switch unit 12i is connected to the input terminal of the latch unit 12j. The switch unit 12i controls the connection between the output terminal of the OF determination unit 12h (the output terminal of the adder circuit) and the input terminal of the latch unit 12j. The latch unit 12j is a digital circuit capable of holding, for example, m bits (m≧1) of information. The output terminal of the latch unit 12j is connected to one end of the switch unit 12g of the pixel 10 included in the next row line. The latch unit 12j outputs the information it holds to the OF determination unit 12h of the pixel 10 included in the next row line via the switch unit 12g of the pixel 10 included in the next row line. The latch unit 12d writes the information it holds to the counter unit 12b of the pixel 10 included in the next row line as an initial value. One end of the switch unit 12g is connected to the output terminal of the latch unit 12j of the pixel 10 included in the row line of the previous stage (the row before the previous one). The other end of the switch unit 12g is connected to the input terminal (input terminal of the rewrite circuit) of the OF determination unit 12h.

[0055] When the multiplexer 12f receives a Low-level FLAG from the OF detection unit 12h, for example, it outputs the photons input via the switch unit 12a to the counter unit 12b. At this time, the counter unit 12b performs photon counting. When the multiplexer 12f receives a High-level FLAG from the OF detection unit 12h, for example, it outputs the clock CLK input from the wiring to the counter unit 12b. At this time, the counter unit 12b performs clock CLK counting. When the counter unit 12b receives a FLAG that shifts from a Low level to a High level from the OF detection unit 12h, for example, it resets the counter and starts counting clock CLK. When the counter unit 12b receives a FLAG that shifts from a High level to a Low level from the OF detection unit 12h, for example, it resets the counter and starts counting photons.

[0056] The OF determination unit 12h outputs, for example, a Low-level FLAG to the multiplexer 12f, the counter unit 12b, and the switch unit 12i when the count value of the counter unit 12b is below a predetermined threshold. The OF determination unit 12h outputs, for example, a High-level FLAG to the multiplexer 12f, the counter unit 12b, and the switch unit 12i when the count value of the counter unit 12b exceeds a predetermined threshold.

[0057] The OF determination unit 12h further determines the saturation of the counter unit 12b based on information input from the latch unit 12j of the pixel 10 included in the row line of the previous stage (previous row) via the switch unit 12g. For example, if a Low level FLAG is input via the switch unit 12g, the OF determination unit 12h determines that the counter unit 12b is not yet saturated in the previous stage (the row before it) and outputs a Low level FLAG to the multiplexer 12f, the counter unit 12b, and the switch unit 12i. For example, if a High level FLAG is input via the switch unit 12g, the OF determination unit 12h determines that the counter unit 12b is already saturated in the previous stage (the row before it) and outputs a High level FLAG to the multiplexer 12f, the counter unit 12b, and the switch unit 12i.

[0058] Figure 22 is a timing diagram showing an example of a control method for a solid-state image sensor 120 equipped with the pixels 10 shown in Figure 21. The control unit 130 turns on the switch unit 12a of each pixel 10 by setting the control signal φ1 to High. As a result, photons are input from the light-receiving unit 11b, where the avalanche phenomenon is occurring, to the counter unit 12b. Subsequently, when a predetermined exposure period has elapsed, the control unit 130 turns off the switch unit 12a of each pixel 10 by setting the control signal φ1 to Low. As a result, the input of photons from the light-receiving unit 11b to the counter unit 12b stops. At this time, in the counter unit 12b, information corresponding to the number of input photons is added to the initial value, and the resulting information (retained information) is retained.

[0059] Next, the control unit 130 turns on the switch units 12c and 12i of each pixel 10 by setting the control signal φ2 to High. As a result, the counter unit 12b outputs the information held in the counter unit 12b (held information) to the latch unit 12d via the switch unit 12c. The latch unit 12d holds the information (held information) input from the counter unit 12b. Furthermore, the OF determination unit 12h outputs the information held in the OF determination unit 12h (held information) to the latch unit 12j via the switch unit 12i. The latch unit 12j holds the information (held information) input from the OF determination unit 12h.

[0060] Subsequently, the control unit 130 turns off the switch units 12c and 12i of each pixel 10 by setting the control signal φ2 to low. Next, the control unit 130 turns on the switch units 12e and 12g of each pixel 10 by setting the control signal φ3 to high. As a result, the latch unit 12d outputs the information held in the latch unit 12d (held information) to the counter unit 12b via the switch unit 12e. The counter unit 12b holds the information (held information) input from the latch unit 12d as an initial value. The latch unit 12j outputs the information (held information) held in the latch unit 12j to the OF determination unit 12h via the switch unit 12g. Based on the information (held information) input from the latch unit 12j, the OF determination unit 12h determines whether the counter unit 12b has already saturated in the previous stage (the previous row). If the counter unit 12b becomes saturated, the OF determination unit 12h shifts the FLAG from a low level (=0) to a high level (=1), for example. Then, the counter unit 12b resets the counter and starts counting the clock CLK.

[0061] In this way, the pixel data of the previous row line is transferred to the next row line, and the pixel data of the previous row line and the pixel data of the next row line are added together. The control unit 130 repeatedly performs this operation, causing the interface unit 122 to output the sum of the multiple pixel data obtained from multiple pixels 10 (column lines) arranged in the column direction, for each column line.

[0062] Here, let's assume that the value ultimately transferred to the interface unit 122 for each column (the sum of multiple pixel data obtained from the column line) is Nc. In this case, the interface unit 122 changes how it handles Nc depending on whether the FLAG is at a low level (=0) or a high level (=1). If the FLAG is at a low level (=0), the interface unit 122 outputs Nc as an integrated value N to the outside via the communication unit 140. On the other hand, if the FLAG is at a high level (=1), the interface unit 122 outputs the value obtained using the following formula as an integrated value N to the outside via the communication unit 140.

[0063] N = Nc × Texp / Tc Tc = Tclk × Nc Nc: Count value in counter unit 12b Texp: Exposure time (μs) during one frame period Tc: Exposure time (remaining time) after the counter unit 12b has saturated (μs) Tclk: Period of the clock CLK (μs)

[0064] As described above, in this modified example, the exposure time (remaining time) after the counter unit 12b has saturated is measured by the counter unit 12b counting the clock CLK. The saturation timing of the counter unit 12b is accelerated as the illuminance of the incident light increases, for example, as shown in Figures 23, 24, and 25. However, after the counter unit 12b has saturated (remaining time), the counter unit 12b counts the clock CLK instead of counting photons. The count value of the clock CLK in the counter unit 12b increases as the illuminance of the incident light increases. Also, the count value of the clock CLK increases as the remaining time increases. Therefore, since the illuminance of the incident light is proportional to the remaining time, the sum of the pixel data of the column lines obtained assuming that the counter unit 12b did not saturated can be obtained by dividing the exposure time Texp in one frame period by the remaining time Tc. From the above, it can be shown that the dynamic range of the detection unit 12 can be expanded by counting the clock CLK during the remaining time.

[0065] In this modified example, instead of starting to count photons when the FLAG reaches a high level (=1), the counter unit 12b may record (hold) the time code at the time the FLAG reached a high level (=1). In this case, the OF determination unit 12h determines whether the counter unit 12b has saturated, and if it determines that the counter unit 12b has saturated, it controls the counter unit 12b to record the time code at that time. The interface unit 122 calculates the remaining time using the time codes transferred for each column. Even in this case, the dynamic range of the detection unit 12 can be expanded by calculating the remaining time using the time code.

[0066] [Differentiation K] In the above embodiment and its modified form, for example, as shown in Figure 26, a detection unit 13 may be provided instead of the detection unit 12. The detection unit 13 has a switch unit 13a and a counter unit 13b.

[0067] The switch unit 13a has an input terminal equal to the number of pixels 10 included in the row line. Figure 26 illustrates the arrangement with eight input terminals. In the switch unit 13a, each input terminal is connected to different optical pulse response units 11 included in the row line. The output terminal of the switch unit 13a is connected to the input terminal of the counter unit 13b. The counter unit 13b is, for example, a k-bit (k≧1) digital counter. The digital counter is a digital circuit configured to include a latch circuit, etc. The counter unit 13b includes a circuit (integration circuit) that integrates the input signals from the switch unit 13a. The output terminal of the counter unit 13b is connected to the interface unit 122.

[0068] Figure 27 conceptually illustrates an example of switching data patterns for each horizontal period in the imaging area of ​​the solid-state image sensor 120. In Figures 26 and 27, p0 to p7 are codes assigned to multiple optical pulse response units 11 included in the column line. Figure 27 suggests that multiple objects arranged in the column direction move in the column direction at a speed equal to the line rate within the imaging area.

[0069] Figure 28 conceptually illustrates an example of switch switching for each horizontal period of the counter unit 13b. In this modified example, the control unit 130 outputs control signals φa to φh to the column line so that each switch unit 13a included in the column line sequentially selects one by one all of the optical pulse response units 11 included in the column line, as shown in Figure 28, during one frame period. By causing the switch unit 13a to sequentially select one by one all of the optical pulse response units 11 included in the column line, the control unit 130 causes the counter unit 13b to perform the integration of information corresponding to the optical pulses obtained from all of the optical pulse response units 11.

[0070] Figure 29 conceptually represents the pixel data output from each counter unit 13b included in the column line as a result of the switch switching shown in Figure 28. In Figure 29, each counter unit 13b included in the column line outputs information (retained information) corresponding to the number of photons input from all optical pulse response units 11 included in the column line to the interface unit 122 at the hatched timing. This retained information corresponds to the sum of multiple pixel data obtained from the column line. Note that at the timing when there is no hatching in Figure 29, each counter unit 13b included in the column line outputs information (retained information) corresponding to the number of photons input from some of the optical pulse response units 11 included in the column line to the interface unit 122.

[0071] In this modified example, each counter unit 13b included in the column line acquires the sum of pixel data obtained from each pixel 10 included in the column line and outputs it to the interface unit 122. This allows the addition process in the column line to be performed by each counter unit 13b included in the column line. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. Therefore, it is possible to provide an imaging device 100 that does not require AD conversion processing or CDS processing.

[0072] [Modified version L] In the modified example K described above, the solid-state image sensor 120 may be arranged in a row direction with multiple column lines, each containing a plurality of pixels 10, each equipped with a detection unit 13, as shown in Figure 30. In this case, the solid-state image sensor 120 may be provided with a transfer unit 14 for each pixel 10 included in the first row line, which transfers the sum of the pixel data obtained in the first row line as the initial value for the detection unit 13 (counter unit 13b) of the next row line.

[0073] The transfer unit 14 includes, for example, two switch units 14a and 14c and a latch unit 14b.

[0074] One end of the switch unit 14a is connected to the output terminal of the detection unit 13 of the corresponding pixel 10. The other end of the switch unit 14a is connected to the input terminal of the latch unit 14b. The switch unit 12i controls the connection between the output terminal of the detection unit 13 of the corresponding pixel 10 and the input terminal of the latch unit 14b. The latch unit 14b is, for example, a digital circuit capable of holding m bits (m≧1) of information. The output terminal of the latch unit 14b is connected to one end of the switch unit 14c. The latch unit 14b outputs the information it holds as an initial value to the detection unit 13 (counter unit 13b) of the corresponding pixel 10 included in the next row line via the switch unit 14c. One end of the switch unit 14c is connected to the output terminal of the latch unit 14b. The other end of the switch unit 14c is connected to the detection unit 13 (input terminal of the rewrite circuit of the counter unit 13b) of the corresponding pixel 10 included in the next row line.

[0075] In this modified example, the counter unit 13b included in the detection unit 13 is, for example, a k-bit (k≧1) digital counter. The digital counter is a digital circuit configured to include a latch circuit and the like. The counter unit 13b includes a circuit (rewrite circuit) that rewrites the initial value based on a signal (rewrite signal) input from the latch unit 14b included in the corresponding transfer unit 14, and a circuit (adder circuit) that adds an input signal (addition signal) from the optical pulse response unit 11 to the initial value.

[0076] Figure 31 conceptually illustrates an example of switch switching for each horizontal period of the counter unit 13b. In this modified example, the control unit 130 outputs control signals φa to φd to each column line, for example, as shown in Figure 31, so that each switch unit 13a included in the column line sequentially selects one by one all the optical pulse response units 11 included in the column line during one frame period.

[0077] Figure 32 conceptually represents the pixel data output from each counter unit 13b included in the column line as a result of the switch switching shown in Figure 31. In Figure 32, each counter unit 13b included in the column line outputs information (holding information) corresponding to the number of photons input from all optical pulse response units 11 included in the column line to the transfer unit 14 (interface unit 122 in the final stage) at the hatched timing. This holding information corresponds to the sum of multiple pixel data obtained from the column line. The transfer unit 14, for example, as shown by the arrow in Figure 32, transfers the sum of the pixel data obtained in the first row line as the initial value for the detection unit 13 (counter unit 13b) of the next row line.

[0078] In this modified example, the information held in the counter unit 13b in each column line is written as an initial value to the counter unit 13b included in the next row line, and the information obtained from the optical pulse response unit 11 in response to light incidence is added to the initial value of the counter unit 13b. As a result, the information held in the counter unit 13b after the addition is written as an initial value to the counter unit 13b included in the next row line. Consequently, addition can be performed without performing AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. Therefore, an imaging device 100 that does not require AD conversion processing or CDS processing can be provided.

[0079] [Differentiation M] In the above embodiment and its modified form, the solid-state image sensor 120 may have a plurality of interface units 122, for example, as shown in Figure 33. Each interface unit 122 processes, for example, the data of multiple column lines assigned to each interface unit 122 (the sum of multiple pixel data obtained from the column lines). In this case, the processing speed of all column line data is improved compared to when one interface unit 122 processes the data of all column lines.

[0080] [Differentiation N] In the above embodiment and its modified form, the solid-state image sensor 120 may have, for example, a pixel array unit 123, a TDI adder unit 124, a frame memory unit 125, and an interface unit 126, as shown in Figure 34.

[0081] The pixel array section 123 has a plurality of pixels 10C that perform photoelectric conversion. The plurality of pixels 10C are arranged in a matrix in the effective pixel area. The solid-state image sensor 120 is based on the premise that the object to be imaged moves at a constant speed in the column direction (vertical direction in the plane of the paper in Figure 34) relative to the pixel array section 123.

[0082] Figure 35 shows an example of the circuit configuration of a pixel 10C. In this modified example, the solid-state image sensor 120 is provided with multiple vertical signal lines VSL, one assigned to each of the multiple pixels 10C (column lines) arranged in a row. The pixel 10C has, for example, an optical pulse response unit 11 and a detection unit 12C, as shown in Figure 35. The optical pulse response unit 11 generates an optical pulse in response to incident light. The optical pulse response unit 11 has, for example, the same configuration as in the above embodiment and its modified examples.

[0083] In each pixel 10C, the detection unit 12C includes, for example, a switch unit 12a, a counter unit 12b, and a switch unit 12k. One end of the switch unit 12a is connected to the signal line L1 of the optical pulse response unit 11. The other end of the switch unit 12a is connected to the input terminal of the counter unit 12b (the input terminal of the integration circuit described later). The switch unit 12a controls the connection between the signal line L1 and the input terminal of the counter unit 12b. The counter unit 12b is, for example, a k-bit (k≧1) digital counter as shown in Figure 4. The digital counter is a digital circuit configured to include a latch circuit, etc. The counter unit 12b includes a circuit (integration circuit) that integrates the input signals from the optical pulse response unit 11. One end of the switch unit 12c is connected to the output terminal of the counter unit 12b (the output terminal of the integration circuit). The other end of the switch unit 12c is connected to the vertical signal line VSL and is connected to the TDI adder 124 via the vertical signal line VSL.

[0084] In this modified example, the control unit 130 performs TDI control on the solid-state image sensor 120 (multiple pixels 10C, TDI adder 124, frame memory 125, interface unit 126). The control unit 130 simultaneously selects multiple pixels 10C (row lines) arranged in a row direction, and outputs the multiple pixel data obtained from the selected row lines to the TDI adder 124 via multiple vertical signal lines VSL. At this time, the control unit 130 sequentially selects multiple row lines at a predetermined cycle.

[0085] In this modified example, the TDI adder 124, frame memory unit 125, and interface unit 126 generate image data by performing TDI processing using multiple pulse signals obtained from multiple sets of pixel lines (row lines) arranged in a row direction from among multiple pixels 10C. The TDI adder 124 acquires multiple pixel data (multiple pixel data of the i-th row) input from the i-th row line selected by the control unit 130 via multiple vertical signal lines VSL. The TDI adder 124 writes the acquired multiple pixel data of the i-th row to the memory at a predetermined row address (i-th row) in the frame memory unit 125. For example, each time the TDI adder 124 acquires multiple pixel data, it writes the acquired multiple pixel data to the row address of the next line in the frame memory unit 125 after the row address to which it was previously written. If the TDI adder 124 previously wrote to the row address of the last row (n-th row), it writes the acquired multiple pixel data to the memory at the row address of the first row (1st row). The TDI summing unit 124 performs this writing process until it has acquired multiple pixel data for the number of row lines.

[0086] The TDI adder 124, each time it acquires pixel data from all pixels 10C in the pixel array 123, shifts the location where the acquired i-th row of pixel data is added to the frame memory 125 by m rows (for example, 1 row). The TDI adder 124 adds the acquired i-th row of pixel data to a memory at a predetermined row address (i + t × m-th row) in the frame memory 125. t corresponds to the number of times pixel data has been acquired from all pixels 10C. When t=n, the sum of multiple (n) pixel data obtained from the column line is stored in each memory of the frame memory 125. When t=n, the interface 126 acquires one frame's worth of pixel data (i.e., image data) from the frame memory 125. The interface 126 outputs the acquired image data to the outside via the communication unit 140.

[0087] (operation) Next, the operation of the imaging device 100 according to this modified example will be described. Figure 36 is a timing diagram showing an example of a solid-state image sensor 120 control method according to this modified example. The control unit 130 turns on the switch unit 12a of each pixel 10C by setting the control signal φ1 to High. As a result, photons are input from the light-receiving unit 11b, where the avalanche phenomenon is occurring, to the counter unit 12b. Subsequently, when a predetermined exposure period has elapsed, the control unit 130 turns off the switch unit 12a of each pixel 10C by setting the control signal φ1 to Low. As a result, the input of photons from the light-receiving unit 11b to the counter unit 12b stops. At this time, the counter unit 12b accumulates information corresponding to the number of input photons, and the information obtained therefrom (retained information) is retained.

[0088] Next, the control unit 130 sequentially turns the switch unit 12c of each pixel 10C on and off for each row line by sequentially changing the control signals φ3_0 to φn-1 one by one from High to Low during the exposure cycle period (one scan period). As a result, the counter unit 12b outputs the information held in the counter unit 12b (held information) to the TDI adder 124 via the switch unit 12c. The TDI adder 124 writes the acquired multiple pixel data of the i-th row to the memory at a predetermined row address (i-th row) in the frame memory unit 125. For example, each time the TDI adder 124 acquires multiple pixel data from a row line where the switch unit 12c has been turned on or off, it writes the acquired multiple pixel data to the row address of the next row in the frame memory unit 125 after the row address to which it was previously written. If the TDI adder 124 previously wrote to the row address of the last row (n-th row), it writes the acquired multiple pixel data to the memory at the row address of the first row (1st row). The TDI adder 124 performs this writing process until all the switch units 12c have finished switching on and off.

[0089] Subsequently, the control unit 130 causes exposure at each light-receiving unit 11b by changing the control signal φ1 from High to Low each time a scan period has elapsed. During each scan period, the control unit 130 sequentially turns the switch unit 12c of each pixel 10C on and off for each row line by sequentially changing the control signals φ3_0 to φn-1 one by one from High to Low. At this time, the TDI adder 124 shifts the location where the acquired i-th row of pixel data is added to the frame memory unit 125 by m rows (for example, 1 row) during each scan period. The TDI adder 124 adds, for example, the acquired i-th row of pixel data to the memory at a predetermined row address (i+t×m row) in the frame memory unit 125. When t=n, the control unit 130 instructs the interface unit 126 to acquire one frame's worth of pixel data (i.e., image data). The control unit 130 instructs the interface unit 126 to output the acquired image data to the outside via the communication unit 140.

[0090] (effect) In this modified example, the information held in the counter unit 12b for each pixel 10C is sequentially written to the frame memory unit 125 line by line, and after each scan period, it is added to the frame memory unit 125 with a shift of m lines (for example, 1 line). This allows addition processing to be performed without AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. Therefore, it is possible to provide an imaging device 100 that does not require AD conversion processing or CDS processing.

[0091] [Modification O] In the above embodiment and its modified form, the optical pulse response unit 11 may have, for example, an active precharge type circuit as shown in Figure 37. The optical pulse response unit 11 shown in Figure 37 includes a light receiving unit 11b, transistors 41, 42, 43, 44, 45, an inverter 46, and a pulse generator 47.

[0092] The pulse generator 47 has a delay element D3 and a NAND circuit NP as internal components. Transistors 41, 42, and 43 are PMOS transistors. Transistors 44 and 45 are NMOS transistors. The gates of transistors 41, 42, 44, and 45 are connected to the output terminal of the NAND circuit NP via signal lines. The sources of transistors 41 and 42 are connected to the power line Vdd. The drain of transistor 41 is connected to the gate of transistor 43, the source of transistor 45, and one end of the light-receiving section 11b (e.g., the cathode of the photodiode) via signal lines. The drains of transistors 44 and 45 are connected to the ground line. The source of transistor 43 is connected to the drain of transistor 42. The drain of transistor 43 is connected to the input terminal of inverter 46 and the source of transistor 44 via signal lines. The output terminal of inverter 46 is connected to signal line L1. The delay element D3 is connected to signal line L1 and one input terminal of the NAND circuit NP. The other input terminal of the NAND circuit NP is connected to terminal xRST. The pulse generator 47 is not limited to the above configuration and may have a different configuration.

[0093] Next, the operation of the optical pulse response unit 11 in this modified example will be described. When the light receiving unit 11b reacts with a photon, the current between the cathode and anode increases, the current between the source and drain of transistor 41 increases, and the voltage drop between the source and drain of transistor 41 causes the gate voltage of transistor 43 to go low. As a result, both transistors 41 and 43 turn on. Consequently, the current flowing between the source and drain of transistor 42 increases. Therefore, the gate-source voltage of transistor 42 increases due to the Id-Vgs characteristic.

[0094] In other words, almost simultaneously with the source-drain connection of transistor 43 turning on, the gate-source connection of transistor 42 also turns on. Because both transistors 42 and 43 are turned on, the voltage at the input terminal of inverter 46 is raised to High by the power supply line Vdd. When a High voltage is input to inverter 46, it outputs a Low voltage to signal line L1. Thus, in the optical pulse response unit 11 according to this modified example, a Low (negative polarity) pulse is output to signal line L1 when a photon is detected.

[0095] When the voltage across signal line L1 goes low, the voltage at one input terminal of the NAND gate NP also goes low with a delay. As a result, the NAND gate NP outputs a high voltage to the gates of transistors 44 and 55. With a high voltage applied to the gates of transistors 44 and 55, the source-drain connections of transistors 44 and 55 turn on. Consequently, the cathode of the light-receiving unit 11b is connected to ground potential, and the avalanche phenomenon stops. In this way, the optical pulse response unit 11 is a circuit that performs active quenching.

[0096] Furthermore, in transistor 42, the source-drain connection is turned off, suppressing through-current from the power supply to ground. Consequently, the voltage at the input terminal of inverter 46 is initialized by the potential of the ground line and becomes low. When the voltage at the input terminal of inverter 46 becomes low, inverter 46 outputs a high voltage to signal line L1. As a result, with a delay due to the delay element D3, the voltage at one input terminal of NAND circuit NP becomes high. Furthermore, when a high voltage is applied to terminal xRST, NAND circuit NP outputs a low voltage. At this time, a low voltage is applied to the gate of transistor 41, so the source-drain connection of transistor 41 turns on. As a result, the potential of the cathode of the light-receiving unit 11b is raised by the voltage of the power supply line Vdd. Since the terminal voltage of the light-receiving unit 11b becomes above the breakdown voltage, the optical pulse response unit 11 can detect photons again.

[0097] In the modified optical pulse response unit 11, when a signal corresponding to a photon is output from the signal line L1, the cathode of the light receiving unit 11b is connected to ground potential, enabling high-speed quenching. Furthermore, since the transistor 41 is also driven by the pulse, the cathode voltage of the light receiving unit 11b is raised in a shorter time compared to when a current source transistor is used. Additionally, the modified optical pulse response unit 11 can output pulses of a constant width to subsequent circuits regardless of the recharge time. Moreover, the modified optical pulse response unit 11 exhibits good input / output characteristics at high illuminance levels.

[0098] [Modified Version P] In the above embodiment and its modified form, the optical pulse response unit 11 may have, for example, an active recharge type circuit as shown in Figure 38. The optical pulse response unit 11 shown in Figure 38 includes a light receiving unit 11b, a transistor 51, an amplifier 52, a count value holding unit 53, and a readout unit 54. The circuit consisting of the transistor 51, amplifier 52, count value holding unit 53, and readout unit 54 corresponds to the quench unit 11a.

[0099] The drain of transistor 51 is connected to the power supply voltage VR, a reset pulse signal is applied to the gate of transistor 51, and the source of transistor 51 is connected to the cathode of the light receiving unit 11b. The anode of the light receiving unit 11b is connected to the ground line. The cathode of the light receiving unit 11b is connected to the input terminal of amplifier 52, and the output terminal of amplifier 52 is connected to one input terminal (clock input terminal) of the count value holding unit 53. An initial setting pulse signal is applied to the other input terminal (initial setting terminal) of the count value holding unit 53.

[0100] When transistor 51 is off, if a photon is incident on the light-receiving unit 11b, the cathode potential of the light-receiving unit 11b decreases due to the photoelectric conversion action. Next, when transistor 51 is turned on by a reset pulse signal applied to the gate of transistor 51, the cathode potential of the light-receiving unit 11b rises to the power supply reduction VR. This rise in the power supply reduction VR is amplified by amplifier 52 and input to count value holding unit 53.

[0101] The count value holding unit 53 holds the counted value obtained from the pulses received from the amplifier 52 as follows. For example, a reset pulse signal is applied to the transistor 51 at an arbitrary period. In this case, suppose that multiple incident photons are incident on the light receiving unit 11b for a period longer than the period of the reset pulse signal (measurement period). In this case, if the measurement period is divided by the period of the reset pulse signal, information on the presence or absence of incident photons can be obtained for each section. The information on the presence or absence of incident photons is not about how many reset pulse signals there are, but rather whether or not there was a reset pulse signal. In sections where there were no incident photons, even if the transistor 51 is turned on, no pulse is output from the amplifier 52. Therefore, if there was one or more incident photons, the count value of the count value holding unit 53 will be the previously held count value plus 1. If there were no incident photons, the count value of the count value holding unit 53 will remain the previously held count value. When a predetermined period of time has elapsed (for example, 1 / 60 sec, 1 / 120 sec, etc.), the reading unit 54 reads the count value held in the count value holding unit 53 from the count value holding unit 53 and outputs it to the detection unit 12.

[0102] In this modified example, photons are counted by periodic reset in the optical pulse response unit 11. This makes the coefficient value output an asymptote, and the range in which the monotonically increasing coefficient value output can be maintained regardless of how much the amount of incident light increases can be greatly expanded. As a result, a dynamic range greater than the number of bits in the count value holding unit 53 can be obtained.

[0103] In the above embodiment and its modified form, the optical pulse response unit 11 may have a passive recharge type circuit.

[0104] [Differentiation Example Q] In the above embodiment and its modified form, the multiple light-receiving units 11b included in the column lines of the pixel array units 121 and 123 may be configured to detect light in a common wavelength band. In the above embodiment and its modified form, the multiple light-receiving units 11b included in the column lines of the pixel array units 121 and 123 may, for example, have a color filter that transmits light in a common wavelength band on the light incident surface of the photodiode. In this case, the interface units 122 and 126 can obtain the sum of multiple pixel data of the same color for each column line.

[0105] Here, in the pixel array units 121 and 123, when three column lines are considered as a set of column line groups, in each column line group, the multiple light-receiving units 11b included in the first column line may be configured to detect red light, the multiple light-receiving units 11b included in the second column line may be configured to detect green light, and the multiple light-receiving units 11b included in the third column line may be configured to detect blue light. In this case, the interface units 122 and 126 can obtain the sum of red pixel data, the sum of green pixel data, and the sum of blue pixel data for each column line group. As a result, the interface units 122 and 126 can generate and output color displayable image data.

[0106] <3. Application Examples> Next, we will describe examples of applications of the imaging device 100 (hereinafter simply referred to as "imaging device 100") according to the above embodiment and its modified form.

[0107] [Application Example 1] Figure 34 shows a schematic example of a factory automation (FA) configuration using the imaging device 100. In the FA, the imaging device 100 is positioned near a conveying device 300 including a belt conveyor and images a predetermined imaging area of ​​the belt conveyor. In the imaging area, multiple items are moving in the direction of travel of the belt conveyor. The imaging device 100 outputs the image data obtained by imaging to the signal processing unit 200. Based on the image data input from the imaging device 100, the signal processing unit 200 performs inspections such as checking for defects in the items included in the image data. In this way, by applying the imaging device 100 to FA, the limitations on the conveying speed of the belt conveyor are relaxed, and the inspection speed can be improved.

[0108] [Application Example 2] Figure 35 shows a schematic configuration example of a biological sample analyzer 6100 in which the imaging device 100 is used as the detection unit 6102, which will be described later.

[0109] The biological sample analyzer 6100 shown in Figure 35 includes a light irradiation unit 6101 that irradiates light onto a biological sample S flowing through a channel C, a detection unit 6102 that detects the light generated by irradiating the biological sample S, and an information processing unit 6103 that processes information related to the light detected by the detection unit 6102. Examples of the biological sample analyzer 6100 include a flow cytometer and an imaging cytometer. The biological sample analyzer 6100 may also include a sorting unit 6104 that sorts specific biological particles P within the biological sample S. An example of a biological sample analyzer 6100 including a sorting unit 6104 is a cell sorter.

[0110] (Biological sample) The biological sample S may be a liquid sample containing biological particles P. The biological particles P are, for example, cells or non-cellular biological particles. The cells may be living cells, and more specifically, blood cells such as red blood cells and white blood cells, and germ cells such as sperm and fertilized eggs. The cells may be directly collected from a sample such as whole blood, or they may be cultured cells obtained after culturing. Examples of non-cellular biological particles include extracellular vesicles, particularly exosomes and microvesicles. The biological particles P may be labeled with one or more labeling substances (for example, dyes (particularly fluorescent dyes) and fluorescent dye-labeled antibodies). In addition, particles other than biological particles P may be analyzed by the biological sample analyzer of this disclosure, and beads, etc., may be analyzed for calibration purposes.

[0111] (Flow channel) The channel C is configured to allow a biological sample S to flow through it. In particular, the channel C may be configured to form a flow in which biological particles P contained in the biological sample S are arranged in a substantially straight line. The channel structure including the channel C may be designed to form a laminar flow. In particular, the channel structure is designed to form a laminar flow in which the flow of the biological sample S (sample flow) is surrounded by the flow of the sheath fluid. The design of the channel structure may be appropriately selected by those skilled in the art, and known designs may be adopted. The channel C may be formed in a flow channel structure such as a microchip (a chip with a channel on the order of micrometers) or a flow cell. The width of the channel C is 1 mm or less, and in particular may be 10 μm or more and 1 mm or less. The channel C and the channel structure including it may be formed from materials such as plastic or glass.

[0112] The biological sample analyzer 6100 is configured such that light from the light irradiation unit 6101 is irradiated onto a biological sample S, particularly biological particles P, flowing through a channel C. The biological sample analyzer 6100 may be configured such that the light irradiation point (interrogation point) for the biological sample S is located within the channel structure in which the channel C is formed, or it may be configured such that the light irradiation point is located outside the channel structure. An example of the former is a configuration in which the light is irradiated onto a channel C in a microchip or flow cell. In the latter case, the light may be irradiated onto biological particles P after they have exited the channel structure (particularly its nozzle), for example, a Jet-in-Air type flow cytometer.

[0113] (Light-irradiated area) The light irradiation unit 6101 includes a light source unit that emits light and a light guide optical system that guides the light to the irradiation point. The light source unit includes one or more light sources. The type of light source is, for example, a laser light source or an LED. The wavelength of the light emitted from each light source may be any of the wavelengths of ultraviolet light, visible light, or infrared light. The light guide optical system includes optical components such as a beam splitter group, a mirror group, or an optical fiber. The light guide optical system may also include a lens group for focusing the light, for example, an objective lens. There may be one or more irradiation points where the light intersects with the biological sample. The light irradiation unit 6101 may be configured to focus light irradiated from one or more different light sources to a single irradiation point.

[0114] (Detection unit) The detection unit 6102 includes at least one photodetector for detecting light generated by light irradiation of biological particles P. The light to be detected is, for example, fluorescence or scattered light (e.g., one or more of forward scattered light, back scattered light, and side scattered light). Each photodetector includes one or more photoreceiving elements, for example, a photoreceiving element array. Each photodetector may include one or more PMTs (photomultiplier tubes) and / or photodiodes such as APDs and MPPCs as photoreceiving elements. The photodetector may include, for example, a PMT array in which multiple PMTs are arranged in a one-dimensional direction. The detection unit 6102 may also include an image sensor such as a CCD or CMOS. The detection unit 6102 can acquire an image of the biological particles P (e.g., a bright-field image, a dark-field image, and a fluorescence image) using the image sensor.

[0115] The detection unit 6102 includes a detection optical system that directs light of a predetermined detection wavelength to a corresponding photodetector. The detection optical system includes a spectroscopic unit such as a prism or diffraction grating, or a wavelength separation unit such as a dichroic mirror or optical filter. The detection optical system is configured to spectrally analyze light generated by, for example, irradiation of biological particles P, and to detect the spectrally analyzed light using a plurality of photodetectors, more than the number of fluorescent dyes to which the biological particles P are labeled. A flow cytometer including such a detection optical system is called a spectral flow cytometer. The detection optical system is also configured to separate light corresponding to the fluorescence wavelength range of a specific fluorescent dye from the light generated by, for example, irradiation of biological particles P, and to detect the separated light using a corresponding photodetector.

[0116] Furthermore, the detection unit 6102 may include a signal processing unit that converts the electrical signal obtained by the photodetector into a digital signal. The signal processing unit may include an A / D converter as the device that performs the conversion. The digital signal obtained by the conversion by the signal processing unit may be transmitted to the information processing unit 6103. The digital signal may be treated by the information processing unit 6103 as data related to light (hereinafter also referred to as "light data"). The light data may be, for example, light data including fluorescence data. More specifically, the light data may be light intensity data, and the light intensity may be light intensity data of light including fluorescence (which may include feature quantities such as Area, Height, and Width).

[0117] (Information Processing Department) The information processing unit 6103 includes, for example, a processing unit that performs processing of various data (e.g., optical data) and a storage unit that stores various data. When the processing unit obtains optical data corresponding to a fluorescent dye from the detection unit 6102, it may perform fluorescence leakage correction (compensation processing) on ​​the optical intensity data. In the case of a spectral flow cytometer, the processing unit also performs fluorescence separation processing on the optical data to obtain optical intensity data corresponding to the fluorescent dye. The fluorescence separation processing may be performed, for example, according to the unmixing method described in Japanese Patent Application Publication No. 2011-232259. If the detection unit 6102 includes an image sensor, the processing unit may obtain morphological information of biological particles based on the image obtained by the image sensor. The storage unit may be configured to store the acquired optical data. The storage unit may further be configured to store spectral reference data used in the unmixing processing.

[0118] If the biological sample analyzer 6100 includes a sorting unit 6104 described later, the information processing unit 6103 may determine whether to sort biological particles P based on optical data and / or morphological information. Based on the result of this determination, the information processing unit 6103 controls the sorting unit 6104, and the sorting unit 6104 may sort the biological particles P.

[0119] The information processing unit 6103 may be configured to output various types of data (e.g., optical data and images). For example, the information processing unit 6103 may output various types of data (e.g., two-dimensional plots, spectral plots, etc.) generated based on the optical data. The information processing unit 6103 may also be configured to accept input of various types of data, for example, to accept gating processing on a plot by a user. The information processing unit 6103 may include an output unit (e.g., a display) or an input unit (e.g., a keyboard) for executing such output or input.

[0120] The information processing unit 6103 may be configured as a general-purpose computer, for example, as an information processing device equipped with a CPU, RAM, and ROM. The information processing unit 6103 may be contained within the housing that houses the light irradiation unit 6101 and the detection unit 6102, or it may be located outside of that housing. Furthermore, various processing or functions performed by the information processing unit 6103 may be implemented by a server computer or cloud connected via a network.

[0121] (Preparative separation section) The sorting unit 6104 sorts the biological particles P according to the determination result by the information processing unit 6103. The sorting method may involve generating droplets containing biological particles by vibration, applying an electric charge to the droplets to be sorted, and controlling the direction of movement of the droplets with electrodes. The sorting method may also involve controlling the direction of movement of the biological particles P within a flow channel structure. The flow channel structure may be provided with a control mechanism, for example, by pressure (injection or suction) or electric charge. An example of such a flow channel structure is a chip (for example, the chip described in Japanese Patent Application Publication No. 2020-76736) in which a flow channel C branches downstream into a recovery flow channel and a waste liquid flow channel, and specific biological particles are recovered into the recovery flow channel.

[0122] The present disclosure has been described above with reference to embodiments, modifications thereof, and application examples. However, the present disclosure is not limited to the above embodiments, and various modifications are possible. The effects described herein are merely illustrative. The effects of the present disclosure are not limited to those described herein. The present disclosure may have effects other than those described herein.

[0123] Furthermore, for example, this disclosure can take the following configuration. (1) Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels. Equipped with, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit including a rewriting circuit for rewriting an initial value and an adding circuit for adding information corresponding to the light pulse to the initial value. It has, The control unit writes the information held in the counter unit as an initial value to the counter unit included in the next row of pixels in the column direction, and then adds the information corresponding to the light pulse obtained from the light pulse response unit to the initial value. Imaging device. (2) The counter unit is a k-bit (k≧1) digital counter. The imaging device described in (1). (3) Each of the aforementioned pixels further has a latch unit that holds information input from the counter unit, The control unit causes the information input from the counter unit, which is held in the latch unit, to be written as an initial value to the counter unit included in the next row of pixels in the column direction. (2) The imaging device described above. (4) The counter unit includes multiple flip-flops for each bit, One of the multiple flip-flops provided for each bit also serves as the latch. (3) The imaging device described above. (5) The higher bits of the counter section are formed outside the region where the plurality of pixels are arranged in a matrix. The imaging device described in any one of (2) to (4). (6) The number of bits in the counter section is greater in the later stages in the column direction compared to the first stage in the column direction. The imaging device described in any one of (2) to (5). (7) Each of the aforementioned pixels has two of the aforementioned counter units, While the control unit writes the information held in the first counter unit, which is one of the counter units, as an initial value to the first counter unit included in the next row of pixels in the column direction, it also writes the information held in the second counter unit, which is the other counter unit, as an initial value to the second counter unit included in the next row of pixels in the column direction. An imaging device as described in any one of (1) to (6). (8) An element is inserted into the wiring through which the control signals input from the control unit to the plurality of pixels propagate, in order to improve the waveform distortion of the control signals. An imaging device as described in any one of (1) to (7). (9) Each pixel further includes a determination unit that determines whether the counter unit is saturated, and if it determines that the counter unit is saturated, it controls the counter unit to add information corresponding to the clock to the initial value instead of information corresponding to the light pulse. An imaging device as described in any one of (1) through (8). (10) Each of the aforementioned pixels further includes a determination unit that determines whether or not the counter unit has saturated, and if it determines that the counter unit has saturated, it controls the counter unit to record the time code at that time. An imaging device as described in any one of (1) through (8). (11) Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels. Equipped with, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A switch unit that connects one of the number of input terminals equal to the number of pixels included in the row line to one output terminal. A counter unit that integrates information corresponding to the optical pulses obtained from all the optical pulse response units included in the row line via the switch unit, It has, The control unit causes the switch unit to sequentially select one by one all of the optical pulse response units included in the row line, thereby causing the counter unit to perform the integration of information corresponding to the optical pulses obtained from all of the optical pulse response units. Imaging device. (12) The counter unit is a digital counter. The imaging device described in (11). (13) The higher bits of the counter section are formed outside the region where the plurality of pixels are arranged in a matrix. The imaging device described in (11) or (12). (14) An element is inserted into the wiring through which the control signals input from the control unit to the plurality of pixels propagate, in order to improve the waveform distortion of the control signals. The imaging device described in any one of (11) to (13). (15) Each pixel further includes a determination unit that determines whether the counter unit is saturated, and if it determines that the counter unit is saturated, it controls the counter unit to add information corresponding to the clock to the initial value instead of information corresponding to the light pulse. The imaging device described in any one of (11) to (14). (16) Each of the aforementioned pixels further includes a determination unit that determines whether or not the counter unit has saturated, and if it determines that the counter unit has saturated, it controls the counter unit to record the time code at that time. The imaging device described in any one of (11) to (14). (17) Multiple pixels arranged in a matrix, A generation unit generates image data by performing TDI (Time Delay Integration) processing using information obtained from multiple sets of pixel lines arranged in a row direction among the aforementioned multiple pixels. Equipped with, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit that integrates information corresponding to the optical pulse obtained from the optical pulse response unit, has Imaging device. (18) The counter unit is a digital counter. The imaging device described in (17). (19) The plurality of pixels included in the row line are configured to detect light in a common wavelength band with respect to each other. An imaging device as described in any one of (1) to (10). (20) When three column lines are considered as a group of column lines, in each of the column line groups, the plurality of pixels in the first column line are configured to detect red light, the plurality of pixels in the second column line are configured to detect green light, and the plurality of pixels in the third column line are configured to detect blue light. The imaging device described in (19). (twenty one) The plurality of pixels included in the row line are configured to detect light in a common wavelength band with respect to each other. The imaging device described in any one of (11) to (16). (twenty two) When three column lines are considered as a group of column lines, in each of the column line groups, the plurality of pixels in the first column line are configured to detect red light, the plurality of pixels in the second column line are configured to detect green light, and the plurality of pixels in the third column line are configured to detect blue light. The imaging device described in (21). (twenty three) The plurality of pixels included in the row line are configured to detect light in a common wavelength band with respect to each other. The imaging device described in (17) or (18). (twenty four) When three column lines are considered as a group of column lines, in each of the column line groups, the plurality of pixels in the first column line are configured to detect red light, the plurality of pixels in the second column line are configured to detect green light, and the plurality of pixels in the third column line are configured to detect blue light. The imaging device described in (23). (twenty five) An imaging device that acquires image data by imaging, A processing unit for processing the image data obtained by the imaging device and Equipped with, The imaging device is Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels, An output unit that outputs the image data obtained by the TDI control to the processing unit. It has, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit including a rewriting circuit for rewriting an initial value and an adding circuit for adding information corresponding to the light pulse to the initial value. It has, The control unit writes the information held in the counter unit as an initial value to the counter unit included in the next row of pixels in the column direction, and then adds the information corresponding to the light pulse obtained from the light pulse response unit to the initial value. Information processing system. (26) An imaging device that acquires image data by imaging, A processing unit for processing the image data obtained by the imaging device and Equipped with, The imaging device is Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels, An output unit that outputs the image data obtained by the TDI control to the processing unit. It has, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A switch unit that connects one of the number of input terminals equal to the number of pixels included in the row line to one output terminal. A counter unit that integrates information corresponding to the optical pulses obtained from all the optical pulse response units included in the row line via the switch unit, It has, The control unit causes the switch unit to sequentially select one by one all of the optical pulse response units included in the row line, thereby causing the counter unit to perform the integration of information corresponding to the optical pulses obtained from all of the optical pulse response units. Information processing system. (27) An imaging device that acquires image data by imaging, A processing unit for processing the image data obtained by the imaging device and Equipped with, The imaging device is Multiple pixels arranged in a matrix, A generation unit generates the image data by performing TDI (Time Delay Integration) processing using information obtained from multiple sets of pixel lines arranged in a row direction among the multiple pixels. It has, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit that integrates information corresponding to the optical pulse obtained from the optical pulse response unit, has Information processing system.

[0124] In the imaging apparatus according to the first aspect of this disclosure and the information processing system according to the second aspect of this disclosure, at each pixel, the information held in the counter unit is written as an initial value to the counter unit included in the next row of pixels in the column direction, and then information corresponding to the optical pulse obtained from the optical pulse response unit is added to the initial value. As a result, the information held in the counter unit after the addition is written as an initial value to the counter unit included in the next row of pixels in the column direction. As a result, addition can be performed without AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. Therefore, it is possible to provide an imaging apparatus that does not require AD conversion processing or CDS processing.

[0125] In the imaging apparatus relating to the third aspect of this disclosure and the information processing system relating to the fourth aspect of this disclosure, a switch unit sequentially selects all optical pulse response units included in the row line one by one, and the counter unit performs integration of information corresponding to the optical pulses obtained from all optical pulse response units. This enables addition processing without AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. Therefore, it is possible to provide an imaging apparatus that does not require AD conversion processing or CDS processing.

[0126] In the imaging apparatus relating to the fifth aspect of this disclosure and the information processing system relating to the sixth aspect of this disclosure, information corresponding to the optical pulse obtained from the optical pulse response unit is integrated at each pixel, and image data is generated by performing TDI processing using information (integrated values) obtained from multiple sets of pixel lines arranged in the row direction from among multiple pixels. This allows TDI processing to be performed without AD conversion processing. Furthermore, since the pixel data obtained in this way is low-noise data, there is no need to perform CDS processing. Therefore, it is possible to provide an imaging apparatus that does not require AD conversion processing or CDS processing. The effects of this disclosure are not necessarily limited to those described herein, and may be any of the effects described herein.

[0127] This application claims priority based on Japanese Patent Application No. 2021-110226, filed with the Japan Patent Office on 1 July 2021, and all contents of that application are incorporated herein by reference.

[0128] Those skilled in the art will understand that various modifications, combinations, subcombinations, and changes can be conceived depending on design requirements and other factors, and that these fall within the scope of the attached claims and their equivalents.

Claims

1. Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels. Equipped with, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit including a rewriting circuit for rewriting an initial value and an adding circuit for adding information corresponding to the light pulse to the initial value. It has, The control unit writes the information held in the counter unit as an initial value to the counter unit included in the next row of pixels in the column direction, and then adds the information corresponding to the light pulse obtained from the light pulse response unit to the initial value. Imaging device.

2. The counter unit is a k-bit (k≧1) digital counter. The imaging apparatus according to claim 1.

3. Each of the aforementioned pixels further has a latch unit that holds information input from the counter unit, The control unit causes the information input from the counter unit, which is held in the latch unit, to be written as an initial value to the counter unit included in the next row of pixels in the column direction. The imaging apparatus according to claim 2.

4. The counter unit includes multiple flip-flops for each bit, One of the multiple flip-flops provided for each bit also serves as the latch. The imaging device according to claim 3.

5. The higher bits of the counter section are formed outside the region where the plurality of pixels are arranged in a matrix. The imaging apparatus according to claim 2.

6. The number of bits in the counter section is greater in the later stages in the column direction compared to the first stage in the column direction. The imaging apparatus according to claim 2.

7. Each of the aforementioned pixels has two of the aforementioned counter units, While the control unit is writing the information held in the first counter unit, which is one of the counter units, as an initial value to the first counter unit included in the next row of pixels in the column direction, it is also writing the information held in the second counter unit, which is the other counter unit, as an initial value to the second counter unit included in the next row of pixels in the column direction. The imaging apparatus according to claim 1.

8. An element is inserted into the wiring through which the control signals input from the control unit to the plurality of pixels propagate, in order to improve the waveform distortion of the control signals. The imaging apparatus according to claim 1.

9. Each pixel further includes a determination unit that determines whether the counter unit is saturated, and if it determines that the counter unit is saturated, it controls the counter unit to add information corresponding to the clock to the initial value instead of information corresponding to the light pulse. The imaging apparatus according to claim 1.

10. Each of the aforementioned pixels further includes a determination unit that determines whether or not the counter unit has saturated, and if it determines that the counter unit has saturated, it controls the counter unit to record the time code at that time. The imaging apparatus according to claim 1.

11. Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels. Equipped with, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A switch unit that connects one of the number of input terminals equal to the number of pixels included in the row line to one output terminal. A counter unit that integrates information corresponding to the optical pulses obtained from all the optical pulse response units included in the row line via the switch unit, It has, The control unit causes the switch unit to sequentially select one by one all of the optical pulse response units included in the row line, thereby causing the counter unit to perform the integration of information corresponding to the optical pulses obtained from all of the optical pulse response units. Imaging device.

12. The counter unit is a digital counter. The imaging apparatus according to claim 11.

13. The higher bits of the counter section are formed outside the region where the plurality of pixels are arranged in a matrix. The imaging apparatus according to claim 11.

14. An element is inserted into the wiring through which the control signals input from the control unit to the plurality of pixels propagate, in order to improve the waveform distortion of the control signals. The imaging apparatus according to claim 11.

15. Each of the aforementioned pixels further includes a determination unit that determines whether or not the counter unit has saturated, and if it determines that the counter unit has saturated, it controls the counter unit to record the time code at that time. The imaging apparatus according to claim 11.

16. Multiple pixels arranged in a matrix, A generation unit generates image data by performing TDI (Time Delay Integration) processing using information obtained from multiple sets of pixel lines arranged in a row direction among the aforementioned multiple pixels. Equipped with, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit that integrates information corresponding to the optical pulse obtained from the optical pulse response unit, has Imaging device.

17. The counter unit is a digital counter. The imaging device according to claim 16.

18. The plurality of pixels included in the row line are configured to detect light in a common wavelength band with respect to each other. The imaging apparatus according to claim 1.

19. When three column lines are considered as a set of column line groups, in each of the column line groups, the plurality of pixels included in the first column line are configured to detect red light, the plurality of pixels included in the second column line are configured to detect green light, and the plurality of pixels included in the third column line are configured to detect blue light. The imaging apparatus according to claim 18.

20. The plurality of pixels included in the row line are configured to detect light in a common wavelength band with respect to each other. The imaging apparatus according to claim 11.

21. When three column lines are considered as a set of column line groups, in each of the column line groups, the plurality of pixels included in the first column line are configured to detect red light, the plurality of pixels included in the second column line are configured to detect green light, and the plurality of pixels included in the third column line are configured to detect blue light. The imaging apparatus according to claim 20.

22. The plurality of pixels included in the row line are configured to detect light in a common wavelength band with respect to each other. The imaging device according to claim 16.

23. When three column lines are considered as a set of column line groups, in each of the column line groups, the plurality of pixels included in the first column line are configured to detect red light, the plurality of pixels included in the second column line are configured to detect green light, and the plurality of pixels included in the third column line are configured to detect blue light. The imaging device according to claim 22.

24. An imaging device that acquires image data by imaging, A processing unit for processing the image data obtained by the imaging device and Equipped with, The imaging device is Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels, An output unit that outputs the image data obtained by the TDI control to the processing unit. It has, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit including a rewriting circuit for rewriting an initial value and an adding circuit for adding information corresponding to the light pulse to the initial value. It has, The control unit writes the information held in the counter unit as an initial value to the counter unit included in the next row of pixels in the column direction, and then adds the information corresponding to the light pulse obtained from the light pulse response unit to the initial value. Information processing system.

25. An imaging device that acquires image data by imaging, A processing unit for processing the image data obtained by the imaging device and Equipped with, The imaging device is Multiple pixels arranged in a matrix, A control unit that performs TDI (Time Delay Integration) control on the plurality of pixels, An output unit that outputs the image data obtained by the TDI control to the processing unit. It has, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A switch unit that connects one of the number of input terminals equal to the number of pixels included in the row line to one output terminal. A counter unit that integrates information corresponding to the optical pulses obtained from all the optical pulse response units included in the row line via the switch unit, It has, The control unit causes the switch unit to sequentially select one by one all of the optical pulse response units included in the row line, thereby causing the counter unit to perform the integration of information corresponding to the optical pulses obtained from all of the optical pulse response units. Information processing system.

26. An imaging device that acquires image data by imaging, A processing unit for processing the image data obtained by the imaging device and Equipped with, The imaging device is Multiple pixels arranged in a matrix, A generation unit generates the image data by performing TDI (Time Delay Integration) processing using information obtained from multiple sets of pixel lines arranged in a row direction among the multiple pixels. It has, Each of the aforementioned pixels is An optical pulse response unit that generates optical pulses in response to incident light, A counter unit that integrates information corresponding to the optical pulse obtained from the optical pulse response unit, has Information processing system.

Images