Light detection device and electronic apparatus

Abstract

[Problem] To achieve both dynamic range expansion and frame rate improvement. [Solution] This light detection device is provided with a plurality of pixels, each of which has a photoelectric conversion element that stores a charge corresponding to the light intensity of incident light. Each of the plurality of pixels has: a charge-voltage conversion unit that converts the charge stored in the photoelectric conversion element into a voltage; a first holding unit that holds charges stored in the photoelectric conversion element; a second holding section that alternately holds the charge stored in the photoelectric conversion element with the first holding section; and a third holding section that holds the charge stored in the photoelectric conversion element using the same timing as other pixels.

Description

Technical Field

[0001] The present disclosure relates to a light detection device and an electronic device. Background Art

[0002] A light detection device (Patent Document 1) provided with a global shutter function for simultaneously exposing all pixels when capturing an image of a moving object has been disclosed. In Patent Document 1, in order to increase the dynamic range, a holding unit for holding the charge overflowing from the photoelectric conversion unit is provided separately from the holding unit for holding the charge of the global shutter. Citation List Patent Document

[0003] Patent Document 1: JP 2020-178163A Summary of the Invention Technical Problem

[0004] However, according to the technique of Patent Document 1, when reading the charge from the holding unit, the charge overflowing from the photoelectric conversion unit cannot be held. Therefore, there is a problem that exposure cannot be performed during the charge readout period and the frame rate decreases.

[0005] Therefore, the present disclosure provides a light detection device and an electronic device capable of simultaneously increasing the dynamic range and improving the frame rate. [Solution to the Technical Problem]

[0006] In order to solve the above problems, the present disclosure provides a light detection device including a plurality of pixels, each pixel including: a photoelectric conversion element that accumulates charge corresponding to the incident light amount, wherein each of the plurality of pixels includes: a charge-voltage conversion unit that converts the charge accumulated in the photoelectric conversion element into a voltage; a first holding unit that holds the charge accumulated in the photoelectric conversion element; a second holding unit that alternately holds the charge accumulated in the photoelectric conversion element with the first holding unit; and a third holding unit that holds the charge accumulated in the photoelectric conversion element at the same timing as other pixels.

[0007] When one of the first holding unit and the second holding unit transfers the charge to the charge-voltage conversion unit, the other of the first holding unit and the second holding unit can hold the charge accumulated in the photoelectric conversion element.

[0008] Each of the plurality of pixels can alternately hold charges in the first holding unit or the second holding unit on a frame-by-frame basis, and hold charges in the third holding unit for each frame.

[0009] Each of the plurality of pixels can output a pixel signal at a reset level and a pixel signal corresponding to the charges held in the first holding unit or the second holding unit and the charges held in the third holding unit for each frame, and The pixel signal at the reset level and the pixel signal corresponding to the charges can be output at different timings from each other.

[0010] The saturation charge levels of the first holding unit and the second holding unit can be higher than the saturation charge level of the third holding unit.

[0011] The charge holding periods of the first holding unit and the second holding unit corresponding to each frame can be longer than the charge holding period of the third holding unit.

[0012] Each of the plurality of pixels can include: A first transfer control unit that controls the transfer of charges from the photoelectric conversion element to the first holding unit; A second transfer control unit that controls the transfer of charges from the first holding unit to the charge-voltage conversion unit; A third transfer control unit that controls the transfer of charges from the photoelectric conversion element to the second holding unit; and A fourth transfer control unit that controls the transfer of charges from the second holding unit to the charge-voltage conversion unit.

[0013] The first transfer control unit or the third transfer control unit can alternately transfer the charges accumulated in the photoelectric conversion element to the first holding unit or the second holding unit on a frame-by-frame basis.

[0014] The second transfer control unit or the fourth transfer control unit can alternately transfer the charges held in the first holding unit or the second holding unit to the charge-voltage conversion unit on a frame-by-frame basis.

[0015] When the first transfer control unit transfers charges from the photoelectric conversion element to the first holding unit, a negative bias voltage can be supplied to the gate of the third transfer control unit, and When the third transfer control unit transfers charges from the photoelectric conversion element to the second holding unit, a negative bias voltage can be supplied to the gate of the first transfer control unit.

[0016] Each of the plurality of pixels may include: A fifth transfer control unit that controls transfer of charge from the photoelectric conversion element to the third holding unit; and A sixth transfer control unit that controls transfer of charge from the third holding unit to the charge-voltage conversion unit.

[0017] The fifth transfer control unit may transfer charge from the photoelectric conversion element to the third holding unit at the same timing as other pixels for each frame, and The sixth transfer control unit may transfer charge from the third holding unit to the charge-voltage conversion unit for each frame.

[0018] For each frame, after charge is transferred from the photoelectric conversion element to the first holding unit or the second holding unit, the fifth transfer control unit may transfer charge from the photoelectric conversion element to the third holding unit.

[0019] Each of the plurality of pixels may include: A fourth holding unit that holds the charge accumulated in the photoelectric conversion element at the same timing as other pixels, and Each of the plurality of pixels may hold charge in the first holding unit or the second holding unit alternately in units of frames, and may hold charge in the third holding unit or the fourth holding unit alternately in units of frames.

[0020] Each of the plurality of pixels may output a pixel signal of a reset level and a pixel signal corresponding to the charge held in the first holding unit and the third holding unit, or a pixel signal corresponding to the charge held in the second holding unit or the fourth holding unit for each frame, and The pixel signal of the reset level and the pixel signal corresponding to the charge may be output at different timings from each other.

[0021] The saturation charge levels of the first holding unit and the second holding unit may be higher than the saturation charge levels of the third holding unit and the fourth holding unit.

[0022] The charge holding periods of the first holding unit and the second holding unit corresponding to each frame may be longer than the charge holding periods of the third holding unit and the fourth holding unit.

[0023] Each of the plurality of pixels may include: A seventh transfer control unit that controls transfer of charge from the photoelectric conversion element to the third holding unit; An eighth transfer control unit that controls the transfer of charge from the third holding unit to the first holding unit; A ninth transfer control unit that controls the transfer of charge from the first holding unit to the charge-voltage conversion unit; A tenth transfer control unit that controls the transfer of charge from the photoelectric conversion element to the fourth holding unit; An eleventh transfer control unit that controls the transfer of charge from the fourth holding unit to the second holding unit; and A twelfth transfer control unit that controls the transfer of charge from the second holding unit to the charge-voltage conversion unit.

[0024] The seventh transfer control unit or the tenth transfer control unit may alternately transfer the charge accumulated in the photoelectric conversion element to the third holding unit or the fourth holding unit in units of frames, The eighth transfer control unit or the eleventh transfer control unit alternately transfers charge from the third holding unit or the fourth holding unit to the first holding unit or the second holding unit in units of frames, and The ninth transfer control unit or the twelfth transfer control unit may alternately transfer the charge held in the first holding unit or the second holding unit to the charge-voltage conversion unit in units of frames.

[0025] An electronic device is provided according to the present disclosure, including: A light detection device; A processing unit that processes pixel data output from the light detection device, where The light detection device includes: A plurality of pixels, each pixel including: a photoelectric conversion element that accumulates charge corresponding to the incident light amount, and Each of the plurality of pixels includes: A charge-voltage conversion unit that converts the charge accumulated in the photoelectric conversion element into a voltage; A first holding unit that holds the charge accumulated in the photoelectric conversion element; A second holding unit that alternately holds the charge accumulated in the photoelectric conversion element with the first holding unit; and A third holding unit that holds the charge accumulated in the photoelectric conversion element at the same timing as other pixels. Description of the Drawings

[0026] Figure 1 is a block diagram of an electronic device according to a first embodiment of the present disclosure. Figure 2It is a block diagram of an exemplary configuration of a light detection device according to a first embodiment of the present disclosure. Figure 3 It is a diagram showing a configuration example of a pixel and its periphery according to a first embodiment of the present disclosure. Figure 4A It is a diagram showing a first example of a stacked structure of a light detection device. Figure 4B It is a diagram showing a second example of a stacked structure of a light detection device. Figure 5 It is a timing diagram showing an example of an imaging operation performed by an electronic device. Figure 6A It is a diagram showing an exposure period of a first frame of a pixel according to a first embodiment of the present disclosure. Figure 6B It is a diagram showing a readout period of a first frame of a pixel according to a first embodiment of the present disclosure. Figure 6C It is a diagram showing an exposure period of a second frame of a pixel according to a first embodiment of the present disclosure after the readout of the first frame. Figure 6D It is a diagram showing a readout period of a second frame of a pixel according to a first embodiment of the present disclosure. Figure 7 It is a timing diagram showing an imaging operation of a pixel according to a first embodiment of the present disclosure. Figure 8 It is a diagram for explaining the barriers of two OF memories. Figure 9 It is a diagram showing a configuration example of a pixel and its periphery according to a comparative example. Figure 10A It is a diagram showing an exposure period of a pixel according to a comparative example. Figure 10B It is a diagram showing a readout period of a pixel according to a comparative example. Figure 11 It is a diagram showing a configuration example of a pixel and its periphery according to a second embodiment of the present disclosure. Figure 12A It is a diagram showing an exposure period of a first frame of a pixel according to a second embodiment of the present disclosure. Figure 12B It is a diagram showing a readout period of a first frame of a pixel and an exposure period of a second frame according to a first embodiment of the present disclosure. Figure 13 It is a timing diagram showing pixel control according to a second embodiment of the present disclosure. Figure 14 It is a diagram showing an example of a schematic configuration of an endoscopic surgical system. Figure 15It is a block diagram showing an example of the functional configuration of a camera head and a CCU. Detailed implementation

[0027] Hereinafter, embodiments of the light detection device and the electronic device will be described with reference to the accompanying drawings. Although the main components of the light detection device and the electronic device will be mainly described below, the light detection device and the electronic device may include components and functions that are not shown or described. The following description is not intended to exclude any elements or functions that are not shown or described.

[0028] (First embodiment) Figure 1 It is a block diagram showing a structural example of the electronic device 1 according to the first embodiment of the present disclosure. The electronic device 1 is configured to capture image data and includes an imaging lens 11, a light detection device 2, a processing unit 3, and a control unit 4. Possible examples of the electronic device 1 include a camera mounted on an industrial robot or a vehicle-mounted camera, but the electronic device 1 can also be used for any specific application and configuration.

[0029] The imaging lens 11 collects incident light and guides the incident light to the light detection device 2. The light detection device 2 is, for example, a complementary metal (CMOS) image sensor, and captures image data by performing photoelectric conversion on the incident light. In addition, the light detection device 2 can generate moving image data by continuously capturing image data. The image data output from the light detection device 2 is input to the processing unit 3 via the transmission line 12. The processing unit 3 performs predetermined image processing on the image data output from the light detection device 2.

[0030] The electronic device 1 may include a recording unit 5. The recording unit 5 records the image data from the light detection device 2. The recording unit 5 may also be arranged on a server or the like connected via a network.

[0031] The control unit 4 performs controls such as control of the imaging timing performed by the light detection device 2 via the control line 13. For example, the control unit 4 controls the light detection device 2 to start and end image capture according to the operation of a shutter operation member (not shown).

[0032] Figure 2 It is a block diagram showing a structural example of the light detection device 2 according to the first embodiment of the present disclosure. The light detection device 2 includes a pixel array unit 20, a vertical drive circuit 21, a system control circuit 22, and a signal processing unit 23.

[0033] The pixel array unit 20 includes a plurality of pixels 30 arranged in each of the first direction X and the second direction Y. In the description herein, Figure 2 the left - right (horizontal) direction will be referred to as the first direction X, and Figure 2The up-and-down (vertical) direction is called the second direction Y. A group of a plurality of pixels 30 arranged along the first direction X is called a pixel row, and a group of a plurality of pixels 30 arranged along the second direction Y is called a pixel column.

[0034] Each pixel 30 includes: a photoelectric conversion element that generates charges corresponding to the amount of incident light. A pixel circuit ( Figure 1 (not shown) connected to the pixel 30 generates a pixel signal Vimg based on the charges on the photoelectric conversion element.

[0035] The vertical drive circuit 21 includes a shift register and an address decoder. The vertical drive circuit 21 simultaneously drives all the pixels 30 in the pixel array unit 20, or drives the pixels 30 row by row. The vertical drive circuit 21 sweeps out charges from the corresponding pixels 30 and reads signals from the pixel circuits.

[0036] During the charge sweeping-out period, unnecessary charges are discharged (reset) from the photoelectric conversion element of the pixel 30. Thus, the photoelectric conversion element of the pixel 30 is prepared for starting a new exposure. The operation of discharging the charges of the photoelectric conversion element and restarting the exposure (starting to accumulate charges) is also called an electronic shutter operation.

[0037] The light detection device 2 performs a global shutter operation that simultaneously performs an electronic shutter operation on all pixels. The feature of the global shutter operation is that there is no offset in the exposure timings between pixels.

[0038] During the signal readout period, a pixel signal Vimg based on the charges accumulated in the photoelectric conversion element in the pixel 30 is read from the pixel circuit. The period from the electronic shutter operation to the signal readout is also called the exposure period. During the signal readout period, a pixel signal Vimg based on the light that has been incident on the photoelectric conversion element of the pixel 30 during the exposure period is read from the pixel circuit.

[0039] The system control circuit 22 includes a timing generator that generates various timing signals. Based on the various timing signals, the system control circuit 22 performs control such as timing control for readout and sweeping-out performed by the vertical drive circuit 21 and control for driving the signal processing unit 23.

[0040] The signal processing unit 23 receives the supply of the pixel signal Vimg from each pixel 30 and the pixel circuit in the pixel array unit 20. The signal processing unit 23 performs predetermined signal processing on the pixel signal Vimg. For example, the predetermined signal processing includes analog-to-digital conversion of the pixel signal Vimg and processing for removing noise superimposed on the pixel signal Vimg.

[0041] As a process of removing noise from the pixel signal Vimg, correlated double sampling (CDS) is used as an example. The signal processing unit 23 compares the signal level of the pixel signal Vimg corresponding to the incident light amount (also referred to as the D-phase signal) with the signal level of the pixel signal at the reset level independent of the incident light amount (also referred to as the P-phase signal). In this way, the noise (also referred to as kTC noise) superimposed on the pixel signal Vimg is removed.

[0042] Figure 3 FIG. is a diagram showing the structure of the pixel 30 and its periphery according to the first embodiment of the present disclosure. Figure 3 The pixel 30 shown outputs a pixel signal Vimg based on the illuminance of the incident light. The pixel 30 includes a photoelectric conversion element (PD) 31, a transfer unit 32, and a charge-voltage conversion unit 33. For example, the photoelectric conversion element 31 is a photodiode (PD) and has an anode and a cathode. One of the anode and the cathode (e.g., the cathode) is connected to the transfer unit 32, and the other (e.g., the anode) is connected to a predetermined reference voltage (VRLD) node such as a ground voltage. The charge-voltage conversion unit 33 is also referred to as a floating diffusion portion (FD). The charge generated by photoelectric conversion accumulated in the photoelectric conversion element 31 is transferred to the charge-voltage conversion unit 33 via the transfer unit 32.

[0043] Around the pixel 30, an amplifying transistor Q1, a selecting transistor Q2, and a resetting transistor Q3 are arranged. Note that the pixel 30 can be described as including Figure 3 at least a part of the amplifying transistor Q1, the selecting transistor Q2, and the resetting transistor Q3 in. In the description herein, the amplifying transistor Q1, the selecting transistor Q2, and the resetting transistor Q3 are referred to as pixel circuits, and these transistors in the pixel circuit are collectively referred to as pixel transistors.

[0044] In the description herein, as an example, an example in which each of the amplifying transistor Q1, the selecting transistor Q2, and the resetting transistor Q3 is an N-channel metal oxide semiconductor (NMOS) transistor will be described. However, each transistor described herein can be of any conductive type. As another example, any one of the above-mentioned transistors can be implemented as a P-channel metal oxide semiconductor (PMOS) transistor.

[0045] The transmission unit 32 includes GS transistors Q4 (the fifth transmission control unit), GS transistors Q5 (the sixth transmission control unit), OF transistors Q6 (the first transmission control unit), OF transistors Q7 (the second transmission control unit), OF transistors Q8 (the third transmission control unit), and OF transistors Q9 (the fourth transmission control unit). The GS transistors Q4 and Q5 are connected in cascade between the photoelectric conversion element 31 and the charge-voltage conversion unit 33. Similarly, the OF transistors Q6 and Q7 are connected in cascade between the photoelectric conversion element 31 and the charge-voltage conversion unit 33. Similarly, the OF transistors Q8 and Q9 are connected in cascade between the photoelectric conversion element 31 and the charge-voltage conversion unit 33.

[0046] The GS memory (the third holding unit) Mg is arranged at the connection node between the GS transistor Q4 and the GS transistor Q5. The GS memory Mg holds the charge transferred from the photoelectric conversion element 31 at the same timing as other pixels 30 according to the global shutter operation. During the signal readout period of the pixel 30, the charge in the GS memory Mg is transferred to the charge-voltage conversion unit 33.

[0047] The GS transistor Q4 transfers the charge from the photoelectric conversion element 31 to the GS memory Mg. The GS transistor Q4 switches to the on state when the signal GS1 input to the gate is at a high level, and transfers the charge in the photoelectric conversion element 31 to the GS memory Mg.

[0048] The GS transistor Q5 transfers the charge in the GS memory Mg to the charge-voltage conversion unit 33. The GS transistor Q5 switches to the on state when the signal GS2 input to the gate is at a high level, and transfers the charge in the GS memory Mg to the charge-voltage conversion unit 33.

[0049] The charge that exceeds the saturation charge level of the photoelectric conversion element 31 and overflows from the photoelectric conversion element 31 is also referred to as overflow charge.

[0050] The OF memory (the first holding unit) M1 is arranged at the connection node between the OF transistor Q6 and the OF transistor Q7. The OF memory M1 temporarily holds the charge accumulated in the photoelectric conversion element 31. More specifically, the OF memory M1 holds the overflow charge during the exposure period. During the signal readout period of the pixel 30, the charge held in the OF memory M1 is transferred to the charge-voltage conversion unit 33. The OF transistor Q6 can be controlled such that, as will be described later, when the amount of light incident on the pixel 30 is low, no charge is held in the OF memory M1.

[0051] The OF transistor Q6 transfers the charge from the photoelectric conversion element 31 to the OF memory M1. The OF transistor Q6 transfers the charge accumulated in the photoelectric conversion element 31 to the OF memory M1 when the signal OF1 input to the gate is at a high level. Alternatively, the voltage level of the signal OF1 can be set to an intermediate voltage level between the high-level voltage and the low-level voltage. In response to the signal OF1 reaching the intermediate voltage level, the OF transistor Q6 can transfer only the charge exceeding the saturation charge level of the photoelectric conversion element 31 to the OF memory M1.

[0052] The OF transistor Q7 transfers the charge in the OF memory M1 to the charge-voltage conversion unit 33. The OF transistor Q7 switches to the on state when the signal OF2 input to the gate is at a high level, and transfers the charge in the OF memory M1 to the charge-voltage conversion unit 33.

[0053] The OF memory (second holding unit) M2 is arranged at the connection node between the OF transistor Q8 and the OF transistor Q9. The OF memory M2 holds the charge transferred from the photoelectric conversion element 31 alternately with the OF memory M1.

[0054] In the present embodiment, the saturation charge levels of the OF memories M1 and M2 are set to be higher than the saturation charge level of the GS memory Mg. Therefore, the OF memories M1 and M2 can hold the charge that cannot be accumulated in the GS memory Mg, thereby increasing the dynamic range. The saturation charge levels of the OF memories M1 and M2 can also be set to be equal to or lower than the saturation charge level of the GS memory Mg.

[0055] The charge is alternately held in and read from the OF memories M1 and M2. That is, when the OF memory M1 transfers the charge to the charge-voltage conversion unit 33, the photoelectric conversion element 31 transfers the charge to the OF memory M2. When the OF memory M2 transfers the charge to the charge-voltage conversion unit 33, the photoelectric conversion element 31 transfers the charge to the OF memory M1.

[0056] The OF transistor Q8 transfers the charge from the photoelectric conversion element 31 to the OF memory M2. The OF transistor Q8 transfers the charge accumulated in the photoelectric conversion element 31 to the OF memory M2 when the signal OF3 input to the gate is at a high level. In the same manner as the signal OF1, the voltage level of the signal OF3 can be an intermediate voltage level between the high-level voltage and the low-level voltage.

[0057] The OF transistor Q9 transfers the charge in the OF memory M2 to the charge-voltage conversion unit 33. The OF transistor Q9 switches to the on state when the signal OF4 input to the gate is at a high level, and transfers the charge in the OF memory M2 to the charge-voltage conversion unit 33.

[0058] During the signal readout period, the charge from the GS memory Mg and the charge from the OF memory M1 or M2 are transferred to the charge-voltage conversion unit 33. Accordingly, the voltage of the charge-voltage conversion unit 33 becomes a level corresponding to the charge accumulated in the photoelectric conversion element 31.

[0059] The gate of the amplification transistor Q1 is at the same voltage as the charge-voltage conversion unit 33 and serves as the input part of the source follower circuit. The drain of the amplification transistor Q1 is connected to the node of the high-voltage side power supply VDD, and the source is connected to the selection transistor Q2. The source voltage of the amplification transistor Q1 varies according to the voltage of the charge-voltage conversion unit 33.

[0060] The selection transistor Q2 controls the signal readout from the pixel 30. A selection signal SEL is applied to the gate of the selection transistor Q2. The selection transistor Q2 switches to the on state when the selection signal SEL is at a high level. Thereby, the pixel signal Vimg having a voltage level corresponding to the voltage of the charge-voltage conversion unit 33 is output from the source of the selection transistor Q2 to a subsequent stage component such as the signal processing unit 23.

[0061] The reset transistor Q3 controls the discharge of the charge of the pixel 30. The source of the reset transistor Q3 is connected to the charge-voltage conversion unit 33, and the drain is connected to the node of the high-voltage side power supply VDD. A reset signal RST is applied to the gate of the reset transistor Q3. The reset transistor Q3 switches to the on state when the reset signal RST is at a high level. Accordingly, the charge of the charge-voltage conversion unit 33 is discharged to the node of the high-voltage side power supply VDD, and thus the charge-voltage conversion unit 33 is reset. By resetting the charge-voltage conversion unit 33, the pixel 30 can be used for the next exposure.

[0062] The light detection device 2 includes, for example, two stacked chips. Figure 4A is a schematic diagram showing a first example of the stacked structure of the light detection device 2. The light detection device 2 includes a pixel chip 41 and a logic chip 42 stacked on the pixel chip 41. These chips are connected via through holes or the like. In addition to through holes, these chips can also be connected by Cu-Cu bonding or bumps.

[0063] For example, a plurality of pixels 30 in the pixel array unit 20 and pixel transistors corresponding to the respective pixels 30 are arranged on the pixel chip 41. For example, the vertical drive circuit 21, the system control circuit 22, and the signal processing unit 23 are arranged on the logic chip 42. In Figure 4A the first example, the pixel 30 is illustrated as including a pixel transistor.

[0064] The light detection device 2 may include three or more stacked chips.Figure 4B is a schematic diagram showing a second example of the stacked structure of the light detection device 2. In Figure 4B the light detection device 2a, the first pixel chip 43 and the second pixel chip 44 are stacked to replace Figure 4A the pixel chip 41 in

[0065] Figure 4B The light detection device 2a in has a structure in which the pixel transistors are not arranged on the first pixel chip 43 but on the second pixel chip 44. Therefore, the light detection device 2a can increase the ratio of the area occupied by the photoelectric conversion element 31 to the chip area, improve the sensitivity, and miniaturize the chip.

[0066] Figure 5 is a timing chart showing the imaging operation performed by the light detection device 2. Figure 5 The left - right direction in is the time - axis direction t. Figure 5 shows an example of reading the pixel signal Vimg from each pixel row among a plurality of pixel rows arranged in the second direction Y in the pixel array unit 20.

[0067] The imaging operation of the light detection device 2 is performed in units of frames synchronized with a predetermined synchronization signal. One frame includes a reset period, an exposure period, and a read - out period.

[0068] In each frame, a global shutter operation is performed. First, at the moment Trst, the charges in all the pixels 30 in the pixel array unit 20 are simultaneously swept out (reset). By this reset, the pixels 30 are made available for exposure.

[0069] During the exposure period, the exposure of all the pixels 30 starts simultaneously at the moment Texp and ends simultaneously at the moment Trd. During the read - out period, starting from the moment Trd, the pixel signals Vimg are sequentially read from each pixel row.

[0070] As described above, the global shutter operation of the light detection device 2 is performed synchronously for all the pixels 30. The global shutter operation is also performed using frames as units. The light detection device 2 can capture a moving image by continuously capturing images of multiple frames.

[0071] Figures 6A to 6D is a diagram showing the imaging operation of the pixel 30 according to the first embodiment of the present disclosure. Figure 6AShows the imaging operation during the exposure period of the first frame. During the exposure period of the first frame, the photoelectric conversion element 31 accumulates charges corresponding to the incident light. During the exposure period of the first frame, the charges accumulated in the photoelectric conversion element 31 are transferred to the GS memory Mg and the OF memory M1, but not to the OF memory M2.

[0072] Figure 6B Is a diagram showing the readout period of the first frame. During the readout period of the first frame, the charges e in the GS memory Mg and the OF memory M1 are transferred to the charge-voltage conversion unit 33.

[0073] During the readout period of the first frame, light also enters the pixel 30, and the photoelectric conversion element 31 retains the charges generated by photoelectric conversion. The charges accumulated in the photoelectric conversion element 31 during the readout period are transferred to the OF memory M2, but not to the OF memory M1. This is because the OF memory M1 is transferring charges to the charge-voltage conversion unit 33.

[0074] In Figure 6B After the readout period of the first frame shown, the charges in the GS memory Mg and the OF memory M1 are reset. Therefore, the GS memory Mg and the OF memory M1 are made available for new charge transfer.

[0075] Figure 6C Is a diagram showing the exposure period of the second frame after reading the first frame. The charges accumulated in the photoelectric conversion element 31 during the exposure period of the second frame are transferred to the GS memory Mg and the OF memory M2, but not to the OF memory M1. In this way, the OF memories M1 and M2 are alternately used for each frame.

[0076] Figure 6D Is a diagram showing the readout period of the second frame. During the readout period of the second frame, the charges e in the GS memory Mg and the OF memory M2 are transferred to the charge-voltage conversion unit 33. The charges accumulated in the photoelectric conversion element 31 during the readout period of the second frame are transferred to the OF memory M1, but not to the OF memory M2. This is because the OF memory M2 is transferring charges to the charge-voltage conversion unit 33.

[0077] For the exposure periods of the third frame and subsequent frames, repeat Figures 6A to 6D The operation.

[0078] As Figures 6A to 6DAs shown, when one of the OF memories M1 and M2 transfers the charge e to the charge-voltage conversion unit 33, the other of the OF memories M1 and M2 holds the charge e accumulated in the photoelectric conversion element 31. The pixel 30 according to the first embodiment of the present disclosure can perform the reading and exposure of the charge e in parallel by using the OF memory M1 or M2 for reading or transferring the charge e. Therefore, the light detection device 2 can achieve a higher frame rate.

[0079] Figure 7 is a timing chart showing an example of the imaging operation of the pixel 30 according to the first embodiment of the present disclosure. Figure 7 The time from t1 to t24 shown includes initialization, the exposure period and readout period of the first frame, the exposure period and readout period of the second frame, and the exposure period of the third frame.

[0080] During the time from t1 to t2, the pixel 30 is initialized before taking an image. Specifically, the reset signal RST, the signals GS1 and GS2, the signals OF1 and OF2, and the signals OF3 and OF4 are set to high level in the same timing within a fixed time period. Therefore, the charges in the photoelectric conversion element 31, the GS memory Mg, the OF memory M1, the OF memory M2, and the charge-voltage conversion unit 33 are discharged to the node of the high-voltage side power supply VDD, and the charge-voltage conversion unit 33 becomes the reset level voltage.

[0081] The time from t3 to t5 is the exposure period of the first frame. As Figure 6A shown, during the exposure period of the first frame, charges are accumulated in the photoelectric conversion element 31. During the time from t3 to t4, since the signal OF1 is set to high level and the OF transistor Q6 is switched to on, the charge accumulated in the photoelectric conversion element 31 is transferred to the OF memory M1.

[0082] During the time from t4 to t5, the signal OF1 is set to low level and the signal GS1 is set to high level. Therefore, the OF transistor Q6 is switched to off, the GS transistor Q4 is switched to on, and the charge of the photoelectric conversion element 31 is transferred to the GS memory Mg.

[0083] As described above, the saturation charge level of the OF memory M1 is higher than that of the GS memory Mg. Therefore, in Figure 7In this case, the period (from time t3 to t4) during which the charge accumulated in the photoelectric conversion element 31 is held in the OF memory M1 is set to be longer than the period (from time t4 to t5) during which the charge is transferred from the photoelectric conversion element 31 to the GS memory Mg. The same applies to the OF memory M2 which will be described later. That is, in each frame, the charge holding period of the OF memory M1 or the OF memory M2 is set to be longer than the charge holding period of the GS memory Mg. Therefore, more charge can be accumulated in the OF memories M1 and M2 than in the GS memory Mg.

[0084] The time from t5 to t13 is the readout period of the first frame. As Figure 6B shown, during the readout period of the first frame, charge is read from the GS memory Mg and the OF memory M1. During the time from t6 to t7, the signals GS2 and OF2 are set to low level, and the selection signal SEL is set to high level. Therefore, the pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read.

[0085] During the time from t8 to t9, the signals GS2 and the selection signal SEL are set to high level. Since the GS transistor Q5 is switched on, the charge in the GS memory Mg is transferred to the charge-voltage conversion unit 33. Therefore, the pixel signal (D-phase signal) corresponding to the charge generated by adding the charge in the GS memory Mg to the charge in the charge-voltage conversion unit 33 is read.

[0086] During the time from t10 to t11, the signal OF2 and the selection signal SEL are set to high level. Since the OF transistor Q7 is switched on, the charge of the OF memory M1 is transferred to the charge-voltage conversion unit 33. Therefore, the pixel signal (D-phase signal) corresponding to the charge generated by further adding the charge of the OF memory M1 to the charge of the GS memory Mg is read from the charge-voltage conversion unit 33.

[0087] During the time from t11 to t12, the signal OF2, the signal GS2, and the reset signal RST are set to high level, and the charge in the GS memory Mg, the OF memory M1, and the charge-voltage conversion unit 33 is discharged to the node of the high-voltage side power supply VDD. Therefore, the charge-voltage conversion unit 33 becomes the reset level voltage.

[0088] During the time from t12 to t13, the signal OF2, the signal GS2, and the selection signal SEL are set to high level, and the pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read.

[0089] The period between time t5 and t13 is the readout period of the first frame and the exposure period of the second frame. During the exposure period of the second frame, a high-level signal OF3 is input, and the charge accumulated in the photoelectric conversion element 31 is held in the memory M2.

[0090] The time from t13 to t15 is the exposure period of the second frame after the readout of the first frame. As Figure 6C shown, during the time from t13 to t14, the charge of the photoelectric conversion element 31 is held in the OF memory M2. During the time from t14 to t15, the signal OF3 is set to a low level, and the signal GS1 is set to a high level. Therefore, the charge in the photoelectric conversion element 31 is transferred to the GS memory Mg.

[0091] The time from t15 to t23 is the readout period of the second frame. As Figure 6D shown, during the readout period of the second frame, under the same control as the readout period of the first frame, charges are read from the GS memory Mg and the OF memory M2.

[0092] That is, during the time from t16 to t17, the selection signal SEL is set to a high level, and a pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read. During the time from t18 to t19, the signals GS2 and the selection signal SEL are set to a high level, and a pixel signal (D-phase signal) corresponding to the charge generated by adding the charge in the GS memory Mg to the charge in the charge-voltage conversion unit 33 is read.

[0093] During the time from t20 to t21, the signals OF4 and the selection signal SEL are set to a high level, and a pixel signal (D-phase signal) corresponding to the charge generated by further adding the charge in the OF memory M2 to the charge in the charge-voltage conversion unit 33 is read. During the time from t21 to t22, the signals OF4, the signal GS2, and the reset signal RST are set to a high level, and the charges in the GS memory Mg, the memory M2, and the charge-voltage conversion unit 33 are swept out. During the time from t22 to t23, the signals OF4, the signal GS2, and the selection signal SEL are set to a high level, and a pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read.

[0094] The time from t15 to t24 is the readout period of the second frame and the exposure period of the third frame. During the exposure period of the third frame, the signal OF1 is set to a high level, and the charge accumulated in the photoelectric conversion element 31 is transferred to the OF memory M1.

[0095] As Figure 7As shown, the transfer unit 32 transfers the charge from the photoelectric conversion element 31 to the charge-voltage conversion unit 33 by controlling each transistor. That is, the OF transistors Q6 or Q8 alternately transfer the charge accumulated in the photoelectric conversion element 31 to the OF memories M1 or M2 in units of frames. The OF transistors Q7 or Q9 alternately transfer the charge held in the OF memories M1 or M2 to the charge-voltage conversion unit 33 in units of frames. For each frame, after the charge is transferred from the photoelectric conversion element 31 to the memories M1 or M2, the GS transistor Q4 transfers the charge from the photoelectric conversion element 31 to the GS memory Mg. The GS transistor Q5 transfers the charge from the GS memory Mg to the charge-voltage conversion unit 33 for each frame.

[0096] The control of each of the multiple pixels 30 in the pixel array unit 20 is performed. Figure 7 That is, each of the multiple pixels 30 alternately holds the charge in the OF memories M1 or M2 in units of frames, and holds the charge in the GS memory Mg for each frame. The charge in the GS transistor Q4 is transferred at the same timing as the other pixels 30 in the pixel array unit 20.

[0097] In addition, as Figure 7 shown, each of the multiple pixels 30 in the pixel array unit 20 outputs a reset-level pixel signal Vimg for each frame. Each of the multiple pixels 30 outputs a pixel signal Vimg corresponding to the charge held in the OF memories M1 or M2 and the charge held in the GS memory Mg at a timing different from that of the reset-level pixel signal Vimg for each frame.

[0098] As Figure 7 shown, the driving that executes the exposure periods (and readout periods) of two or more frames in parallel in an offset manner from each other is also referred to as pipelined driving.

[0099] In Figure 7 , an example has been described in which when the charge is transferred from the photoelectric conversion element 31 to the OF memories M1, M2, the OF transistors Q6 and Q8 are switched between conduction and cutoff; however, the height of the barrier of the OF memories M1, M2 can also be controlled.

[0100] Figure 8 is a diagram for explaining the barriers of the OF memories M1, M2. By setting the signal OF1 or OF3 input to one of the gates of the OF transistors Q6 and Q8 to a negative bias voltage, the barrier can be increased. Figure 8 shows an example in which the signal OF3 input to the gate of the OF transistor Q8 is set to a negative bias voltage. In Figure 8In the example, due to the high barrier, the charges accumulated in the photoelectric conversion element 31 are not transferred to the OF memory M2, but are transferred to the OF memory M1 with a low barrier.

[0101] Therefore, by controlling the height of the barrier, the transfer destination of the charges accumulated in the photoelectric conversion element 31 can be switched. Therefore, an element structure for controlling the barrier height can be provided instead of the OF transistors Q6 and Q8.

[0102] Figure 9 FIG. shows the structure of the pixel 100 and its periphery in a comparative example. Figure 9 The pixel 100 shown is different from Figure 3 the pixel 30 shown in that it does not have the OF transistors Q8 and Q9 and the OF memory M2.

[0103] Figure 10A and Figure 10B FIG. is a schematic diagram showing the imaging operation of the pixel 100 according to a comparative example. Figure 10A FIG. shows the exposure period of the pixel 100. In the same manner as Figure 6A shown, during the exposure period, the charges e are transferred from the photoelectric conversion element 31 to the GS memory Mg and the OF memory M1.

[0104] Figure 10B FIG. shows the readout period of the pixel 100. During the readout period, in the same manner as Figure 6B shown, the charges e are transferred from the GS memory Mg and the OF memory M1 to the charge-voltage conversion unit 33. Also in the pixel 100, the charges e are accumulated in the photoelectric conversion element 31.

[0105] However, since the pixel 100 does not have the OF memory M2, when the charges are transferred from the OF memory M1 to the charge-voltage conversion unit 33, the exposure of the next frame cannot start. Therefore, it is necessary to ensure a long interval between the exposure periods, and thus the frame rate remains low.

[0106] In the above manner, the light detection device 2 of the first embodiment of the present disclosure includes the GS memory Mg corresponding to each pixel 30, and the two OF memories M1 and M2. In addition to the GS memory Mg, the light detection device 2 also uses the OF memories M1 and M2 to transfer the charges from the photoelectric conversion element 31 for each frame. Therefore, the light detection device 2 can perform exposure while reading the charges for each frame, and can achieve a higher frame rate. In addition, as described above, the dynamic range is increased by the OF memories M1 and M2. That is, the light detection device 2 can simultaneously achieve an expanded dynamic range and an increased frame rate.

[0107] (Second Embodiment) Figure 3The configuration of the transmission unit 32 shown is an example, and various modifications can be made. Figure 11 is a diagram showing the configuration of the pixel 30a and its periphery in the second embodiment of the present disclosure. As described above, Figure 3 the transmission unit 32 in [embodiment 1] includes two OF memories M1 and M2 and one GM memory Mg, while Figure 11 the transmission unit 32a in [embodiment 2] includes two GS memories Mg1 (third holding unit) and Mg2 (fourth holding unit), and two OF memories M1a (first holding unit) and M2a (second holding unit).

[0108] In addition, Figure 11 the transmission unit 32a in [embodiment 2] includes two GS transistors Q4a and Q5a, and four OF transistors Q6a, Q7a, Q8a, and Q9a. Figure 11 In [embodiment 2], the GS transistor Q4a (seventh transmission control unit), the OF transistor Q6a (eighth transmission control unit), and the OF transistor Q7a (ninth transmission control unit) are connected in cascade between the photoelectric conversion element 31 and the charge-voltage conversion unit 33. The GS memory Mg1 is arranged at the connection node between the OF transistor Q4a and the OF transistor Q6a. The OF memory M1a is arranged at the connection node between the OF transistor Q6a and the OF transistor Q7a.

[0109] The OF transistor Q4a transfers the charge from the photoelectric conversion element 31 to the GS memory Mg1 when the signal GS1 input to the gate is at a high level. The OF transistor Q6a transfers the charge from the GS memory Mg1 to the OF memory M1a when the signal OF1 input to the gate is at a high level. The OF transistor Q7a transfers the charge from the OF memory M1a to the charge-voltage conversion unit 33 when the signal OF2 input to the gate is at a high level.

[0110] Figure 11 In [embodiment 2], the GS transistor Q5a (tenth transmission control unit), the OF transistor Q8a (eleventh transmission control unit), and the OF transistor Q9a (twelfth transmission control unit) are connected in cascade between the photoelectric conversion element 31 and the charge-voltage conversion unit 33. The GS memory Mg2 is provided at the connection node between the OF transistor Q5a and the OF transistor Q8a. The OF memory M2a is provided at the connection node between the OF transistor Q8a and the OF transistor Q9a.

[0111] When the signal GS2 input to the gate of the OF transistor Q5a is at a high level, the OF transistor Q5a transfers charges from the photoelectric conversion element 31 to the GS memory Mg2. When the signal OF3 input to the gate of the OF transistor Q8a is at a high level, the OF transistor Q8a transfers charges from the GS memory Mg2 to the OF memory M2a. When the signal OF4 input to the gate of the OF transistor Q9a is at a high level, the OF transistor Q9a transfers charges from the OF memory M2a to the charge-voltage conversion unit 33.

[0112] As described above, using the same number of transistors as in the Figure 3 transfer unit 32 in Figure 11 the transfer unit 32a in

[0113] can be provided with two GS memories Mg1 and Mg2 in the same manner as Figure 3 the pixel 30 in

[0114] Figure 12A Preferably, the saturation charge levels of the OF memories M1a and M2a are higher than those of the GS memories Mg1 and Mg2 in the same manner as the pixel 30 in Figure 12B In the same manner as the pixel 30, the GS memories Mg1 and Mg2 hold the charges accumulated in the photoelectric conversion element 31 at the same timing as the other pixels 30a. Figure 12A shows the exposure period of the first frame of the pixel 30a. As Figure 12A shown, in the first frame, the charges e of the photoelectric conversion element 31 are transferred to the OF memory M1a via the GS memory Mg1. The charges e of the photoelectric conversion element 31 are transferred to the GS memory Mg1 immediately before the readout period of the first frame.

[0115] Figure 12B shows the readout period of the first frame of the pixel 30a and the exposure period of the second frame. During the readout period of the first frame, the charges e of the OF memory M1a and the GS memory Mg1 are transferred to the charge-voltage conversion unit 33. In parallel with this, in the exposure period of the second frame, in the same manner as Figure 12A shown, the charges e of the photoelectric conversion element 31 are transferred to the OF memory M2a via the GS memory Mg2.

[0116] In the second frame and subsequent frames, the OF memory M1a and the GS memory Mg1 are alternately used with the OF memory M2a and the GS memory Mg2 in the same manner as in the first embodiment to read and transfer the charges e. That is, in the same manner as in the first embodiment, the light detection device 2 according to the second embodiment can also expose the pixel 30a while reading the charges e from the pixel 30a.

[0117] As described above, the pixel 30a according to the second embodiment includes two GS memories Mg1 and Mg2. That is, the pixel 30a is characterized in that even when one of the GS memories Mg1 and Mg2 is transferring the charge e to the charge-voltage conversion unit 33, the other of the GS memories Mg1 and Mg2 can hold the charge e of the photoelectric conversion element 31. Therefore, the pixel 30a can improve the frame rate more than the pixel 30 according to the first embodiment.

[0118] Figure 13 is a timing chart showing the control of the pixel 30a according to the second embodiment of the present disclosure. Figure 13 The time period from t31 to t55 shown includes initialization, the exposure period and the readout period of the first frame, the exposure period and the readout period of the second frame, and the exposure period of the third frame.

[0119] During the time period from t31 to t32, the pixel 30a before capturing an image is initialized. Specifically, in the same manner, the reset signal RST, the signals GS1 and GS2, and the signals OF1 to OF4 are set to high level at the same timing within a fixed time period. Therefore, the charges of the photoelectric conversion element 31, the GS memories Mg1 and Mg2, the OF memories M1a and M2a, and the charge-voltage conversion unit 33 are discharged to the node of the high-voltage side power supply VDD, and the charge-voltage conversion unit 33 becomes the reset level voltage. Figure 7 During the time period from t33 to t35 is the exposure period of the first frame. During the exposure period of the first frame, the charge of the photoelectric conversion element 31 is transferred to the OF memory M1a and the GS memory Mg1.

[0120] During the time period from t33 to t34, since the signal GS1 is set to high level, the signal OF1 is set to high level, and the OF transistors Q4a and Q6a are switched to on, the charge accumulated in the photoelectric conversion element 31 is transferred to the OF memory M1a via the GS memory Mg1. Figure 13 In the same way, the charge holding periods of the OF memories M1a and M2a corresponding to each frame are longer than the charge holding periods of the GS memories Mg1 and Mg2.

[0121] During the time period from t34 to t35, the signal GS1 is at high level, and the signal OF1 is switched to low level. As a result, the OF transistor Q4a is switched to on, and the OF transistor Q6a is turned off, and the charge of the photoelectric conversion element 31 is transferred to the GS memory Mg1.

[0122] Note that, in the same way, the charge holding periods of the OF memories M1a and M2a corresponding to each frame are longer than the charge holding periods of the GS memories Mg1 and Mg2.

[0123] Note that, in the same way, the charge holding periods of the OF memories M1a and M2a corresponding to each frame are longer than the charge holding periods of the GS memories Mg1 and Mg2. Figure 7 In the same way, the charge holding periods of the OF memories M1a and M2a corresponding to each frame are longer than the charge holding periods of the GS memories Mg1 and Mg2.

[0124] The time from t35 to t45 is the readout period of the first frame. During the time from t36 to t37, the signals OF1 and OF2 are at a low level, and the selection signal SEL is at a high level. Therefore, in the same manner as Figure 7 , the pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read.

[0125] During the time from t38 to t39, the signal OF2 and the selection signal SEL are set to a high level. Since the OF transistor Q7a is turned on, the charge of the OF memory M1a is transferred to the charge-voltage conversion unit 33. Therefore, the pixel signal (D-phase signal) corresponding to the charge generated by adding the charge of the OF memory M1a to the charge of the charge-voltage conversion unit 33 is read.

[0126] During the time from t39 to t40, the reset signal RST is set to a high level. Therefore, the charge of the charge-voltage conversion unit 33 is discharged to the node of the high-voltage side power supply VDD.

[0127] During the time from t40 to t41, the signals OF1, OF2, and the selection signal SEL are set to a high level. Since the OF transistors Q6a and Q7a are turned on, the charges in the GS memory Mg1 and the OF memory M1a are transferred to the charge-voltage conversion unit 33. Therefore, the pixel signal (D-phase signal) corresponding to the charge generated by further adding the charge of the OF memory M1a to the charge of the GS memory Mg1 is read from the charge-voltage conversion unit 33.

[0128] During the time from t41 to t43, the signals OF1, OF2, and the reset signal RST are set to a high level, and the charges of the GS memory Mg1, the OF memory M1a, and the charge-voltage conversion unit 33 are discharged to the node of the high-voltage side power supply VDD. Therefore, the charge-voltage conversion unit 33 becomes a reset level voltage.

[0129] During the time from t43 to t44, the signals OF1, OF2, and the selection signal SEL are set to a high level, and the pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read.

[0130] The time from t35 to t45 is the readout period of the first frame and the exposure period of the second frame. During the time from t35 to t42, the signal GS2 is set to a high level, and the signal OF3 is set to a high level. Therefore, the charge accumulated in the photoelectric conversion element 31 is transferred to the OF memory M2a. During the time from t42 to t45, the signal GS2 is at a high level, and the signal OF3 is switched to a low level. Therefore, the charge of the photoelectric conversion element 31 is transferred to the GS memory Mg2.

[0131] At this time, in Figure 7 , the time t42 when the signal OF3 is set to a low level is set between the times t41 and t43 in the readout period of the first frame, but is not limited thereto. The time when the signal OF3 is switched to a low level can be set at any time during the readout period of the first frame.

[0132] The time from t45 to t55 is the readout period of the second frame. During the readout period of the second frame, charges are read from the GS memory Mg2 and the OF memory M2a under the same control as the readout period of the first frame.

[0133] That is, at the time from t46 to t47, the selection signal SEL is set to a high level, and a pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read. During the time from t48 to t49, the signals OF4 and the selection signal SEL are set to a high level, and a pixel signal (D-phase signal) corresponding to the charge generated by adding the charge of the OF memory M2a to the charge of the charge-voltage conversion unit 33 is read.

[0134] During the time from t49 to t50, the reset signal RST is set to a high level, and the charge-voltage conversion unit 33 becomes the reset level voltage. During the time from t50 to t51, the signals OF3, OF4, and the selection signal SEL are set to a high level, and a pixel signal (D-phase signal) corresponding to the charge generated by further adding the charge of the OF memory M2a to the charge of the GS memory Mg2 is read from the charge-voltage conversion unit 33. During the time from t51 to t53, the signals OF3, OF4, and the reset signal RST are set to a high level, and the charge-voltage conversion unit 33 becomes the reset level voltage. During the time from t53 to t54, the signals OF3, OF4, and the selection signal SEL are set to a high level, and a pixel signal (P-phase signal) corresponding to the reset level of the charge-voltage conversion unit 33 is read.

[0135] The time from t45 to t55 is the readout period of the second frame and the exposure period of the third frame. During the time from t45 to t52, the signal GS1 is set to a high level, and the signal OF1 is set to a high level. Therefore, the charge accumulated in the photoelectric conversion element 31 is transferred to the OF memory M1a. During the time from t52 to t55, the signal GS1 is at a high level, and the signal OF1 is at a low level, and the charge of the photoelectric conversion element 31 is transferred to the GS memory Mg1.

[0136] As described above, the OF transistors Q4a or Q5a transfer the charges accumulated in the photoelectric conversion element 31 to the GS memories Mg1 or Mg2 in units of frames, respectively. The OF transistors Q6a or Q8a alternately transfer the charges from the GS memories Mg1 or Mg2 to the OF memories M1a or M2a in units of frames. The OF transistors Q7a or Q9a alternately transfer the charges held in the OF memories M1a or M2a to the charge-voltage conversion unit 33 in units of frames. Each of the plurality of pixels 30a in the pixel array unit 20 alternately holds the charges in the OF memories M1a or M2a in units of frames and alternately holds the charges in the GS memories Mg1 or Mg2 in units of frames.

[0137] Figure 13 The timing chart in Figure 7 differs from the timing chart in

[0138] in that an exposure period for the next frame is also provided when reading the charges from the GS memories Mg1 or Mg2. That is, compared with the pixel 30 of the first embodiment, the pixel 30a according to the second embodiment can further extend the exposure period during the imaging operation.

[0139] (Application Example) The technology of the present disclosure (this technology) can be applied to various products. For example, the technology according to the present disclosure can be applied to an endoscopic surgery system.

[0140] Figure 14 shows an example of a schematic configuration of an endoscopic surgery system to which the technology of the present disclosure (this technology) is applied.

[0141] Figure 14Shows a state where a surgeon (doctor) 11131 performs surgery on a patient 11132 on a hospital bed 11133 using an endoscopic surgery system 11000. As shown in the figure, the endoscopic surgery system 11000 includes an endoscope 11100, other surgical tools 11110 such as a pneumoperitoneum tube 11111 or an energy treatment tool 11112, a support arm device 11120 for supporting the endoscope 11100, and a cart 11200 equipped with various devices for endoscopic surgery.

[0142] The endoscope 11100 includes a lens barrel 11101 and a camera head 11102. A region of the lens barrel with a predetermined length starting from the distal end of the endoscope is inserted into the body cavity of the patient 11132, and the camera head is connected to the proximal end of the lens barrel 11101. In the illustrated example, the endoscope 11100 is configured as a so-called rigid endoscope having a rigid lens barrel 11101, but the endoscope 11100 can also be configured as a so-called flexible endoscope having a flexible lens barrel.

[0143] An opening in which an objective lens is assembled is provided at the distal end of the lens barrel 11101. A light source device 11203 is connected to the endoscope 11100, and the light generated by the light source device 11203 is guided through an optical fiber extending inside the lens barrel 11101 to the distal end of the lens barrel 11101, and the light is irradiated onto an observation object in the body cavity of the patient 11132 via the objective lens. The endoscope 11100 can be a direct-view endoscope, a forward-oblique endoscope, or a side-view endoscope.

[0144] An optical system and an imaging element are provided inside the camera head 11102, and the reflected light (observation light) from the observation object is converged on the imaging element through the optical system. The imaging element performs photoelectric conversion on the observation light and generates an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image. The image signal is transmitted as raw data to a camera control unit (CCU) 11201.

[0145] The CCU 11201 is composed of a central processing unit (CPU), a graphics processing unit (GPU), etc., and comprehensively controls the operations of the endoscope 11100 and the display device 11202. In addition, for example, the CCU 11201 receives the image signal from the camera head 11102 and performs various image processing operations for displaying an image based on the image signal on the display device, such as developing the image signal (demosaicing processing), etc.

[0146] The display device 11202 displays an image of the image signal processed by the CCU 11201 under the control of the CCU 11201.

[0147] The light source device 11203 includes a light source such as a light-emitting diode (LED) and provides illumination light to the endoscope 11100 when an image of a surgical site or the like is to be captured.

[0148] The input device 11204 is an input interface of the endoscopic surgery system 11000. A user can input various types of information or commands into the endoscopic surgery system 11000 via the input device 11204. For example, the user inputs a command to change the imaging conditions (type of illumination light, magnification, focal length, etc.) of the endoscope 11100.

[0149] The treatment tool control device 11205 controls the drive of the energy treatment tool 11112 to cauterize or incise tissue and seal blood vessels. The pneumoperitoneum device 11206 supplies gas into the body cavity of the patient 11132 through the pneumoperitoneum tube 11111 to expand the body cavity, so as to ensure the field of view of the endoscope 11100 and ensure the working space for the surgeon. The recorder 11207 is a device capable of recording various information related to the surgery. The printer 11208 is a device capable of printing various information related to the surgery in various forms such as text, image, or graph.

[0150] For example, the light source device 11203 that provides illumination light to the endoscope 11100 to image the surgical site can be composed of an LED, a laser light source, or a white light source obtained by combining them. In the case where the white light source is composed of a combination of RGB laser light sources, since the output intensity and output timing of each color (each wavelength) can be controlled with high precision, the light source device 11203 can adjust the white balance of the captured image. In this case, by irradiating the observation object with the laser light from the RGB laser light sources in a time-division manner and controlling the drive of the imaging element of the camera head 11102 in synchronization with the irradiation timing, it is also possible to capture images corresponding to RGB in a time-division manner. According to this method, even when no color filter is provided in the imaging element, a color image can be obtained.

[0151] In addition, the drive of the light source device 11203 can be controlled so as to change the intensity of the output light at predetermined intervals. By controlling the drive of the imaging element of the camera head 11102 in synchronization with the change timing of the light intensity, acquiring images in a time-division manner, and synthesizing the images, a high-dynamic-range image without so-called blocked shadows and clipped whites can be generated.

[0152] The light source device 11203 may have a configuration capable of providing light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, by utilizing the wavelength dependence of light absorbed by body tissue and emitting light in a narrower wavelength band than the illumination light (i.e., white light) during normal observation, so-called narrow-band light observation (narrow-band imaging) for capturing high-contrast images of predetermined tissues such as superficial mucosal blood vessels is performed. Alternatively, in special light observation, fluorescence observation may be performed to obtain an image by fluorescence generated by emitting excitation light. Fluorescence observation may be performed by irradiating body tissue with excitation light and observing the fluorescence from the body tissue (autofluorescence observation), or by locally injecting a reagent such as indocyanine green (ICG) into the body tissue, irradiating the body tissue with excitation light corresponding to the fluorescence wavelength of the reagent, and obtaining a fluorescence image. The light source device 11203 may be configured to supply narrow-band light and / or excitation light corresponding to such special light observation.

[0153] Figure 15 shows an Figure 14 example of the functional configuration of the camera head 11102 and the CCU 11201 shown.

[0154] The camera head 11102 includes a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 includes a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are communicably connected to each other via a transmission cable 11400.

[0155] The lens unit 11401 is an optical system provided at the connection portion with the lens barrel 11101. Observation light converged from the distal end of the lens barrel 11101 is guided to the camera head 11102 and incident on the lens unit 11401. The lens unit 11401 is configured as a combination of a plurality of lenses including a zoom lens and a focusing lens.

[0156] The number of imaging elements constituting the imaging unit 11402 may be one imaging element (so-called single-board type) or multiple imaging elements (so-called multi-board type). For example, in the case where the imaging unit 11402 is of the multi-board type, each imaging element may generate an image signal corresponding to RGB, and a color image may be obtained by synthesizing these image signals. Alternatively, the imaging unit 11402 may include a pair of imaging elements for acquiring a right-eye image signal and a left-eye image signal corresponding to three-dimensional (3D) display. By performing 3D display, the surgeon 11131 can more accurately identify the depth of the living tissue in the surgical site. When the imaging unit 11402 is configured as a multi-board type, a system of multiple lens units 11401 may also be provided corresponding to each imaging element.

[0157] The imaging unit 11402 is not necessarily provided in the camera head 11102. For example, the imaging unit 11402 may be provided immediately behind the objective lens inside the lens barrel 11101.

[0158] The drive unit 11403 is composed of an actuator and moves the zoom lens and the focusing lens of the lens unit 11401 along the optical axis by a predetermined distance under the control of the camera head control unit 11405. Therefore, the magnification and focus of the image captured by the imaging unit 11402 can be appropriately adjusted.

[0159] The communication unit 11404 is composed of a communication device for exchanging various information with the CCU 11201. The communication unit 11404 transmits the image signal acquired from the imaging unit 11402 to the CCU 11201 as RAW data via the transmission cable 11400.

[0160] The communication unit 11404 receives a control signal for controlling the drive of the camera head 11102 from the CCU 11201 and provides the control signal to the camera head control unit 11405. For example, the control signal includes information related to imaging conditions, such as information specifying the frame rate of the captured image, information specifying the exposure value when capturing the image, and / or information specifying the magnification and focus of the captured image.

[0161] Imaging conditions such as frame rate, exposure value, magnification, or focus can be appropriately specified by the user or can be automatically set by the control unit 11413 of the CCU 11201 based on the acquired image signal. In the latter case, the endoscope 11100 has a so-called automatic exposure (AE) function, a so-called automatic focusing (AF) function, and a so-called automatic white balance (AWB) function.

[0162] The camera head control unit 11405 controls the drive of the camera head 11102 based on the control signal received from the CCU 11201 via the communication unit 11404.

[0163] The communication unit 11411 is composed of a communication device for transmitting and receiving various types of information between the camera head 11102. The communication unit 11411 receives the image signal transmitted from the camera head 11102 via the transmission cable 11400.

[0164] In addition, the communication unit 11411 transmits a control signal for controlling the driving of the camera head 11102 to the camera head 11102. The image signal and the control signal can be transmitted through electrical communication, optical communication, etc.

[0165] The image processing unit 11412 performs various image processes on the image signal as RAW data transmitted from the camera head 11102.

[0166] The control unit 11413 performs various types of control to cause the endoscope 11100 to capture an image of a surgical site or the like, and displays the captured image obtained by capturing the image of the surgical site or the like. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102.

[0167] The control unit 11413 causes the display device 11202 to display the captured image showing the surgical site or the like based on the image signal that has been image-processed by the image processing unit 11412. At this time, the control unit 11413 can identify various objects in the captured image by using various image recognition techniques. For example, the control unit 11413 can identify surgical tools such as surgical forceps, specific living body parts, bleeding, fog when using the energy treatment tool 11112, etc. by detecting the shape, color, etc. of the edges of the objects included in the captured image. When the control unit 11413 causes the display device 11202 to display the captured image, the control unit 11413 can superimpose and display various surgical support information on the image of the surgical site by using the recognition result. By superimposing and displaying the surgical support information and presenting it to the surgeon 11131, the burden on the surgeon 11131 can be reduced and the surgeon 11131 can perform the surgery reliably.

[0168] The transmission cable 11400 that connects the camera head 11102 and the CCU 11201 to each other is an electrical signal cable that supports electrical signal communication, an optical fiber that supports optical communication, or a composite cable thereof.

[0169] Although wired communication is used with the transmission cable 11400 in the illustrated example, wireless communication can also be performed between the camera head 11102 and the CCU 11201.

[0170] So far, examples of endoscopic surgical systems to which the technology according to the present disclosure can be applied have been described. For example, in the above configuration, the technology according to the present disclosure can be applied to the camera head 11102. Specifically, Figure 1 the light detection device 2 or the electronic device 1 in Figure 1 can be applied to the imaging unit 11402 of the camera head 11102. By applying the technology according to the present disclosure to the imaging unit 11402, a clearer image of the surgical site can be obtained quickly, so that the surgeon can reliably examine the surgical site.

[0171] Here, although an endoscopic surgical system has been described as an example, the technology according to the present disclosure can be applied to other systems such as a microsurgical system, for example.

[0172] The present technology can have the following configuration. (1) A light detection device including a plurality of pixels, each pixel including: a photoelectric conversion element that accumulates charges corresponding to the incident light amount, wherein each of the plurality of pixels includes: a charge-voltage conversion unit that converts the charges accumulated in the photoelectric conversion element into a voltage; a first holding unit that holds the charges accumulated in the photoelectric conversion element; a second holding unit that alternately holds the charges accumulated in the photoelectric conversion element with the first holding unit; and a third holding unit that holds the charges accumulated in the photoelectric conversion element at the same timing as other pixels. (2) The light detection device according to (1), wherein when one of the first holding unit and the second holding unit transfers charges to the charge-voltage conversion unit, the other of the first holding unit and the second holding unit holds the charges accumulated in the photoelectric conversion element. (3) The light detection device according to (1) or (2), wherein each of the plurality of pixels alternately holds charges in the first holding unit or the second holding unit in units of frames, and holds charges in the third holding unit for each frame. (4) The light detection device according to any one of (1) to (3), wherein each of the plurality of pixels outputs a pixel signal of a reset level and a pixel signal corresponding to the charges held in the first holding unit or the second holding unit and the charges held in the third holding unit for each frame, and the pixel signal of the reset level and the pixel signal corresponding to the charges are output at different timings from each other. ‎(5) The optical detection device according to any one of (1) to (4), wherein the saturation charge levels of the first holding unit and the second holding unit are higher than the saturation charge level of the third holding unit. ‎(6) The optical detection device according to any one of (1) to (5), wherein the charge holding periods corresponding to each frame of the first holding unit and the second holding unit are longer than the charge holding period of the third holding unit. ‎(7) The optical detection device according to any one of (1) to (6), wherein each of the plurality of pixels includes: A first transfer control unit that controls the transfer of charge from the photoelectric conversion element to the first holding unit; A second transfer control unit that controls the transfer of charge from the first holding unit to the charge-voltage conversion unit; A third transfer control unit that controls the transfer of charge from the photoelectric conversion element to the second holding unit; and A fourth transfer control unit that controls the transfer of charge from the second holding unit to the charge-voltage conversion unit. ‎(8) The optical detection device according to (7), wherein the first transfer control unit or the third transfer control unit alternately transfers the charge accumulated in the photoelectric conversion element to the first holding unit or the second holding unit in units of frames. (9) The optical detection device according to (7) or (8), wherein the second transfer control unit or the fourth transfer control unit alternately transfers the charge held in the first holding unit or the second holding unit to the charge-voltage conversion unit in units of frames. ‎(10) The optical detection device according to any one of (7) to (9), wherein when the first transfer control unit transfers charge from the photoelectric conversion element to the first holding unit, a negative bias voltage is supplied to the gate of the third transfer control unit, and when the third transfer control unit transfers charge from the photoelectric conversion element to the second holding unit, a negative bias voltage is supplied to the gate of the first transfer control unit. ‎(11) The optical detection device according to any one of (7) to (10), wherein each of the plurality of pixels includes: A fifth transfer control unit that controls the transfer of charge from the photoelectric conversion element to the third holding unit; and A sixth transfer control unit that controls the transfer of charge from the third holding unit to the charge-voltage conversion unit. (12) The optical detection device according to (11), wherein the fifth transfer control unit transfers charges from the photoelectric conversion element to the third holding unit for each frame at the same timing as other pixels, and the sixth transfer control unit transfers charges from the third holding unit to the charge-voltage conversion unit for each frame. (13) The optical detection device according to (11) or (12), wherein for each frame, after charges are transferred from the photoelectric conversion element to the first holding unit or the second holding unit, the fifth transfer control unit transfers charges from the photoelectric conversion element to the third holding unit. (14) The optical detection device according to (1) or (2), wherein each pixel among the plurality of pixels includes: a fourth holding unit that holds the charges accumulated in the photoelectric conversion element at the same timing as other pixels, and each pixel among the plurality of pixels alternately holds charges in the first holding unit or the second holding unit in units of frames, and alternately holds charges in the third holding unit or the fourth holding unit in units of frames. (15) The optical detection device according to (14), wherein each pixel among the plurality of pixels outputs a pixel signal at a reset level and a pixel signal corresponding to the charges held in the first holding unit and the third holding unit, or a pixel signal corresponding to the charges held in the second holding unit or the fourth holding unit for each frame, and the pixel signal at the reset level and the pixel signal corresponding to the charges are output at different timings from each other. (16) The optical detection device according to (14) or (15), wherein the saturation charge levels of the first holding unit and the second holding unit are higher than the saturation charge levels of the third holding unit and the fourth holding unit. (17) The optical detection device according to any one of (14) to (16), wherein the charge holding periods of the first holding unit and the second holding unit corresponding to each frame are longer than the charge holding periods of the third holding unit and the fourth holding unit. (18) The optical detection device according to any one of (14) to (17), wherein each pixel among the plurality of pixels includes: a seventh transfer control unit that controls the transfer of charges from the photoelectric conversion element to the third holding unit; an eighth transfer control unit that controls the transfer of charges from the third holding unit to the first holding unit; A ninth transfer control unit that controls the transfer of charge from the first holding unit to the charge-voltage conversion unit; A tenth transfer control unit that controls the transfer of charge from the photoelectric conversion element to the fourth holding unit; An eleventh transfer control unit that controls the transfer of charge from the fourth holding unit to the second holding unit; and A twelfth transfer control unit that controls the transfer of charge from the second holding unit to the charge-voltage conversion unit. ‎(19) The light detection device according to (18), wherein the seventh transfer control unit or the tenth transfer control unit alternately transfers the charge accumulated in the photoelectric conversion element to the third holding unit or the fourth holding unit in units of frames, the eighth transfer control unit or the eleventh transfer control unit alternately transfers charge from the third holding unit or the fourth holding unit to the first holding unit or the second holding unit in units of frames, and the ninth transfer control unit or the twelfth transfer control unit alternately transfers the charge held in the first holding unit or the second holding unit to the charge-voltage conversion unit in units of frames. (20) An electronic device, comprising: A light detection device; A processing unit that processes pixel data output from the light detection device, wherein the light detection device includes: A plurality of pixels, each pixel including: a photoelectric conversion element that accumulates charge corresponding to the amount of incident light, Each of the plurality of pixels includes: A charge-voltage conversion unit that converts the charge accumulated in the photoelectric conversion element into a voltage; A first holding unit that holds the charge accumulated in the photoelectric conversion element; A second holding unit that alternately holds the charge accumulated in the photoelectric conversion element with the first holding unit; and A third holding unit that holds the charge accumulated in the photoelectric conversion element at the same timing as other pixels.

[0173] Aspects of the present disclosure are not limited to the above-described respective embodiments, and include various modifications that can be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above effects. In other words, various additions, modifications, and partial deletions can be made without departing from the conceptual idea and spirit of the present disclosure, and the conceptual idea and spirit of the present disclosure can be derived from the details defined in the claims and their equivalents. List of Reference Numerals

[0174] 1 Electronic device 2, 2a Light detection device 3 Processing unit 4 Control unit 5 Recording unit 11 Imaging lens 12 Transmission line 13 Control line 20 Pixel array unit 21 Vertical drive circuit 22 System control circuit 23 Signal processing unit 30, 30a, 100 Pixel 31 Photoelectric conversion element 32, 32a Transmission unit 33 Charge-voltage conversion unit 41 Pixel chip 42 Logic chip 43 First pixel chip 44 Second pixel chip

Claims

1. An optical detection device, comprising a plurality of pixels, each pixel including: A photoelectric conversion element that accumulates charges corresponding to the incident light amount, wherein each of the plurality of pixels includes: a charge-voltage conversion unit that converts the charges accumulated in the photoelectric conversion element into a voltage; a first holding unit that holds the charges accumulated in the photoelectric conversion element; a second holding unit that alternately holds the charges accumulated in the photoelectric conversion element with the first holding unit; and a third holding unit that holds the charges accumulated in the photoelectric conversion element at the same timing as other pixels.

2. The optical detection device according to claim 1, wherein, When one of the first holding unit and the second holding unit transfers charges to the charge-voltage conversion unit, the other of the first holding unit and the second holding unit holds the charges accumulated in the photoelectric conversion element.

3. The light detection device according to claim 1, wherein each of the plurality of pixels alternately holds charges in the first holding unit or the second holding unit in units of frames, and holds charges in the third holding unit for each frame.

4. The light detection device according to claim 1, wherein each of the plurality of pixels outputs a pixel signal at a reset level and a pixel signal corresponding to the charges held in the first holding unit or the second holding unit and the charges held in the third holding unit for each frame, and the pixel signal at the reset level and the pixel signal corresponding to the charges are output at different timings from each other.

5. The light detection device according to claim 1, wherein the saturation charge levels of the first holding unit and the second holding unit are higher than the saturation charge level of the third holding unit.

6. The light detection device according to claim 1, wherein the charge holding periods of the first holding unit and the second holding unit corresponding to each frame are longer than the charge holding period of the third holding unit.

7. The light detection device according to claim 1, wherein each of the plurality of pixels includes: a first transfer control unit that controls the transfer of charges from the photoelectric conversion element to the first holding unit; a second transfer control unit that controls the transfer of charges from the first holding unit to the charge-voltage conversion unit; a third transfer control unit that controls the transfer of charges from the photoelectric conversion element to the second holding unit; and a fourth transfer control unit that controls the transfer of charges from the second holding unit to the charge-voltage conversion unit.

8. The light detection device according to claim 7, wherein the first transfer control unit or the third transfer control unit alternately transfers the charges accumulated in the photoelectric conversion element to the first holding unit or the second holding unit in units of frames.

9. The light detection device according to claim 7, wherein the second transfer control unit or the fourth transfer control unit alternately transfers the charges held in the first holding unit or the second holding unit to the charge-voltage conversion unit in units of frames.

10. The optical detection device according to claim 7, wherein when the first transfer control unit transfers charges from the photoelectric conversion element to the first holding unit, a negative bias voltage is supplied to the gate of the third transfer control unit, and when the third transfer control unit transfers charges from the photoelectric conversion element to the second holding unit, a negative bias voltage is supplied to the gate of the first transfer control unit.

11. The optical detection device according to claim 7, wherein each of the plurality of pixels includes: a fifth transfer control unit that controls the transfer of charges from the photoelectric conversion element to the third holding unit; and a sixth transfer control unit that controls the transfer of charges from the third holding unit to the charge-voltage conversion unit.

12. The optical detection device according to claim 11, wherein the fifth transfer control unit transfers charges from the photoelectric conversion element to the third holding unit at the same timing as other pixels for each frame, and the sixth transfer control unit transfers charges from the third holding unit to the charge-voltage conversion unit for each frame.

13. The optical detection device according to claim 11, wherein for each frame, after charges are transferred from the photoelectric conversion element to the first holding unit or the second holding unit, the fifth transfer control unit transfers charges from the photoelectric conversion element to the third holding unit.

14. The optical detection device according to claim 1, wherein Each of the plurality of pixels includes: a fourth holding unit that holds the charges accumulated in the photoelectric conversion element at the same timing as other pixels, and each of the plurality of pixels alternately holds charges in the first holding unit or the second holding unit in units of frames, and alternately holds charges in the third holding unit or the fourth holding unit in units of frames.

15. The optical detection device according to claim 14, wherein each of the plurality of pixels outputs a pixel signal at a reset level and a pixel signal corresponding to the charges held in the first holding unit and the third holding unit, or a pixel signal corresponding to the charges held in the second holding unit or the fourth holding unit for each frame, and the pixel signal at the reset level and the pixel signal corresponding to the charges are output at different timings from each other.

16. The optical detection device according to claim 14, wherein the saturation charge levels of the first holding unit and the second holding unit are higher than the saturation charge levels of the third holding unit and the fourth holding unit.

17. The optical detection device according to claim 14, wherein the charge holding periods corresponding to each frame of the first holding unit and the second holding unit are longer than the charge holding periods of the third holding unit and the fourth holding unit.

18. The optical detection device according to claim 14, wherein each of the plurality of pixels includes: a seventh transfer control unit that controls the transfer of charges from the photoelectric conversion element to the third holding unit; An eighth transfer control unit that controls the transfer of charge from the third holding unit to the first holding unit; A ninth transfer control unit that controls the transfer of charge from the first holding unit to the charge-voltage conversion unit; A tenth transfer control unit that controls the transfer of charge from the photoelectric conversion element to the fourth holding unit; An eleventh transfer control unit that controls the transfer of charge from the fourth holding unit to the second holding unit; and A twelfth transfer control unit that controls the transfer of charge from the second holding unit to the charge-voltage conversion unit.

19. The optical detection device according to claim 18, wherein the seventh transfer control unit or the tenth transfer control unit alternately transfers the charge accumulated in the photoelectric conversion element to the third holding unit or the fourth holding unit in units of frames, the eighth transfer control unit or the eleventh transfer control unit alternately transfers charge from the third holding unit or the fourth holding unit to the first holding unit or the second holding unit in units of frames, and the ninth transfer control unit or the twelfth transfer control unit alternately transfers the charge held in the first holding unit or the second holding unit to the charge-voltage conversion unit in units of frames.

20. An electronic device, comprising: An optical detection device; A processing unit that processes pixel data output from the optical detection device, wherein The optical detection device includes: A plurality of pixels, each pixel including: a photoelectric conversion element that accumulates charge corresponding to the amount of incident light, and Each of the plurality of pixels includes: A charge-voltage conversion unit that converts the charge accumulated in the photoelectric conversion element into a voltage; A first holding unit that holds the charge accumulated in the photoelectric conversion element; A second holding unit that alternately holds the charge accumulated in the photoelectric conversion element with the first holding unit; and A third holding unit that holds the charge accumulated in the photoelectric conversion element at the same timing as other pixels.

Images