Semiconductor device

Abstract

To provide a semiconductor device that can perform signal synchronization and control optimally.SOLUTION: The semiconductor device has a signal generation unit that generates a first signal, a synchronization unit that synchronizes the first signal with either a second signal or a third signal, both of which are mutually synchronized and are input from an external source, and a control unit that controls at least one of the second signal and the third signal on the basis of the first signal after synchronization by the synchronization unit.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a semiconductor device. [Background technology]

[0002] Patent Document 1 discloses a synchronization circuit that synchronizes a plurality of mutually asynchronous signals and outputs them to another circuit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-167920 A Summary of the Invention [Problem to be solved by the invention]

[0004] 2. Description of the Related Art In a semiconductor device having a function of synchronizing an externally input signal with an internally generated signal, there are cases where more suitable signal synchronization and control is required.

[0005] An object of the present invention is to provide a semiconductor device that can suitably perform signal synchronization and control. [Means for solving the problem]

[0006] According to one disclosure of the present specification, there is provided a semiconductor device comprising: a signal generating unit that generates a first signal; a synchronization unit that synchronizes the first signal with either a second signal or a third signal that are input from outside and are synchronized with each other; and a control unit that controls at least one of the second signal and the third signal based on the first signal after synchronization by the synchronization unit. Effect of the Invention

[0007] According to the present invention, a semiconductor device capable of suitably performing signal synchronization and control is provided. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of a photoelectric conversion device according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of the configuration of a pixel according to the first embodiment; [Diagram 3] 4 is a timing chart showing the operation of the pixel according to the first embodiment. [Figure 4] 2 is a block diagram showing an example of the configuration of a vertical scanning circuit according to the first embodiment. FIG. [Diagram 5] 4 is a timing chart showing the operation of the vertical scanning circuit according to the first embodiment. [Figure 6] FIG. 11 is a block diagram showing an example of the configuration of a vertical scanning circuit according to a second embodiment. [Figure 7] FIG. 11 is a block diagram showing an example of the configuration of a pixel according to a second embodiment. [Figure 8] 10 is a timing chart showing the operation of the vertical scanning circuit according to the second embodiment. [Figure 9] FIG. 13 is a block diagram showing an example of the configuration of a vertical scanning circuit according to a third embodiment. [Figure 10] 13 is a timing chart showing the operation of the vertical scanning circuit according to the third embodiment. [Figure 11] FIG. 13 is a block diagram of a light detection system according to a fourth embodiment. [Figure 12] FIG. 13 is a block diagram of a light detection system according to a fifth embodiment. [Figure 13] FIG. 13 is a schematic diagram of an endoscopic surgery system according to a sixth embodiment. [Figure 14] FIG. 13 is a schematic diagram of a light detection system according to a seventh embodiment. [Figure 15] FIG. 13 is a schematic diagram of a moving body according to a seventh embodiment. [Figure 16] 13 is a flowchart showing the operation of the light detection system according to the seventh embodiment. [Figure 17] FIG. 13 is a diagram showing a specific example of an electronic device according to the eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The same elements or corresponding elements in multiple drawings are denoted by the same reference numerals, and the description thereof may be omitted or simplified.

[0010] [First embodiment] 1 is a block diagram of a photoelectric conversion device 1 according to this embodiment. The photoelectric conversion device 1 may be, for example, a solid-state imaging device, a focus detection device, a distance measuring device, or the like.

[0011] One of the distance measurement methods used in distance measuring devices is Time-of-Flight (ToF), which measures the distance from a distance measuring device to an object based on the time it takes for light to be emitted from a light source and for the light reflected from the object to be received by the pixels of the distance measuring device.

[0012] Also, one type of ToF is a time-gate ToF. In the time-gate ToF, the distance measuring device can selectively detect reflected light from an object at a distance corresponding to the photon detection period by detecting photons incident within the photon detection period. Here, the photon detection period is determined by a control signal (gate pulse) indicating a gate window. By determining the distance corresponding to the photon detection period in which the reflected light is detected, distance information to the object can be obtained. In the following description, the photoelectric conversion device 1 of this embodiment is assumed to be a distance measuring device that performs distance measurement by the time-gate ToF, but is not limited to this.

[0013] The control signal indicating this gate window needs to be synchronized with the light emission of the light source, and therefore may be input from outside the chip constituting the photoelectric conversion device 1. Meanwhile, the control signal used for signal readout in the photoelectric conversion device 1 may be generated within the photoelectric conversion device 1. In general, the control signal generated within the photoelectric conversion device 1 and the control signal input from outside the photoelectric conversion device 1 are asynchronous, and therefore it is necessary to synchronize these control signals within the photoelectric conversion device 1. In addition, the control signal generated within the photoelectric conversion device 1 may be used to control the enabling or disabling of the control signal input from outside the photoelectric conversion device 1. The photoelectric conversion device 1 of this embodiment includes a semiconductor device such as a synchronization circuit that synchronizes a plurality of asynchronous control signals as described above.

[0014] In this specification, "synchronization" includes not only a state in which multiple timings indicated by multiple signals are controlled to be simultaneous, but also a state in which multiple timings are controlled to have a predetermined time difference. "Asynchronous" means a state in which these controls are not performed and multiple timings indicated by multiple signals are determined independently. The above-mentioned timing is, for example, the time of the rising edge or falling edge of a signal. For example, when multiple signals are controlled so that the time difference between the edge times is a predetermined value (including zero), the multiple signals are synchronized. Also, the operation of adjusting the timing of one or more of multiple signals so that multiple timings indicated by multiple signals are simultaneous or have a predetermined time difference is sometimes called "synchronization".

[0015] The photoelectric conversion device 1 includes a pixel array 10, a vertical scanning circuit 20, a digital front-end circuit (hereinafter referred to as DFE) 30, an output circuit 40, an output line 52, a group of control signal lines 51, a group of control signal lines 53, and a group of control signal lines 54. The pixel array 10 includes a plurality of pixels 100 arranged in a plurality of rows and a plurality of columns. Each of the plurality of pixels 100 includes a photoelectric conversion unit 110 including an avalanche photodiode (hereinafter referred to as APD), and a pixel signal processing unit 120. The photoelectric conversion unit 110 converts incident light into an electric signal. The pixels 100 in each column are provided for each column and connected to an output line 52 extending in the column direction. The pixel signal processing unit 120 outputs the converted electric signal to the DFE 53 via the output line 52 of the corresponding column. The row direction refers to the left-right direction in FIG. 1, and the column direction refers to the up-down direction in FIG. 1. Each of the control signal line group 51, the control signal line group 53, and the control signal line group 54 includes a plurality of control signal lines.

[0016] The vertical scanning circuit 20 receives control signals input via a control signal line group 54 from an external device 2 disposed outside the semiconductor chip constituting the photoelectric conversion device 1, and control signals input via a control signal line group 53 from the DFE 30. Based on these control signals, the vertical scanning circuit 20 supplies control signals to each of the multiple pixels 100 via a control signal line group 51 that is provided for each row and extends in the row direction. The vertical scanning circuit 20 may include logic circuits such as a shift register and an address decoder.

[0017] The external device 2 is, for example, a control device for a distance measuring device that supplies control signals to the light source and the photoelectric conversion device 1. The control signal output from the external device 2 to the vertical scanning circuit 20 is input to the vertical scanning circuit 20 at a timing synchronized with the light emission timing of a light source for distance measuring arranged outside the photoelectric conversion device 1. This control signal is supplied to the pixel 100 after a predetermined control is performed in the vertical scanning circuit 20, and is used to control the photon detection period in the pixel 100. This allows the photoelectric conversion device 1 to selectively detect photons that are incident on the photoelectric conversion device 1 at a time delayed by a predetermined time from the light emission timing of the light source, and it is possible to obtain an image of an object at a specific distance, for example. In addition, the photon detection period, i.e., the distance at which the measurement is performed, is changed and the measurement is repeated, so that the photoelectric conversion device 1 can obtain distance information of the subject.

[0018] The signals input to the DFE 30 from each of the multiple pixels 100 are processed in the DFE 30. The DFE 30 outputs the processed signals via the output circuit 40 to a storage device or signal processing device external to the photoelectric conversion device 1.

[0019] The pixels 100 may be arranged one-dimensionally. Moreover, it is not necessary that one pixel signal processing unit 120 is provided for each pixel 100. For example, one pixel signal processing unit 120 may be shared by a plurality of pixels 100. In this case, the pixel signal processing unit 120 provides a signal processing function to the plurality of pixels 100 by sequentially processing signals output from the plurality of photoelectric conversion units 110.

[0020] Fig. 2 is a schematic block diagram showing an example of the configuration of one pixel of the photoelectric conversion unit 110 and the pixel signal processing unit 120 according to this embodiment. Note that in Fig. 2, the control signal lines constituting the control signal line group 51 between the vertical scanning circuit 20 and the pixel signal processing unit 120 in Fig. 1 are individually shown.

[0021] The photoelectric conversion unit 110 has an APD 111. The pixel signal processing unit 120 has a recharge circuit 121, a waveform shaping unit 122, a storage unit 123, and a selection circuit 124. The recharge circuit 121 is, for example, a MOS transistor, and has a first terminal, a second terminal, and a control terminal. It is sufficient that the pixel signal processing unit 120 has at least one of the waveform shaping unit 122, the storage unit 123, and the selection circuit 124, and the recharge circuit 121.

[0022] The APD 111 generates charges according to the incident light by photoelectric conversion. An electric potential VL is supplied to the anode of the APD 111. The cathode of the APD 111 is connected to a first terminal of the recharge circuit 121 and an input terminal of the waveform shaping unit 122. The electric potential of the connection node of the cathode of the APD 111, the first terminal of the recharge circuit 121, and the waveform shaping unit 122 is VCATH. A electric potential VH higher than the electric potential VL supplied to the anode of the APD 111 is supplied to a second terminal of the recharge circuit 121. As a result, a reverse bias voltage is supplied to the anode and cathode of the APD 111 such that the APD 111 performs an avalanche multiplication operation. When electric charges are generated by the incident light in the APD 111 to which the reverse bias voltage is supplied, the electric charges cause avalanche multiplication, and an avalanche current is generated.

[0023] In addition, there are two operation modes when a reverse bias voltage is supplied to the APD 111: a Geiger mode and a linear mode. The Geiger mode is a mode in which the APD 111 operates with a potential difference between the anode and the cathode that is greater than the breakdown voltage. The linear mode is a mode in which the APD 111 operates with a potential difference between the anode and the cathode that is close to or less than the breakdown voltage. The APD 111 may operate in either the linear mode or the Geiger mode.

[0024] An APD operated in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In this case, for example, the potential VL is −30V and the potential VH is 1V.

[0025] The recharge circuit 121 has a function of converting a change in avalanche current occurring in the APD 111 into a voltage signal. The recharge circuit 121 is controlled by a control signal VR_OUT input to a control terminal via one of the control signal lines in the control signal line group 51. The recharge circuit 121 can switch between a state in which a quench operation is performed and a state in which a recharge operation is performed according to the control signal VR_OUT. The quench operation is an operation in which the recharge circuit 121 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, suppressing the voltage supplied to the APD 111 and suppressing the avalanche multiplication. The recharge operation is an initialization operation in which a potential VH is supplied to the cathode of the APD 111 to initialize the cathode potential, thereby restoring the state in which avalanche multiplication is possible again.

[0026] The waveform shaping unit 122 shapes the potential change of the cathode of the APD 111 obtained upon photon detection, and outputs a pulse signal. The waveform shaping unit 122 may be, for example, an inverter circuit or a monostable circuit using an inverter circuit.

[0027] The storage unit 123 stores the presence or absence of a pulse signal input from the waveform shaping unit 122 within a predetermined period. The storage unit 123 may be configured to store only a signal indicating whether or not a pulse signal has been input, or may be configured to include a counter that counts and stores the number of pulse signals. The storage unit 123 is controlled by control signals VG_OUT and MEM_RES input to a control terminal via two control signal lines of the control signal line group 51. The storage unit 123 can switch between an input acceptance state in which the input of a pulse signal can be accepted and a holding state in which information on the input pulse signal is held, according to the control signal VG_OUT. That is, the control signal VG_OUT indicates the timing of signal holding in the storage unit 123. Also, the storage unit 123 can initialize a stored value according to the control signal MEM_RES. That is, the control signal MEM_RES indicates the timing of initializing a signal based on the incident light to the APD 111.

[0028] The selection circuit 124 is controlled by a control signal WRITE input to a control terminal via one of the control signal lines in the control signal line group 51. The selection circuit 124 outputs a signal held in the storage unit 123 to the output line 52 in response to the control signal WRITE. That is, the control signal WRITE indicates the timing of signal output from the pixel 100. The selection circuit 124 includes, for example, a buffer circuit for outputting a signal.

[0029] 2, the selection circuit 124 controls the signal output from the storage unit 123 to the output line 52, but the method of controlling the signal output to the output line 52 is not limited to this. For example, a switch such as a transistor may be disposed at a node between the recharge circuit 121 and the APD 111, between the photoelectric conversion unit 110 and the pixel signal processing unit 120, or the like, and the signal output to the output line 52 may be controlled by switching between electrical connection and non-connection. Also, the signal output to the output line 52 may be controlled by changing the value of the potential VH or potential VL supplied to the photoelectric conversion unit 110 using a switch such as a transistor.

[0030] 3 is a timing chart showing the operation of the pixel 100 according to this embodiment. In FIG. 3, the control signals VR_OUT and VG_OUT, the potential VCATH of the cathode of the APD 111, and the output signal DATA of the storage unit 123 are shown. "Photon incidence" in FIG. 3 shows the timing of incidence of photons to the APD 111. The recharge circuit 121 performs a quenching operation when the control signal VR_OUT is at a low level, and performs a recharge operation when the control signal VR_OUT is at a high level. In addition, the storage unit 123 is in a holding state when the control signal VG_OUT is at a low level, and in an input receiving state when the control signal VG_OUT is at a high level.

[0031] It is assumed that avalanche multiplication occurs in the APD 111 due to incident light at a time prior to time t0. As a result, at time t0, the potential VCATH is lower than the potential VH. Note that, at time t0, the control signals VR_OUT and VG_OUT are at a low level, and the value of the output signal DATA is "0."

[0032] At time t1, the control signal VR_OUT transitions from low level to high level, causing the recharge circuit 121 to switch to a state in which a recharge operation is performed, causing the potential VCATH to be recharged to the potential VH.

[0033] At time t2, the control signal VR_OUT transitions from high to low, causing the recharge circuit 121 to switch to a state in which a quenching operation is performed.

[0034] When a photon is incident on the APD 111 at time t3, a reverse current due to avalanche multiplication flows through the APD 111. At this time, the recharge circuit 121 is in a state in which a quench operation is being performed, so the potential VCATH drops. Thereafter, when the amount of potential drop becomes larger and the reverse bias voltage applied to the APD 111 becomes smaller, the avalanche multiplication in the APD 111 stops, and the drop in the potential VCATH stops at a certain value. When the potential VCATH transitions across the threshold value of the waveform shaping unit 122, the potential change is shaped by the waveform shaping unit 122. This shaped signal is input to the storage unit 123.

[0035] At time t4, the control signal VG_OUT transitions from low to high. This switches the state of the storage unit 123 to an input receiving state. At this time, the input signal to the storage unit 123 is at low level due to the photon incident at time t3, so the storage unit 123 receives a low-level signal indicating this photon detection.

[0036] At time t5, the control signal VG_OUT transitions from high level to low level. This switches the state of the storage unit 123 and puts it into a holding state. At this time, the output signal DATA of the storage unit 123 is updated and its value transitions from "0" to "1". The period from time t2 when the control signal VR_OUT transitions to low level to time t5 when the control signal VG_OUT transitions to low level is the photon detection period of the APD 111. In other words, the photon detection period of the APD 111 is controlled by the timing of the falling edges of the pulses of the control signals VR_OUT and VG_OUT.

[0037] Next, a method for generating the control signals VR_OUT, VG_OUT, MEM_RES, and WRITE input to the pixel 100 will be described. FIG. 4 is a block diagram showing an example of the configuration of the vertical scanning circuit 20 according to this embodiment. FIG. 4 shows the configuration of the vertical scanning circuit 20 and the connection relationship between the vertical scanning circuit 20, the DFE 30, the pixel 100, and the external device 2 in more detail. The vertical scanning circuit 20 has a synchronizer 201 (synchronization unit) and a control circuit 202 (control unit). In addition, the vertical scanning circuit 20 is a semiconductor device that synchronizes the input control signals.

[0038] As described above, the external device 2 is disposed outside the semiconductor chip constituting the photoelectric conversion device 1. The external device 2 outputs the control signal VG_IN (third signal) and the control signal VR_IN (second signal) to the vertical scanning circuit 20 via the control signal line group 54. The DFE 30 may be disposed in the semiconductor chip constituting the photoelectric conversion device 1. The DFE 30 (signal generating unit) outputs the control signal MEM_RES (sixth signal), the control signal WRITE (fifth signal), and the control signal P_RG_EN_IN (first signal) to the vertical scanning circuit 20 via the control signal line group 53. The control signals VG_IN and VR_IN are synchronous with each other. The control signals MEM_RES, WRITE, and P_RG_EN_IN are also synchronous with each other. The control signals VG_IN and VR_IN are asynchronous with the control signals MEM_RES, WRITE, and P_RG_EN_IN.

[0039] The control signal P_RG_EN_IN is input to an input terminal of the synchronizer 201. The control signal VG_IN is input to a control terminal of the synchronizer 201. The synchronizer 201 outputs the control signal P_RG_EN_IN as a control signal P_RG_EN_OUT to a control terminal of the control circuit 202 in synchronization with the falling edge of the control signal VG_IN. By this operation, the control signal P_RG_EN_OUT is synchronized with the control signals VG_IN and VR_IN. The synchronizer 201 may include a flip-flop circuit. The flip-flop circuit can realize the above-mentioned signal synchronization process by holding the control signal P_RG_EN_IN at the timing of the falling edge of the control signal VG_IN. The synchronizer 201 may also include a circuit in which two or more flip-flop circuits are connected in series to prevent the propagation of metastable states.

[0040] The control signals VG_IN and VR_IN are input to an input terminal of the control circuit 202. The control circuit 202 controls whether or not to output the control signals VG_IN and VR_IN to the pixels 100 as the control signals VG_OUT and VR_OUT based on the signal level of the control signal P_RG_EN_OUT. That is, the control signal P_RG_EN_OUT is a signal that switches between enabling and disabling the external output of the control signals VG_OUT and VR_OUT from the vertical scanning circuit 20. The control circuit 202 may include a selector or the like to perform this switching process.

[0041] The vertical scanning circuit 20 outputs the input control signals MEM_RES, WRITE as is to the pixels 100. As described above, the vertical scanning circuit 20 receives the control signals MEM_RES, WRITE, P_RG_EN_IN generated inside the photoelectric conversion device 1, and the control signals VG_IN, VR_IN input from outside the photoelectric conversion device 1. The vertical scanning circuit 20 outputs the control signals MEM_RES, WRITE, VG_OUT, VR_OUT to the pixels 100 based on these control signals.

[0042] FIG. 5 is a timing chart showing the operation of the vertical scanning circuit 20 according to this embodiment. FIG. 5 shows control signals MEM_RES, WRITE, P_RG_EN_IN, P_RG_EN_OUT, VG_IN, VR_IN, VG_OUT, and VR_OUT. When P_RG_EN_OUT is at a low level, the control circuit 202 enables the output of the control signals VG_OUT and VR_OUT, and when P_RG_EN_OUT is at a high level, the control circuit 202 disables the output of the control signals VG_OUT and VR_OUT. The storage unit 123 initializes the value stored therein when the control signal MEM_RES becomes high. The selection circuit 124 outputs the signal held in the storage unit 123 to the output line 52 when the control signal WRITE becomes high.

[0043] At time t10, the control signal VR_IN goes to high level. At time t10, the control signal P_RG_EN_OUT is at low level, and the output of the control signal VR_OUT in the control circuit 202 is enabled. Therefore, at time t10, the control signal VR_OUT also goes to high level.

[0044] At time t11, the control signal VR_IN goes to low level. This causes the control signal VR_OUT to also go to low level. The operation from time t10 to time t11 corresponds to the operation from time t1 to time t2 in FIG. 3. That is, the control signal VR_OUT goes to high level from time t10 to time t11, causing a recharge operation in the pixel 100.

[0045] At time t12, the control signal P_RG_EN_IN goes to high level. Since the control signal VG_IN is maintained at low level at time t12, the level of the output signal of the synchronizer 201 does not change at this point. Therefore, the control signal P_RG_EN_OUT remains at low level at time t12.

[0046] At time t13, the control signal VG_IN goes to high level. At time t13, the control signal P_RG_EN_OUT is at low level, and the output of the control signal VG_OUT from the control circuit 202 is enabled. Therefore, at time t13, the control signal VR_OUT also goes to high level.

[0047] At time t14, the control signal VG_IN goes to low level. This causes the control signal VG_OUT to also go to low level. The operation from time t13 to time t14 corresponds to the operation from time t4 to time t5 in FIG. 3. That is, by the control signal VG_OUT going to high level from time t13 to time t14, the pixel 100 performs a signal reception operation based on the detection of a photon, and the value in the memory unit 123 is updated. The period from time t11 to time t14 is the photon detection period.

[0048] Also, at time t14, in synchronization with the falling edge of the control signal VG_IN, the synchronizer 201 outputs the control signal P_RG_EN_IN as the control signal P_RG_EN_OUT to the control terminal of the control circuit 202. As a result, the control signal P_RG_EN_OUT becomes high level, and the outputs of the control signals VG_OUT and VR_OUT are invalidated.

[0049] In this way, the synchronizer 201 synchronizes the control signal P_RG_EN_IN to coincide with the falling edge of the signal whose potential changes at the latest time among the control signals VG_IN and VR_IN. This makes it possible to set the rising timing of the control signal P_RG_EN_OUT so as not to overlap with the photon detection period.

[0050] In the period from time t15 to time t16, the control signal WRITE goes high, causing the selection circuit 124 to output the signal held in the storage unit 123 to the output line 52.

[0051] In the period from time t17 to time t18, the control signal MEM_RES goes to high level, causing the storage unit 123 to initialize the stored values.

[0052] Also, at time t18, the control signal VR_IN goes to high level. At time t18, the control signal P_RG_EN_OUT is at high level, and the output of the control signal VR_OUT from the control circuit 202 is disabled. Therefore, at time t18, the control signal VR_OUT remains at low level.

[0053] At time t19, the control signal VR_IN goes to low level. At this time, the control signal VR_OUT remains at low level. In this manner, the control signal VR_OUT is maintained at low level during the period from time t18 to time t19, so that the recharge operation in the pixel 100 is not performed.

[0054] At time t20, the control signal P_RG_EN_IN goes to low level. Since the control signal VG_IN is maintained at low level at time t20, the level of the output signal of the synchronizer 201 does not change at this point. Therefore, the control signal P_RG_EN_OUT remains at high level at time t20.

[0055] At time t21, the control signal VG_IN goes to high level. At time t21, the control signal P_RG_EN_OUT is at high level, and the output of the control signal VG_OUT from the control circuit 202 is disabled. Therefore, at time t21, the control signal VG_OUT remains at low level.

[0056] At time t22, the control signal VG_IN goes to low level. At this time, the control signal VR_OUT remains at low level. In this manner, the control signal VG_OUT is maintained at low level during the period from time t21 to time t22, so that the pixel 100 does not receive a signal based on the detection of a photon.

[0057] Also, at time t22, the synchronizer 201 outputs the control signal P_RG_EN_IN as the control signal P_RG_EN_OUT to the control terminal of the control circuit 202 in synchronization with the falling edge of the control signal VG_IN. As a result, the control signal P_RG_EN_OUT becomes low level, and the output of the control signals VG_OUT and VR_OUT is enabled. The operation of the period after time t22 is the same as that described above, and therefore a description thereof will be omitted.

[0058] In this manner, during the period from time t14 to time t22 when the control signal P_RG_EN_OUT is at a high level, exposure control processing related to photon detection and value retention is disabled in the pixel 100. By performing a read control processing related to outputting a signal from the pixel 100 and initializing the storage unit 123 during this period, it is possible to operate in such a way that the two processes do not interfere with each other.

[0059] In this embodiment, the synchronizer 201 synchronizes a signal generated inside the photoelectric conversion device 1 with a signal input from outside the photoelectric conversion device 1. Then, the control circuit 202 controls the signal input from outside the photoelectric conversion device 1 using the synchronized signal. This allows the control circuit 202 to control the signal in a suitable manner. Therefore, according to this embodiment, a semiconductor device capable of suitable synchronization and control of signals is provided. In addition, by using the signal generated in this manner to control the pixel 100, a photoelectric conversion device 1 is provided that can be suitably controlled even if the signal input from outside and the signal generated inside are asynchronous. In addition, by applying this photoelectric conversion device 1 to the generation of exposure and readout control signals in a time-gate ToF, a distance measuring device capable of suitable distance measurement is provided.

[0060] In addition, the control circuit 202 in this embodiment performs control to switch between enabling and disabling the output of both the control signals VG_OUT and VR_OUT based on the synchronized control signal P_RG_EN_OUT, but the control performed by the control circuit 202 is not limited to this example. For example, the control circuit 202 may perform control on either one of the control signals VG_OUT or VR_OUT. Also, for example, the control circuit 202 may perform control other than enabling and disabling the output on at least one of the control signals VG_OUT and VR_OUT based on the synchronized control signal P_RG_EN_OUT.

[0061] [Second embodiment] In this embodiment, a modified example of the configuration of the pixel 100 and the vertical scanning circuit 20 in the first embodiment will be described. In this embodiment, the description of elements common to the first embodiment may be omitted or simplified.

[0062] Fig. 6 is a block diagram showing an example of the configuration of the vertical scanning circuit 20 according to this embodiment. Fig. 6 is different from Fig. 4 in that two control signals VG1_IN (third signal) and control signal VG2_IN (fourth signal) are input to the vertical scanning circuit 20 instead of the control signal VG_IN. The external device 2 outputs control signals VG1_IN, VG2_IN, and VR_IN to the vertical scanning circuit 20 via a control signal line group 54. The control signals VG1_IN, VG2_IN, and VR_IN are synchronous with each other. The control signals VG1_IN, VG2_IN, and VR_IN are asynchronous with the control signals MEM_RES, WRITE, and P_RG_EN_IN.

[0063] The control signal VG1_IN is input to a control terminal of the synchronizer 201. The synchronizer 201 outputs the control signal P_RG_EN_IN to a control terminal of the control circuit 202 as a control signal P_RG_EN_OUT in synchronization with the falling edge of the control signal VG1_IN.

[0064] The control signals VG1_IN, VG2_IN, and VR_IN are input to an input terminal of the control circuit 202. The control circuit 202 controls whether or not to output the control signals VG1_IN, VG2_IN, and VR_IN to the pixels 100 as control signals VG1_OUT, VG2_OUT, and VR_OUT based on the signal level of the control signal P_RG_EN_OUT. That is, the control signal P_RG_EN_OUT is a signal that switches between enabling and disabling the external output of the control signals VG1_OUT, VG2_OUT, and VR_OUT from the vertical scanning circuit 20.

[0065] The vertical scanning circuit 20 receives control signals MEM_RES, WRITE, and P_RG_EN_IN generated inside the photoelectric conversion device 1, and control signals VG1_IN, VG2_IN, and VR_IN input from outside the photoelectric conversion device 1. The vertical scanning circuit 20 outputs control signals MEM_RES, WRITE, VG1_OUT, VG2_OUT, and VR_OUT to the pixels 100 based on these control signals.

[0066] FIG. 7 is a block diagram showing an example of the configuration of the pixel 100 according to the present embodiment. In FIG. 7, the difference from FIG. 2 is that two storage units 123A and 123B are arranged instead of the storage unit 123. The pulse signal output from the waveform shaping unit 122 is input to each of the storage units 123A and 123B. The storage unit 123A is controlled by control signals VG1_OUT and MEM_RES input to a control terminal via two control signal lines of the control signal line group 51. The storage unit 123B is controlled by control signals VG2_OUT and MEM_RES input to a control terminal via two control signal lines of the control signal line group 51. The storage units 123A and 123B can be switched between an input acceptance state in which the input of a pulse signal can be accepted and a holding state in which information of the input pulse signal is held, according to the control signals VG1_OUT and VG2_OUT, respectively. That is, the control signal VG1_OUT indicates the timing at which a signal is held in the storage unit 123A, and the control signal VG2_OUT indicates the timing at which a signal is held in the storage unit 123B.

[0067] 8 is a timing chart showing the operation of the pixel 100 according to the present embodiment. In FIG. 8, the first difference from FIG. 5 is that the control signal VG2_IN is at a high level during the period from time t11 to time t13, and the control signal VG1_IN is at a high level during the period from time t13 to time t14. In these periods, the control signal P_RG_EN_OUT is at a low level. Therefore, the control signal VG2_OUT is at a high level during the period from time t11 to time t13, and the control signal VG1_OUT is at a high level during the period from time t11 to time t14. In this configuration, the synchronizer 201 outputs the control signal P_RG_EN_IN to the control terminal of the control circuit 202 as the control signal P_RG_EN_OUT in synchronization with the falling edge of the control signal VG1_IN at time t14. That is, the synchronizer 201 synchronizes the control signal P_RG_EN_IN so as to coincide with the falling edge of the signal whose potential changes at the latest time among the control signals VG1_IN, VG2_IN, and VR_IN.

[0068] 8, the second difference from FIG. 5 is that the control signal VG2_IN is at a high level during the period from time t19 to time t21, and the control signal VG1_IN is at a high level during the period from time t21 to time t22. During these periods, the control signal P_RG_EN_OUT is at a high level. Therefore, the control signals VG1_OUT and VG2_OUT are maintained at a low level during the period from time t19 to time t22.

[0069] The period from time t11 to time t14 is a first photon detection period by the storage unit 123A. Moreover, the period from time t11 to time t13 is a second photon detection period by the storage unit 123B. In this manner, in this embodiment, the two storage units 123A and 123B can detect photons in different photon detection periods and store signals.

[0070] In this embodiment, a semiconductor device, a photoelectric conversion device 1, and a distance measuring device are provided that can obtain the same effects as those of the first embodiment. In addition, in this embodiment, the pixel 100 can detect photons in a plurality of photon detection periods that are different from each other. This configuration is more suitable for measuring, for example, a time change in the amount of light emitted from a subject.

[0071] In this embodiment, the number of storage units in one pixel and the number of types of photon detection periods are both two, but are not limited to this. That is, the number of storage units in one pixel and the number of types of photon detection periods may be three or more. Also, while Fig. 8 shows an example in which the first photon detection period and the second photon detection period partially overlap, these periods do not have to overlap.

[0072] [Third embodiment] In this embodiment, a modified example of the configuration of the pixel 100 and the vertical scanning circuit 20 in the second embodiment will be described. In this embodiment, the description of elements common to the first and second embodiments may be omitted or simplified.

[0073] FIG. 9 is a block diagram showing an example of the configuration of the vertical scanning circuit 20 according to this embodiment. In FIG. 9, the difference from FIG. 6 is that the control circuit 202 of the vertical scanning circuit 20 is changed to a control unit composed of two control circuits 202A and 202B. The control signal P_RG_EN_OUT output from the synchronizer 201 is input to a control terminal of the control circuit 202A. The control signal VR_IN is input to an input terminal of the control circuit 202A. The control circuit 202A controls whether or not to output the control signal VR_IN as the control signal VR_OUT based on the signal level of the control signal P_RG_EN_OUT. That is, the control signal P_RG_EN_OUT is a signal that switches between enabling and disabling the output of the control signal VR_OUT.

[0074] The control signal VR_OUT output from the control circuit 202A is input to a first control terminal of the control circuit 202B. The control signal VR_OUT is also output to the pixel 100. The control signal P_RG_EN_OUT output from the synchronizer 201 is also input to a second control terminal of the control circuit 202B. The control signals VG1_IN and VG2_IN are input to input terminals of the control circuit 202B. The control circuit 202B controls whether or not to output the control signals VG1_IN and VG2_IN to the pixel 100 as control signals VG1_OUT and VG2_OUT based on the signal levels of the control signals P_RG_EN_OUT and VR_OUT. That is, each of the control signals P_RG_EN_OUT and VR_OUT is a signal that switches between enabling and disabling the external output of the control signals VG1_OUT and VG2_OUT from the vertical scanning circuit 20.

[0075] In this way, the vertical scanning circuit 20 outputs the control signals MEM_RES, WRITE, VG1_OUT, VG2_OUT, and VR_OUT to the pixel 100. The circuit configuration of the pixel 100 is similar to that in FIG.

[0076] FIG. 10 is a timing chart showing the operation of the pixel 100 according to the present embodiment. In FIG. 10, the first difference from FIG. 8 is that the control signal VG2_IN is at a high level during a period from time t10 to time t13, and the control signal VG1_IN is at a high level during a period from time t10 to time t14. That is, a part of the period during which the control signals VG1_IN and VG2_IN are at a high level overlaps with the period during which the control signal VR_IN is at a high level. The control circuit 202B performs control so that the control signals VG1_IN and VG2_IN are output to the pixel 100 as the control signals VG1_OUT and VG2_OUT from time t11 when the control signal VR_OUT is changed from a high level to a low level. In other words, the control circuit 202B performs control so as to disable the control signals VG1_IN and VG2_IN so as not to output them to the pixel 100 as the control signals VG1_OUT and VG2_OUT during the period during which the control signal VR_OUT is at a high level. Moreover, the synchronizer 201 synchronizes the control signal P_RG_EN_IN so as to coincide with the falling edge of the signal whose potential falls at the latest time among the control signals VG1_IN, VG2_IN, and VR_IN.

[0077] The second difference between Fig. 10 and Fig. 8 is that the control signal VG2_IN is at a high level from time t18 to time t21, and the control signal VG1_IN is at a high level from time t18 to time t22. During these periods, the control signal P_RG_EN_OUT is at a high level. Therefore, the control signals VG1_OUT and VG2_OUT are maintained at a low level from time t18 to time t22.

[0078] In this embodiment, a semiconductor device, a photoelectric conversion device 1, and a distance measuring device are provided that can obtain the same effects as those of the first and second embodiments. In addition, in this embodiment, a control signal VR_OUT based on a control signal VR_IN, which is one of the mutually synchronous control signals VG1_IN, VG2_IN, and VR_IN, is used to control the invalidation of the other control signals VG1_IN and VG2_IN. This can reduce the possibility of erroneously detecting photons that have entered the APD 111 before the recharge operation by the control signal VR_OUT.

[0079] In this embodiment, the control circuit 202B controls the control signals VG1_OUT and VG2_OUT so that they rise in synchronization with the falling edge of the control signal VR_OUT. However, the timing at which the control signals VG1_OUT and VG2_OUT rise is not limited to this. The effect of this embodiment can be obtained as long as the timing at which the control signals VG1_OUT and VG2_OUT rise is after the rising edge of the control signal VR_OUT, that is, after the start of the recharge operation.

[0080] In this embodiment, the number of storage units in one pixel is two as in the second embodiment, but is not limited to this. For example, the method of this embodiment can be applied to a case where the number of storage units in one pixel is one as in the first embodiment. In that case, a similar effect can be obtained by using a control signal VR_OUT based on a control signal VR_IN, which is one of the control signals VG_IN and VR_IN that are synchronized with each other, to control the invalidation of the other control signal VG_IN.

[0081] [Fourth embodiment] A light detection system according to a fourth embodiment of the present invention will be described with reference to Fig. 11. Fig. 11 is a block diagram of the light detection system according to this embodiment. The light detection system according to this embodiment is an imaging system that acquires an image based on incident light.

[0082] The photoelectric conversion device in the above-described embodiment can be applied to various imaging systems. Examples of imaging systems include digital still cameras, digital camcorders, camera heads, copiers, fax machines, mobile phones, vehicle-mounted cameras, observation satellites, and surveillance cameras. Fig. 11 shows a block diagram of a digital still camera as an example of an imaging system.

[0083] The imaging system 7 shown in Fig. 11 includes a barrier 706, a lens 702, an aperture 704, an imaging device 70, a signal processing unit 708, a timing generating unit 720, an overall control and calculation unit 718, a memory unit 710, a recording medium control I / F unit 716, a recording medium 714, and an external I / F unit 712. The barrier 706 protects the lens, and the lens 702 forms an optical image of a subject on the imaging device 70. The aperture 704 varies the amount of light that passes through the lens 702. The imaging device 70 is configured like the photoelectric conversion device of the above-mentioned embodiment, and converts the optical image formed by the lens 702 into image data. The signal processing unit 708 performs various types of correction, data compression, and other processes on the imaging data output from the imaging device 70.

[0084] The timing generating unit 720 outputs various timing signals to the imaging device 70 and the signal processing unit 708. The overall control / calculation unit 718 controls the entire digital still camera, and the memory unit 710 temporarily stores image data. The recording medium control I / F unit 716 is an interface for recording or reading image data to the recording medium 714, and the recording medium 714 is a removable recording medium such as a semiconductor memory for recording or reading image data. The external I / F unit 712 is an interface for communicating with an external computer or the like. Timing signals and the like may be input from outside the imaging system 7, and the imaging system 7 only needs to have at least the imaging device 70 and a signal processing unit 708 for processing image signals output from the imaging device 70.

[0085] In this embodiment, the imaging device 70 and the signal processing unit 708 may be disposed on the same semiconductor substrate, or the imaging device 70 and the signal processing unit 708 may be disposed on separate semiconductor substrates.

[0086] Furthermore, each pixel of the imaging device 70 may include a first photoelectric conversion unit and a second photoelectric conversion unit. The signal processing unit 708 processes a pixel signal based on the charge generated in the first photoelectric conversion unit and a pixel signal based on the charge generated in the second photoelectric conversion unit, and can acquire distance information from the imaging device 70 to the subject.

[0087] [Fifth embodiment] Fig. 12 is a block diagram of a light detection system according to this embodiment, more specifically, a block diagram of a range image sensor using the photoelectric conversion device described in the above embodiment.

[0088] 12, the range image sensor 401 includes an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The range image sensor 401 receives light (modulated light, pulsed light) emitted from a light source device 411 toward a subject and reflected from the surface of the subject. The range image sensor 401 can obtain a range image according to the distance to the subject, based on the time from emission to reception of light.

[0089] The optical system 402 includes one or more lenses, guides image light (incident light) from a subject to the photoelectric conversion device 403 , and forms an image on the light receiving surface (sensor portion) of the photoelectric conversion device 403 .

[0090] The photoelectric conversion device according to each of the above-mentioned embodiments can be applied as the photoelectric conversion device 403. The photoelectric conversion device 403 supplies the image processing circuit 404 with a distance signal indicating a distance determined from a received light signal.

[0091] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 403. The distance image (image data) obtained by the image processing can be displayed on a monitor 405 and stored (recorded) in a memory 406.

[0092] The range image sensor 401 configured in this manner can obtain an accurate range image by applying the above-mentioned photoelectric conversion device.

[0093] [Sixth embodiment] The technology according to the present disclosure may be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system, which is an example of an optical detection system.

[0094] Fig. 13 is a schematic diagram of the endoscopic surgery system in this embodiment. Fig. 13 shows a state in which an operator (doctor) 1131 performs surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1103. As shown in the figure, the endoscopic surgery system 1103 includes an endoscope 1100, a surgical tool 1110, an arm 1121, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0095] The endoscope 1100 includes a lens barrel 1101, the region of which from the tip is a predetermined length that is inserted into a body cavity of a patient 1132, and a camera head 1102 that is connected to the base end of the lens barrel 1101. Although Fig. 13 shows the endoscope 1100 configured as a so-called rigid scope having a rigid lens barrel 1101, the endoscope 1100 may also be configured as a so-called flexible scope having a flexible lens barrel.

[0096] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100. Light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target in the body cavity of the patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0097] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is focused on the photoelectric conversion device by the optical system. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. As the photoelectric conversion device, the photoelectric conversion device described in each of the above-mentioned embodiments may be used. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.

[0098] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0099] Under the control of the CCU 1135 , the display device 1136 displays an image based on an image signal that has been subjected to image processing by the CCU 1135 .

[0100] The light source device 1203 includes a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when imaging an operation site or the like.

[0101] The input device 1137 is an input interface for the endoscopic surgery system 1103. A user can input various information and instructions to the endoscopic surgery system 1103 via the input device 1137.

[0102] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0103] The light source device 1203 can supply irradiation light to the endoscope 1100 when imaging the surgical site, and can be, for example, a white light source made of an LED, a laser light source, or a combination of these. When a white light source is configured by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision. Therefore, the light source device 1203 can adjust the white balance of the captured image. In this case, the laser light from each of the RGB laser light sources can be irradiated to the observation target in a time-division manner, and the drive of the image sensor of the camera head 1102 can be controlled in synchronization with the irradiation timing. This makes it possible to capture images corresponding to each of the RGB colors in a time-division manner. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0104] Furthermore, the driving of the light source device 1203 may be controlled so that the intensity of the light output from the light source device 1203 is changed at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.

[0105] Furthermore, the light source device 1203 may be configured to be capable of supplying light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue can be utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface can be photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) during normal observation. Alternatively, in the special light observation, a fluorescent observation may be performed in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to be capable of supplying narrow band light and / or excitation light corresponding to such special light observation.

[0106] [Seventh embodiment] The light detection system and the moving body of this embodiment will be described with reference to Figures 14, 15(a), 15(b), 15(c), and 16. In this embodiment, an example of an in-vehicle camera will be shown as the light detection system.

[0107] FIG. 14 is a schematic diagram of a light detection system according to this embodiment, showing an example of a vehicle system and a light detection system mounted on the vehicle system. The light detection system 1301 includes a photoelectric conversion device 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of a subject on the photoelectric conversion device 1302. The photoelectric conversion device 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric conversion device 1302 is any of the photoelectric conversion devices according to the above-mentioned embodiments. The image preprocessing unit 1315 performs predetermined signal processing on the signal output from the photoelectric conversion device 1302. The function of the image preprocessing unit 1315 may be incorporated in the photoelectric conversion device 1302. The light detection system 1301 is provided with at least two sets of an optical system 1314 , a photoelectric conversion device 1302 , and an image pre-processing unit 1315 , and the output from each set of the image pre-processing unit 1315 is input to the integrated circuit 1303 .

[0108] The integrated circuit 1303 is an integrated circuit for use in an imaging system, and includes an image processing unit 1304 including a storage medium 1305, an optical distance measuring unit 1306, a parallax calculation unit 1307, an object recognition unit 1308, and an abnormality detection unit 1309. The image processing unit 1304 performs image processing such as development processing and defect correction on the output signal of the image pre-processing unit 1315. The storage medium 1305 performs primary storage of the captured image and stores the defective positions of the captured pixels. The optical distance measuring unit 1306 performs focusing or distance measurement of the subject. The parallax calculation unit 1307 calculates distance measurement information from multiple image data acquired by multiple photoelectric conversion devices 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 1309 detects an abnormality in the photoelectric conversion device 1302, it notifies the main control unit 1313 of the abnormality.

[0109] The integrated circuit 1303 may be realized by dedicated hardware, a software module, or a combination of these. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or a combination of these.

[0110] The main control unit 1313 supervises and controls the operations of the light detection system 1301, the vehicle sensor 1310, the control unit 1320, etc. The main control unit 1313 may not be provided, and the light detection system 1301, the vehicle sensor 1310, and the control unit 1320 may each have a communication interface and transmit and receive control signals via a communication network according to, for example, the CAN standard.

[0111] The integrated circuit 1303 has a function of receiving a control signal from a main control unit 1313 or transmitting a control signal or a set value to the photoelectric conversion device 1302 by its own control unit.

[0112] The optical detection system 1301 is connected to a vehicle sensor 1310, and can detect the vehicle's driving state such as vehicle speed, yaw rate, steering angle, etc., the environment outside the vehicle, and the state of other vehicles and obstacles. The vehicle sensor 1310 is also a distance information acquisition unit that acquires distance information to an object. The optical detection system 1301 is also connected to a driving assistance control unit 1311 (mobile body control unit) that performs various driving assistance functions such as automatic steering, automatic cruising, and collision prevention functions. In particular, regarding the collision determination function, it determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the optical detection system 1301 and the vehicle sensor 1310. As a result, avoidance control is performed when a collision is estimated, and a safety device is activated when a collision occurs.

[0113] The light detection system 1301 is also connected to an alarm device 1312 that issues an alarm to the driver based on the result of the determination by the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 1313 performs vehicle control such as applying the brakes, releasing the accelerator, and suppressing engine output to avoid a collision or reduce damage. The alarm device 1312 issues an alarm to the user using means such as issuing an alarm such as a sound, displaying alarm information on the display screen of a car navigation system or a meter panel, or applying vibration to a seat belt or steering wheel.

[0114] The light detection system 1301 in this embodiment can capture images of the surroundings of the vehicle, for example, the front or rear. Figures 15(a), 15(b), and 15(c) are schematic diagrams of a moving body in this embodiment, showing a configuration in which the light detection system 1301 captures an image of the area in front of the vehicle.

[0115] The two photoelectric conversion devices 1302 are disposed in front of the vehicle 1300. Specifically, it is preferable to regard the center line of the vehicle 1300 relative to the forward / backward direction or the outer shape (e.g., vehicle width) as an axis of symmetry, and to dispose the two photoelectric conversion devices 1302 in line symmetry with respect to the axis of symmetry. This makes it possible to effectively obtain distance information between the vehicle 1300 and an object to be photographed, and to determine the possibility of a collision. It is also preferable that the photoelectric conversion device 1302 is disposed in a position that does not obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 1300 from the driver's seat. It is preferable that the alarm device 1312 is disposed in a position that is easily within the driver's field of vision.

[0116] Next, a fault detection operation of the photoelectric conversion device 1302 in the photodetection system 1301 will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the operation of the photodetection system in this embodiment. The fault detection operation of the photoelectric conversion device 1302 can be performed in accordance with steps S1410 to S1480 shown in Fig. 16.

[0117] In step S1410, startup settings are made for the photoelectric conversion device 1302. That is, setting information for the operation of the photoelectric conversion device 1302 is transmitted from outside the photodetection system 1301 (e.g., the main control unit 1313) or from inside the photodetection system 1301, and the photoelectric conversion device 1302 starts imaging operation and fault detection operation.

[0118] Next, in step S1420, the photoelectric conversion device 1302 acquires pixel signals from the valid pixels. In addition, in step S1430, the photoelectric conversion device 1302 acquires output values ​​from failure detection pixels provided for failure detection. The failure detection pixels include photoelectric conversion elements, like the valid pixels. A predetermined voltage is written to the photoelectric conversion elements. The failure detection pixels output signals corresponding to the voltage written to the photoelectric conversion elements. Note that steps S1420 and S1430 may be executed in the reverse order.

[0119] Next, in step S1440, the photodetection system 1301 performs a correspondence determination between the output expected value of the fault detection pixel and the actual output value from the fault detection pixel. If the correspondence determination result in step S1440 indicates that the output expected value and the actual output value match, the photodetection system 1301 proceeds to processing in step S1450, determines that the imaging operation is performed normally, and proceeds to processing in step S1460. In step S1460, the photodetection system 1301 transmits the pixel signal of the scanning row to the storage medium 1305 and temporarily stores it. Thereafter, the photodetection system 1301 returns to the processing in step S1420 and continues the fault detection operation. On the other hand, if the correspondence determination result in step S1440 indicates that the output expected value and the actual output value do not match, the photodetection system 1301 proceeds to processing in step S1470. In step S1470, the light detection system 1301 determines that there is an abnormality in the imaging operation, and issues an alarm to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays on the display unit that an abnormality has been detected. After that, in step S1480, the light detection system 1301 stops the photoelectric conversion device 1302, and ends the operation of the light detection system 1301.

[0120] In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for each set of lines, or the fault detection operation may be performed for each frame. The issuance of the alarm in step S1470 may be notified to the outside of the vehicle via a wireless network.

[0121] In addition, in this embodiment, the control to prevent collision with other vehicles has been described, but the control can also be applied to automatic driving control to follow other vehicles, automatic driving control not to go out of the lane, etc. Furthermore, the light detection system 1301 is not limited to vehicles such as the vehicle itself, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the light detection system 1301 is not limited to moving bodies, but can be applied to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).

[0122] The photoelectric conversion device of the present invention may further be configured to be capable of acquiring various types of information such as distance information.

[0123] [Eighth embodiment] FIG. 17(a) is a diagram showing a specific example of an electronic device in this embodiment, showing glasses 1600 (smart glasses). The glasses 1600 are provided with a photoelectric conversion device 1602 described in each of the above-mentioned embodiments. That is, the glasses 1600 are an example of a light detection system to which the photoelectric conversion device 1602 described in each of the above-mentioned embodiments can be applied. A display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. In addition, multiple types of photoelectric conversion devices may be combined. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG. 17(a).

[0124] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the above-mentioned display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is provided with an optical system for collecting light on the photoelectric conversion device 1602.

[0125] FIG. 17(b) shows glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. The photoelectric conversion device in the control device 1612 and an optical system for projecting light emitted from the display device are arranged in the lens 1611, and an image is projected onto the lens 1611. The control device 1612 functions as a power source that supplies power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device 1612 may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used to detect the line of sight. The infrared light emission unit emits infrared light toward the eyeball of a user gazing at a display image. An image of the eyeball is obtained by detecting the reflected light of the emitted infrared light from the eyeball with an imaging unit having a light receiving element. By providing a reduction unit that reduces the amount of light from the infrared light emitting unit to the display unit in a plan view, degradation of image quality is reduced.

[0126] The control device 1612 detects the user's line of sight with respect to the displayed image from the captured image of the eyeball obtained by capturing infrared light. Any known method can be applied to the line of sight detection using the captured image of the eyeball. As an example, a line of sight detection method based on the Purkinje image formed by the reflection of irradiated light on the cornea can be used.

[0127] More specifically, a gaze detection process based on the pupil-corneal reflex method is performed. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0128] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information on the user's line of sight from the photoelectric conversion device.

[0129] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0130] The display area may include a first display area and a second display area different from the first display area. A high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0131] In addition, AI (Artificial Intelligence) may be used in determining the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using an image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI ​​program may be provided in either the display device or the photoelectric conversion device, or may be provided in an external device. If the external device has the AI ​​program, it may be transmitted to the display device from a server or the like via communication.

[0132] In the case where display control is performed based on visual recognition detection, the present embodiment can be preferably applied to smart glasses further including a photoelectric conversion device that captures an image of the outside world. The smart glasses can display captured outside information in real time.

[0133] [Modified embodiment] The present invention is not limited to the above-described embodiments and can be modified in various ways. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with a part of the configuration of another embodiment, is also an embodiment of the present invention.

[0134] The disclosure of this specification includes the complement of the concepts described in this specification. In other words, if the specification states, for example, that "A is B" (A=B), the specification is deemed to disclose or suggest that "A is not B" even if the statement that "A is not B" (A≠B) is omitted. This is because when it states that "A is B," it is assumed that the case that "A is not B" is taken into consideration.

[0135] The disclosure of this specification includes the following configurations. (Configuration 1) a signal generating unit that generates a first signal; a synchronization unit that synchronizes the first signal with either a second signal or a third signal that are input from an external source and are synchronized with each other; a control unit that controls at least one of the second signal and the third signal based on the first signal after synchronization by the synchronization unit; A semiconductor device comprising: (Configuration 2) The control unit controls both the second signal and the third signal based on the first signal after synchronization by the synchronization unit. 2. The semiconductor device according to configuration 1. (Configuration 3) The control unit performs control to switch between enabling and disabling an external output of at least one of the second signal and the third signal. 3. The semiconductor device according to configuration 1 or 2. (Configuration 4) The synchronization unit performs the synchronization by matching a timing at which a potential of the first signal changes with a timing at which a potential of either the second signal or the third signal changes. 4. The semiconductor device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes. (Configuration 5) The synchronization unit performs the synchronization by matching a timing at which the potential of the first signal changes with a timing at which the potential of the second signal changes and a timing at which the potential of the third signal changes, whichever is later. 5. The semiconductor device according to configuration 4. (Configuration 6) The control unit further controls the other of the second signal and the third signal based on one of the second signal and the third signal that has been controlled based on the first signal. 6. The semiconductor device according to claim 1, wherein the first and second electrodes are electrically connected to each other. (Configuration 7) The control unit performs control for switching between enabling and disabling an external output of the other of the second signal and the third signal, based on one of the second signal and the third signal that has been controlled based on the first signal. 7. The semiconductor device according to claim 1, wherein the first and second electrodes are electrically connected to each other. (Configuration 8) The synchronization unit includes a flip-flop circuit to which the first signal and any one of the second signal and the third signal are input. 8. The semiconductor device according to claim 1, wherein the first and second electrodes are electrically connected to each other. (Configuration 9) A semiconductor device according to any one of configurations 1 to 8, a pixel including a photoelectric conversion element and controlled by a signal output from the semiconductor device; A photoelectric conversion device comprising: (Configuration 10) The photoelectric conversion element is an avalanche photodiode. 10. The photoelectric conversion device according to configuration 9. (Configuration 11) The control unit controls whether or not at least one of the second signal and the third signal is supplied to the pixel. 11. The photoelectric conversion device according to configuration 9 or 10. (Configuration 12) The second signal is a signal indicating a timing for initializing the potential of the photoelectric conversion element. 12. The photoelectric conversion device according to any one of configurations 9 to 11. (Configuration 13) The third signal is a signal indicating a timing for holding a signal in the pixel. 13. The photoelectric conversion device according to any one of configurations 9 to 12. (Configuration 14) a fourth signal synchronized with the second signal and the third signal is further input to the synchronization unit, The synchronization unit synchronizes the first signal with any one of the second signal, the third signal, and the fourth signal. 14. The photoelectric conversion device according to any one of configurations 9 to 13. (Configuration 15) the third signal and the fourth signal are both signals indicating a timing for holding a signal in the pixel, The timing of signal holding indicated by the third signal and the timing of signal holding indicated by the fourth signal are different from each other. 15. The photoelectric conversion device according to configuration 14. (Configuration 16) the signal generating unit further generates a fifth signal that is synchronized with the first signal; The fifth signal is a signal indicating a timing of a signal output from the pixel. 16. The photoelectric conversion device according to any one of configurations 9 to 15. (Configuration 17) the signal generating unit further generates a sixth signal synchronized with the first signal; The sixth signal is a signal indicating a timing for initializing a signal that is held in the pixel and is based on incident light to the photoelectric conversion element. 17. The photoelectric conversion device according to any one of configurations 9 to 16, (Configuration 18) The light emitted from a light source, the light emission timing of which is synchronized with the second signal and the third signal, is received by the pixel to obtain distance information to the object. 18. The photoelectric conversion device according to any one of configurations 9 to 17. (Configuration 19) The photoelectric conversion device according to any one of configurations 1 to 18, a signal processing unit that processes a signal output from the photoelectric conversion device; 1. An optical detection system comprising: (Configuration 20) A mobile object, The photoelectric conversion device according to any one of configurations 1 to 18, a moving object control unit that controls the moving object based on distance information generated from a signal output from the photoelectric conversion device; A moving object comprising:

[0136] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0137] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0138] 1 Photoelectric conversion device 2 External device 20 Vertical scanning circuit 30 Digital Front-End Circuit (DFE) 100 pixels 201 Synchronizer 202 Control circuit

Claims

1. A signal generating unit that generates a first signal; a synchronization unit that synchronizes the first signal with either a second signal or a third signal that are input from an external source and are synchronized with each other; a control unit that controls at least one of the second signal and the third signal based on the first signal after synchronization by the synchronization unit; A semiconductor device comprising:

2. The control unit controls both the second signal and the third signal based on the first signal after synchronization by the synchronization unit.

2. The semiconductor device according to claim 1,

3. The control unit performs control to switch between enabling and disabling an external output of at least one of the second signal and the third signal.

2. The semiconductor device according to claim 1,

4. The synchronization unit performs the synchronization by matching a timing at which a potential of the first signal changes with a timing at which a potential of either the second signal or the third signal changes.

2. The semiconductor device according to claim 1,

5. The synchronization unit performs the synchronization by matching a timing at which the potential of the first signal changes with a timing at which the potential of the second signal changes and a timing at which the potential of the third signal changes, whichever is later.

5. The semiconductor device according to claim 4.

6. The control unit further controls the other of the second signal and the third signal based on one of the second signal and the third signal that has been controlled based on the first signal.

2. The semiconductor device according to claim 1,

7. The control unit performs control for switching between enabling and disabling an external output of the other of the second signal and the third signal, based on one of the second signal and the third signal that has been controlled based on the first signal.

2. The semiconductor device according to claim 1,

8. The synchronization unit includes a flip-flop circuit to which the first signal and any one of the second signal and the third signal are input.

2. The semiconductor device according to claim 1,

9. A semiconductor device according to claim 1; a pixel including a photoelectric conversion element and controlled by a signal output from the semiconductor device; A photoelectric conversion device comprising:

10. The photoelectric conversion element is an avalanche photodiode.

10. The photoelectric conversion device according to claim 9.

11. The control unit controls whether or not at least one of the second signal and the third signal is supplied to the pixel.

10. The photoelectric conversion device according to claim 9.

12. The second signal is a signal indicating a timing for initializing the potential of the photoelectric conversion element.

10. The photoelectric conversion device according to claim 9.

13. The third signal is a signal indicating a timing for holding a signal in the pixel.

10. The photoelectric conversion device according to claim 9.

14. a fourth signal synchronized with the second signal and the third signal is further input to the synchronization unit, The synchronization unit synchronizes the first signal with any one of the second signal, the third signal, and the fourth signal.

10. The photoelectric conversion device according to claim 9.

15. the third signal and the fourth signal are both signals indicating a timing for holding a signal in the pixel, The timing of signal holding indicated by the third signal and the timing of signal holding indicated by the fourth signal are different from each other.

15. The photoelectric conversion device according to claim 14.

16. the signal generating unit further generates a fifth signal that is synchronized with the first signal; The fifth signal is a signal indicating a timing of a signal output from the pixel.

10. The photoelectric conversion device according to claim 9.

17. the signal generating unit further generates a sixth signal synchronized with the first signal; The sixth signal is a signal indicating a timing for initializing a signal based on incident light to the photoelectric conversion element, the signal being held in the pixel.

10. The photoelectric conversion device according to claim 9.

18. The light emitted from a light source, the light emission timing of which is synchronized with the second signal and the third signal, is received by the pixel to acquire distance information to the object.

10. The photoelectric conversion device according to claim 9.

19. The photoelectric conversion device according to any one of claims 1 to 18, a signal processing unit that processes a signal output from the photoelectric conversion device; 1. An optical detection system comprising:

20. A mobile object, The photoelectric conversion device according to any one of claims 1 to 18, a moving object control unit that controls the moving object based on distance information generated from a signal output from the photoelectric conversion device; A moving object comprising:

Images