Signal generation device and signal generation system

Abstract

The invention relates to a signal generating apparatus and a signal generating system. The apparatus includes a plurality of photoelectric conversion elements, the apparatus generates a plurality of subframes using signals from the plurality of photoelectric conversion elements, and generates a distance measurement frame using signals of the plurality of subframes, in which the plurality of subframes include at least a first subframe in which the plurality of subframes are separated from each other and a second subframe in which the plurality of subframes are separated from each other, and the distance measurement frame is generated using signals of the plurality of subframes. In a first subframe, the exposure period starts after a first period has elapsed after light emission from the light-emitting element, and in a second subframe, the exposure period starts after a second period longer than the first period has elapsed after light emission from the light-emitting element, where the first subframe includes at least the first exposure period and the second exposure period. Wherein a start timing of the first exposure period and a start timing of the second exposure period are different.

Description

Technical Field

[0001] Aspects of the embodiments relate to devices and systems. Background Art

[0002] The specification of U.S. Patent Application Publication No. 2017 / 0052065 discusses a distance measuring device that measures the distance to an object by emitting light from a light source and using a light receiving element to receive light including light reflected from the object. In the distance measuring device discussed in the specification of U.S. Patent Application Publication No. 2017 / 0052065, a single photon avalanche diode (SPAD) element is used as the light receiving element, and the SPAD element obtains a signal by multiplying electrons generated by photoelectric conversion. The specification of U.S. Patent Application Publication No. 2017 / 0052065 discusses a distance measuring method in which measurements are repeatedly performed while changing the start timing of the exposure period (gating interval) of each sub-frame for photon detection in the SPAD element.

[0003] In the distance measuring method discussed in the specification of U.S. Patent Application Publication No. 2017 / 0052065, there is a trade-off between distance resolution and reduction of distance measurement time. More specifically, if the exposure period in a sub-frame is increased, although the distance measurement time is shortened, the distance resolution deteriorates. Conversely, if the exposure period in a sub-frame is decreased, although the distance resolution is improved, the distance measurement time increases. However, in order to improve distance measurement performance, it may be required to ensure an appropriate distance resolution without increasing the distance measurement time. Summary of the Invention

[0004] According to an aspect of an embodiment, a signal generation device includes: a plurality of photoelectric conversion elements, wherein the signal generation device generates a plurality of sub-frames using signals from the plurality of photoelectric conversion elements, and generates a distance measurement frame using signals of the plurality of sub-frames, wherein the plurality of sub-frames at least include a first sub-frame and a second sub-frame, wherein in the first sub-frame, the exposure period starts after a first period has elapsed since light emission from a light emitting element, and in the second sub-frame, the exposure period starts after a second period longer than the first period has elapsed since light emission from the light emitting element, wherein the first sub-frame at least includes a first exposure period and a second exposure period, and wherein the start timing of the first exposure period and the start timing of the second exposure period are different.

[0005] Other features of the present disclosure will become apparent from the following description of exemplary embodiments with reference to the drawings. Brief Description of the Drawings

[0006] Figure 1It is a block diagram showing a schematic structure of a distance information generation device according to a typical embodiment.

[0007] Figure 2 It is a schematic diagram showing a photoelectric conversion device included in a signal generation device according to a typical embodiment.

[0008] Figure 3 It is a diagram showing a configuration in a sensor substrate of a photoelectric conversion device according to a typical embodiment.

[0009] Figure 4 It is a diagram showing a structure of a circuit substrate of a photoelectric conversion device according to a typical embodiment.

[0010] Figure 5 It shows a structural example of a pixel circuit of a photoelectric conversion device according to a typical embodiment.

[0011] Figure 6 (A) of is a schematic diagram showing driving of a pixel circuit of a photoelectric conversion device according to a typical embodiment.

[0012] Figure 6 (B) of is a schematic diagram showing driving of a pixel circuit of a photoelectric conversion device according to a typical embodiment.

[0013] Figure 6 (C) of is a schematic diagram showing driving of a pixel circuit of a photoelectric conversion device according to a typical embodiment.

[0014] Figure 7 It is a driving timing diagram according to a comparative typical embodiment.

[0015] Figure 8A It shows a graph indicating a gate drive and an effective gate profile per unit time in an indication drive according to a comparative typical embodiment.

[0016] Figure 8B It shows a graph indicating a gate drive and an effective gate profile per unit time in an indication drive according to a comparative typical embodiment.

[0017] Figure 9 It is a driving timing diagram according to a first typical embodiment.

[0018] Figure 10A It shows a graph indicating a gate drive and an effective gate profile per unit time in an indication drive according to a first typical embodiment.

[0019] Figure 10B It shows a graph indicating a gate drive and an effective gate profile per unit time in an indication drive according to a first typical embodiment.

[0020] Figure 11AA graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the second exemplary embodiment.

[0021] Figure 11B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the second exemplary embodiment.

[0022] Figure 12A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to a modification of the second exemplary embodiment.

[0023] Figure 12B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to a modification of the second exemplary embodiment.

[0024] Figure 12C A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to a modification of the second exemplary embodiment.

[0025] Figure 13A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the third exemplary embodiment.

[0026] Figure 13B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the third exemplary embodiment.

[0027] Figure 14A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the fourth exemplary embodiment.

[0028] Figure 14B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the fourth exemplary embodiment.

[0029] Figure 15A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to Modification 1 of the fourth exemplary embodiment.

[0030] Figure 15B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to Modification 1 of the fourth exemplary embodiment.

[0031] Figure 16A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to Modification 2 of the fourth exemplary embodiment.

[0032] Figure 16B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to Modification 2 of the fourth exemplary embodiment.

[0033] Figure 17A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to Modification 3 of the fourth exemplary embodiment.

[0034] Figure 17B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to Modification 3 of the fourth exemplary embodiment.

[0035] Figure 18A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the fifth exemplary embodiment.

[0036] Figure 18B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the fifth exemplary embodiment.

[0037] Figure 19A A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the comparative example of the fifth exemplary embodiment.

[0038] Figure 19B A graph showing the gate drive per unit time and the effective gate profile in the indication drive according to the comparative example of the fifth exemplary embodiment.

[0039] Figure 20 A functional block diagram of a signal generation system according to the sixth exemplary embodiment.

[0040] Figure 21A A functional block diagram of a signal generation system according to the seventh exemplary embodiment.

[0041] Figure 21B A functional block diagram of a signal generation system according to the seventh exemplary embodiment.

[0042] Figure 22A A functional block diagram of a signal generation system according to the eighth exemplary embodiment.

[0043] Figure 22B A functional block diagram of a signal generation system according to the eighth exemplary embodiment.

[0044] Figure 23A A functional block diagram of a signal generation system according to the ninth exemplary embodiment.

[0045] Figure 23B A functional block diagram of a signal generation system according to the ninth exemplary embodiment. Detailed Description

[0046] The following exemplary embodiments are provided to embody the technical concept of the present disclosure and are not intended to limit the present invention. The sizes and positional relationships of the components shown in the drawings may be exaggerated for clarity of description. In the following description, the same components are assigned the same reference numerals, and the description thereof may be omitted.

[0047] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings. In the following description, terms indicating specific directions and positions (e.g., "upper", "lower", "right", "left", and other terms including these terms) are appropriately used. These terms are used to facilitate the understanding of the exemplary embodiments to be described with reference to the drawings. The technical scope of the present disclosure is not limited by the meanings of these terms.

[0048] In this specification, a "plane" refers to a surface viewed from a direction perpendicular to the light incident surface of the semiconductor layer. A cross-section refers to a surface in a direction perpendicular to the light incident surface of the semiconductor layer. In the case where the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the plane is defined based on the light incident surface of the semiconductor layer viewed macroscopically.

[0049] In the following description, the anode of the avalanche photodiode (APD) is set to a fixed potential, and a signal is taken out from the cathode side. Therefore, a semiconductor region of a first conductivity type having charges with the same polarity as the signal charge as majority carriers is an N-type semiconductor region, and a semiconductor region of a second conductivity type having charges with a different polarity from the signal charge as majority carriers is a P-type semiconductor region. Even in the case where the cathode of the APD is set to a fixed potential and a signal is taken out from the anode side, the exemplary embodiments of the present disclosure can be implemented. In this case, a semiconductor region of a first conductivity type having charges with the same polarity as the signal charge as majority carriers is a P-type semiconductor region, and a semiconductor region of a second conductivity type having charges with a different polarity from the signal charge as majority carriers is an N-type semiconductor region. The following will give a description of the case where one node of the APD is set to a fixed potential, but the potentials at both nodes can be made variable.

[0050] In the following exemplary embodiments, the connection between the elements of the circuit may be described. In this case, even when other elements are inserted between the elements to be observed, unless otherwise specified, the elements to be observed are also regarded as being connected. For example, assume that element A is connected to one node of a capacitive element C having a plurality of nodes, and element B is connected to another node. Even in this case, unless otherwise specified, element A and element B are regarded as being connected.

[0051] Figure 1It is a hardware block diagram showing an example of the schematic structure of a distance information generation device 30 according to a typical embodiment. The distance information generation device 30 includes a light emitting device 31, a signal generation device 32, and a signal processing circuit 33. The signal generation device 32 may include a photoelectric conversion device 100 and the signal processing circuit 33. Figure 1 The structure of the shown distance information generation device 30 is merely an example, and the structure of the distance information generation device 30 is not limited to Figure 1 the shown structure. For example, the first frame generation unit 37 and the second frame generation unit 38 included in the signal processing circuit 33 may be included in the photoelectric conversion device 100.

[0052] The distance information generation device 30 is a device that measures the distance to an object X as a distance measurement target using technologies such as light detection and ranging (LiDAR). The distance information generation device 30 measures the distance from the distance information generation device 30 to the object X based on the time lag from when the light is emitted from the light emitting device 31 until the light is reflected by the object X and received by the photoelectric conversion device 100. The distance information generation device 30 can also emit a laser within a predetermined distance measurement range including the object X, and measure the distance at multiple points two-dimensionally by receiving the reflected light with a pixel array. The distance information generation device 30 can thus output distance information. Alternatively, the distance information generation device 30 can output image information based on the distance information (image information including color differences and contrast differences corresponding to the distance information).

[0053] The light to be received by the signal generation device 32 contains, in addition to the light reflected from the object X, ambient light such as sunlight. Therefore, the distance information generation device 30 uses the following method to perform distance measurement with reduced ambient light influence, which is a method for measuring the amount of incident light in each of multiple time periods (sub-interval periods) and determining the period when the reflected light has entered and the light amount has reached a peak.

[0054] The light emitting device 31 is a device that emits light such as a laser to the outside of the distance information generation device 30. A vertical cavity surface emitting laser (VCSEL) that can be easily formed in a two-dimensional array can be used for the laser, for example.

[0055] The signal processing circuit 33 may include a processor for performing computational processing on digital signals and a memory for storing digital signals. As the memory, a semiconductor memory can be used, for example. The distance information generation device 30 does not necessarily include the signal processing circuit 33. In this case, at least a part of the structure included in the signal processing circuit 33 is provided in the photoelectric conversion device 100. In this case, the signal generation device 32 is the same as the photoelectric conversion device 100.

[0056] The signal generation device 32 generates a pulse signal including pulses based on incident light. In the present exemplary embodiment, the photoelectric conversion device 100 included in the signal generation device 32 generates the pulse signal. For example, a photoelectric conversion device including an APD as a photoelectric conversion element can be used. In this case, if a photon enters the APD and generates a charge, a pulse is generated by avalanche multiplication. The photoelectric conversion device 100 included in the signal generation device 32 is not limited to a photoelectric conversion device using an APD as a photoelectric conversion element, and can be a photoelectric conversion device using a different photodiode as a photoelectric conversion element.

[0057] In the present exemplary embodiment, the photoelectric conversion device 100 includes a pixel array in which a plurality of photoelectric conversion elements (pixels) are arranged in a plurality of rows and a plurality of columns. The photoelectric conversion device 100 will now be described with reference to Figures 2 to 6 (C) thereof. The structure of the photoelectric conversion device to be described below is merely an example. The photoelectric conversion device is not limited thereto, and any photoelectric conversion device can be used as long as it enables the functions of the respective exemplary embodiments to be realized as described below.

[0058] Figure 2 FIG. is a diagram showing the structure of the stacked photoelectric conversion device 100 included in the distance information generation device 30. The photoelectric conversion device 100 includes two stacked substrates corresponding to a sensor substrate 11 (first substrate) and a circuit substrate 21 (second substrate) that are electrically connected. The sensor substrate 11 includes a first semiconductor layer and a first wiring structure, and the first semiconductor layer includes a photoelectric conversion unit 102 to be described below. The circuit substrate 21 includes a second semiconductor layer and a second wiring structure, and the second semiconductor layer includes a signal detection circuit such as a signal processing unit 103 to be described below. The second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer included in the photoelectric conversion device 100 are stacked in this order. The photoelectric conversion device 100 described in each exemplary embodiment is a back-illuminated photoelectric conversion device in which light enters from the first surface side of the first semiconductor layer of the sensor substrate 11, and the circuit substrate is disposed on the second surface facing the first surface of the first semiconductor layer of the sensor substrate 11.

[0059] Hereinafter, the sensor substrate 11 and the circuit substrate 21 will be described as chips separated by dicing, but the sensor substrate 11 and the circuit substrate 21 are not limited to chips. For example, the sensor substrate 11 and the circuit substrate 21 can be wafers. Alternatively, after the sensor substrate 11 and the circuit substrate 21 are stacked in a wafer state, the sensor substrate 11 and the circuit substrate 21 can be separated by dicing, or the sensor substrate 11 and the circuit substrate 21 can be chipized in a wafer state, and then the sensor substrate 11 and the circuit substrate 21 can be bonded by stacking the chips.

[0060] A photoelectric conversion region 12 in which a plurality of photoelectric conversion elements are arranged in a two-dimensional array is disposed on a sensor substrate 11, and a circuit region 22 for processing signals detected in the photoelectric conversion region 12 is disposed on a circuit substrate 21.

[0061] Figure 3 FIG. is a diagram showing an example of the configuration in the sensor substrate 11. The photoelectric conversion elements 101 each including a photoelectric conversion unit 102 having an APD are arranged in a two-dimensional array in a plan view and form the photoelectric conversion region 12.

[0062] The photoelectric conversion element 101 is sufficient to measure the arrival time and amount of light.

[0063] For example, when the photoelectric conversion element 101 is used in a time-of-flight (TOF) sensor, it is not necessary to form an image. However, the photoelectric conversion element 101 can be a pixel for forming an image.

[0064] Figure 4 FIG. is a structural diagram of the circuit substrate 21. The circuit substrate 21 includes a signal processing unit 103, a readout circuit 112, a control pulse generation unit 115, a horizontal scan circuit unit 111, signal lines 113, a vertical scan circuit unit 110, an output circuit 114, and drive lines 116. The signal processing units 103 each process Figure 2 the charges photoelectrically converted by the photoelectric conversion unit 102 shown.

[0065] Figure 3 each of the photoelectric conversion units in the photoelectric conversion unit 102 shown and Figure 4 the corresponding signal processing units in the signal processing unit 103 shown are electrically connected via connection wirings provided for each photoelectric conversion element.

[0066] The vertical scan circuit unit 110 receives control pulses supplied from the control pulse generation unit 115 and supplies the control pulses to each photoelectric conversion element via the drive lines 116. A logic circuit such as a shift register or an address decoder serves as the vertical scan circuit unit 110.

[0067] The signals output from the photoelectric conversion units 102 of the respective photoelectric conversion elements 101 are processed by the corresponding signal processing units in the signal processing unit 103. A counter and a memory are provided in each signal processing unit 103, and digital values are stored in the memory.

[0068] The horizontal scan circuit unit 111 inputs control pulses for sequentially selecting each column to the signal processing unit 103 to read out signals from the memories for storing digital signals of the respective photoelectric conversion elements 101.

[0069] Output the signal from the signal processing unit 103 of the photoelectric conversion elements selected on the selected columns in the vertical scanning circuit unit 110 to the signal line 113.

[0070] Output the signal output to the signal line 113 to a recording unit or a signal processing unit provided outside the photoelectric conversion device 100 via the output circuit 114.

[0071] In Figure 2 , the photoelectric conversion units in the photoelectric conversion region may be arranged in a one-dimensional manner. It is not necessary to provide the function of the signal processing unit for each photoelectric conversion unit among all the photoelectric conversion units. For example, one signal processing unit can be shared by multiple photoelectric conversion units, and signal processing can be performed sequentially.

[0072] As Figure 3 and Figure 4 shown, a plurality of signal processing units 103 are arranged in a region overlapping with the photoelectric conversion region 12 in the plan view. Then, the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the readout circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged in a region between the edge of the sensor substrate 11 and the edge of the photoelectric conversion region 12 in the plan view. In other words, the sensor substrate 11 includes the photoelectric conversion region 12 and a non-photoelectric conversion region arranged around the photoelectric conversion region 12. Then, the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the readout circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged in a region overlapping with the non-photoelectric conversion region in the plan view.

[0073] Figure 5 is a block diagram showing an example of a photoelectric conversion element including the Figure 3 and Figure 4 shown equivalent circuit. Figure 5 A block diagram showing a photoelectric conversion element including a typical APD.

[0074] In Figure 5 , the photoelectric conversion unit 102 including the APD 201 is provided on the sensor substrate 11, and other components are provided on the circuit substrate 21.

[0075] The APD 201 generates a pair of charges corresponding to the incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage for causing the APD 201 to perform an avalanche multiplication operation is supplied to the anode and the cathode. By supplying such a voltage to the APD 201, the charges generated by the incident light cause avalanche multiplication, and an avalanche current is generated.

[0076] When a reverse bias voltage is supplied, the APD 201 operates in Geiger mode or linear mode. In Geiger mode, the APD 201 operates with a potential difference between the anode and the cathode that is greater than the breakdown voltage. In linear mode, the APD 201 operates with a potential difference between the anode and the cathode that is close to the breakdown voltage or equal to or less than the breakdown voltage.

[0077] An APD operating in Geiger mode will be referred to as a single photon avalanche diode (SPAD). For example, the voltage VL is -30V and the voltage VH is 1V. The APD 201 can operate in linear mode or can operate in Geiger mode.

[0078] The quenching element 202 is connected to the APD 201 and the power supply for supplying the voltage VH. The quenching element 202 serves as a load circuit (quenching circuit) when the signal is multiplied by avalanche multiplication, and has a function (quenching operation) for suppressing avalanche multiplication by reducing the voltage to be supplied to the APD 201. The quenching element 202 also has the following function (recharging operation), which is used to return the voltage to be supplied to the APD 201 to the voltage VH by causing a current corresponding in amount to the voltage drop caused by the quenching operation to flow.

[0079] In Figure 5 , the quenching element 202 is constituted by a transistor, and a potential is supplied to the gate of the quenching element 202 via the drive line VR. The potential supplied from the drive line VR is a reset signal for resetting the cathode potential at the node A of the APD 201 by switching the resistance value of the quenching element 202.

[0080] The signal processing unit 103 includes a waveform shaping unit 210, a gate circuit 321 (first selection circuit), a counter 211, and an output circuit 306 (second selection circuit). In this specification, in one embodiment, the signal processing unit 103 includes any of the waveform shaping unit 210, the gate circuit 321, the counter 211, and the output circuit 306.

[0081] The waveform shaping unit 210 outputs a pulse signal by shaping the potential change of the cathode of the APD 201 obtained at the time of photon detection. For example, an inverter circuit is used as the waveform shaping unit 210. Figure 5 An example of using one inverter as the waveform shaping unit 210 is shown, but a circuit in which a plurality of inverters are connected in series can be used as the waveform shaping unit 210, or other circuits having a waveform shaping effect can be used as the waveform shaping unit 210.

[0082] The gate circuit 321 may be constituted by, for example, an AND circuit. One input terminal of the AND circuit is connected to the waveform shaping unit 210, and the other input terminal is connected to the driving line GATE. By controlling the supply of the signal to the gate circuit 321, the exposure period described below is set. The gate circuit 321 outputs the output signal of the waveform shaping unit 210 to the counter 211 during the period when the gate signal input from the outside of the photoelectric conversion element via the driving line GATE is at the high (H) level. On the other hand, the gate circuit 321 does not output the output signal of the waveform shaping unit 210 to the counter 211 during the period when the gate signal is at the low (L) level. By inputting, for example, a pulse of nanoseconds to picoseconds during the high (H) level period as the gate signal, it is possible to selectively detect only the photon signals that have entered during the sub-frame period which is the period to be observed.

[0083] The counter 211 counts the number (times) of the pulse signals output from the waveform shaping unit 210 and stores the count value. The counter 211 measures the amount of light entering the photoelectric conversion element. According to the control signal input from the outside of the photoelectric conversion element via the driving line CTRL, the counter 211 switches whether to perform the counting operation of the counter 211 and whether to stop the counting operation. Examples of the counter 211 include multi-digit counters, but a one-digit memory or an analog memory using a capacitive element may be used. The counter 211 and the output circuit 306 are connected by wiring corresponding to the number of digits of the counter 211. When the control pulse pRES is supplied via the driving line RES, the signal stored in the counter 211 is reset.

[0084] The output circuit 306 receives the selection signal input from the outside of the photoelectric conversion element via the driving line SEL, and outputs the signal output from the counter 211 to the signal line 113. In this exemplary embodiment, via Figure 5 the driving line SEL in Figure 4 the vertical scanning circuit unit 110 supplies the control pulse pSEL, and switches the electrical connection between the counter 211 and the signal line 113 between the connected state and the disconnected state.

[0085] The output circuit 306 includes, for example, a buffer circuit for outputting a signal, and a tri-state buffer may be used, for example.

[0086] A switch such as a transistor can be provided between the quenching element 202 and the APD 201, and between the photoelectric conversion unit 102 and the signal processing unit 103 to switch the electrical connection therebetween. Alternatively, a switch such as a transistor can be used to electrically switch the supply of the voltage VH or the voltage VL to be supplied to the photoelectric conversion unit 102. Further, by switching the voltage applied to the gate of the transistor constituting the quenching element 202, the above electrical connection can be switched without providing an additional switch.

[0087] Figure 6 of (A) to Figure 6 of (C) is a diagram schematically showing the relationship between the operation of the APD and the output signal.

[0088] Figure 6 of (A) selectively shows Figure 5 the APD 201, the quenching element 202, and the waveform shaping unit 210 shown in Figure 6 In (A) of Figure 6 the input side of the waveform shaping unit 210 is regarded as node A, and the output side is regarded as node B. Figure 6 of (B) shows Figure 6 the waveform change at node A in (A) of Figure 6 and of (C) shows

[0089] During the period from time t0 to time t1, a potential difference VH - VL is applied to the APD 201 in (A) of Figure 6 If photons enter the APD 201 at time t1, avalanche multiplication occurs in the APD 201, the avalanche multiplication current flows to the quenching element 202, and the voltage at node A drops. If the amount of voltage drop further increases and the potential difference applied to the APD 201 becomes smaller, at time t2, the avalanche multiplication of the APD 201 stops. Therefore, the voltage at node A stops dropping below a certain value. Thereafter, during the period from time t2 to time t3, a current for compensating the voltage drop flows from the voltage VL to node A, and at time t3, the potential of node A stabilizes statically at the original potential level. At this time, the waveform shaping unit 210 corrects the portion of the output waveform at node A that exceeds a certain threshold and outputs it as a signal at node B.

[0090] The configuration of the signal line 113 and the configurations of the readout circuit 112 and the output circuit 114 are not limited to Figure 5 the configuration shown in

[0091] will be described with reference to Figure 9 , Figure 10A and Figure 10B a photoelectric conversion device according to a first exemplary embodiment. With reference to Figures 7 to 8B as a comparative exemplary embodiment, will be described with reference to Figure 9 , Figure 10A and Figure 10B the principle based on which an appropriate distance resolution can be ensured without increasing the distance measurement time (as an effect of a preset exemplary embodiment of the present disclosure).

[0092] Figure 7 is a driving timing chart according to a comparative exemplary embodiment, and Figure 8A and Figure 8B show graphs indicating the gate driving per unit time in the indication driving according to a comparative exemplary embodiment.

[0093] In Figure 7 , the emission light indicates the timing of emitting light from the light emitting element. The reflected light indicates the timing when the light emitted from the light emitting element is reflected by the object and detected by the photoelectric conversion device serving as the signal detection device.

[0094] Using a signal including N sub - frames (N is an integer of 2 or more) including a first sub - frame and a second sub - frame, a distance measurement frame is generated. In each sub - frame, the period at the H level is the period during which light from the light emitting element can be detected. In each sub - frame, the period at the L level is the period during which light from the light emitting element cannot be detected. In the present exemplary embodiment, the period when the signal is at the H level corresponds to the exposure period during which light can be detected by a plurality of photoelectric conversion elements arranged in the photoelectric conversion region, and the period when the signal is at the L level corresponds to the non - exposure period during which light cannot be detected by the plurality of photoelectric conversion elements. The exposure period is the period during which the photoelectric conversion unit 102 is active and a signal detection circuit such as a counter circuit reads out the signal from the photoelectric conversion unit 102. The non - exposure period is the period during which a signal detection circuit such as a counter circuit does not read out the signal from the photoelectric conversion unit 102.

[0095] For example, the exposure period is the period during which the gate signal input to the gate circuit via the driving line GATE is at the H level. For example, the non - exposure period is the period during which the gate signal input to the gate circuit via the driving line GATE is at the L level. In the driving in which the quenching element 202 is reset by inputting a cyclic pulse to the driving line VR, the exposure period can be defined as the period from when the driving line VR switches to the H level until the gate signal switches to the L level.

[0096] The exposure period and the non-exposure period are not limited to the above examples. The exposure period can be defined as a period during which a reverse bias potential that enables avalanche multiplication is applied to the APD, the quenching element is in a non-quenched state, and a signal from the APD can be read out by a signal detection circuit such as a counter. The non-exposure period can be defined as a period during which the quenching element is in a quenched state and a signal from the APD is not read out via the quenching element. As another example, the non-exposure period can be defined as a period during which the potential difference applied to the APD is reduced to prevent avalanche multiplication in the APD, and the exposure period can be defined as a period during which the potential difference enables avalanche multiplication in the APD. Alternatively, the non-exposure period can be defined as a period that is controlled in a manner that does not drive a signal detection circuit such as a counter, and the exposure period can be defined as a period that is controlled in a manner that drives a signal detection circuit such as a counter.

[0097] For example, Figure 1 The first frame generation unit 37 shown generates a plurality of sub-frames, and the second frame generation unit 38 generates a distance measurement frame. The first frame generation unit 37 and the second frame generation unit 38 may be configured in the photoelectric conversion device 100. In this case, for example, the first frame generation unit 37 and the second frame generation unit 38 may be configured in Figure 4 the readout circuit 112 shown, and may calculate distance information based on the signal output from the APD. In this case, the control pulse generation unit 115 may generate frame timing.

[0098] To make the description easier to understand, in Figure 7 the emitted light, the reflected light, and the sub-frames are arranged in a row. Although there are multiple timings of the emitted light, Figure 7 one light emission timing in each sub-frame is shown. Specifically, in actual driving, based on the first emitted light, the first light quantity measurement of the first sub-frame is performed. In addition, based on the second emitted light, the second light quantity measurement of the first sub-frame is performed. Based on the m-th emitted light (m is an integer of 2 or more), the m-th light quantity measurement of the first sub-frame is performed, and based on the (m + 1)-th emitted light, the first light quantity measurement of the second sub-frame is performed. Subsequently, light quantity measurements are similarly performed until the N-th sub-frame.

[0099] As Figure 7As shown, during the period of generating the first sub-frame, without changing the timing from light emission to the start of the exposure period of the first sub-frame, the light quantity measurement of the first sub-frame including light emission is repeated multiple times. After that, the light quantity measurement of the second sub-frame is performed. The period from light emission to the start of the exposure period in the light quantity measurement of the second sub-frame is set to a period longer than the period from light emission to the start of the exposure period in the light quantity measurement of the first sub-frame. Similarly to the light quantity measurement of the first sub-frame, without changing the timing from light emission to the start of the exposure period of the second sub-frame, the light quantity measurement of the second sub-frame including light emission is repeated multiple times. Then, the light quantity measurement of the Nth sub-frame is performed. In the Nth sub-frame, similarly to the foregoing sub-frames, without changing the timing from light emission to the start of the exposure period, the light quantity measurement of the Nth sub-frame is performed multiple times. Based on the results of the light quantity measurements in the multiple sub-frames including the first sub-frame to the Nth sub-frame, histogram information related to the reflected light is generated. Based on the time information corresponding to the level with the largest number (frequency) of detected photons, the distance to the object is calculated.

[0100] In a comparative exemplary embodiment, in one sub-frame, detection is performed multiple times with the same start timing and end timing of the exposure period, and in the next sub-frame, the start timing and end timing of the exposure period are shifted relative to the timing in the previous sub-frame.

[0101] Figure 8A Shows the gate profile and the dwell period distribution in each sub-frame according to a comparative exemplary embodiment. The gate profile is a function representing the temporal variation of the sensitivity to incident photons in a single light quantity measurement of a predetermined sub-frame, and in the comparative exemplary embodiment, the gate profile corresponds to the period during which the gate is turned on. In Figure 8A For ease of description, the gate profile is represented by a rectangular function. In reality, due to pulse delay or finite reset time in the sensor, distortion, overshoot, or ringing may occur in the rising and falling waveforms. The dwell period distribution refers to the distribution indicating the number of times to accumulate the light reception timing when repeatedly measuring the emitted light and the received light in a certain sub-frame. In the comparative exemplary embodiment, since in the sub-frame, the reflected light is measured multiple times and accumulated under the condition that the period from light emission to the start timing of the exposure period is the same, the dwell period distribution indicates a δ function.

[0102] Figure 8B Shows the effective gate profile obtained by the convolution of the gate profile and the dwell period distribution according to a comparative exemplary embodiment.

[0103] Next, the present exemplary embodiment will be described. Figure 9 Is a drive timing diagram according to the present exemplary embodiment, and Figure 10A And Figure 10B Shows a graph indicating the gate drive per unit time in the drive according to the present exemplary embodiment.

[0104] In Figure 9 this, since the emitted light, reflected light, each sub-frame, and histogram are similar to those shown in Figure 7 it, the description thereof will be omitted.

[0105] In Figure 9 this, during the light amount measurement of each sub-frame, the exposure periods of a plurality of photoelectric conversion elements arranged in an array of a plurality of rows and columns included in the signal generation device are simultaneously controlled.

[0106] For example, in the photoelectric conversion region, the exposure periods of a plurality of photoelectric conversion elements arranged in an array of rows and columns are simultaneously controlled. Specifically, among at least two or more photoelectric conversion elements of the plurality of photoelectric conversion elements, the first exposure period starts simultaneously, and thereafter, the second exposure period starts simultaneously. Among the plurality of photoelectric conversion elements, the exposure periods of the plurality of photoelectric conversion elements arranged in the region where signal generation is to be performed are simultaneously controlled.

[0107] In this exemplary embodiment, one sub-frame includes a first exposure period P1 and a second exposure period P2, and the start timing of the first exposure period P1 and the start timing of the second exposure period P2 are different. The first sub-frame includes a first exposure period that starts exposure after a predetermined period from the light emission of the light-emitting element and a second exposure period that starts exposure after a period longer than the predetermined period from the light emission of the light-emitting element. Here, the predetermined period includes 0. In this exemplary embodiment, the period from the light emission of the light-emitting element until the start of the second exposure period is shorter than the period from the light emission of the light-emitting element until the end of the first exposure period. In other words, in one sub-frame, the second exposure period includes both a period overlapping with the first exposure period and a period not overlapping with the first exposure period. With this structure, since the exposure periods can overlap in one sub-frame, in the case of generating a histogram, distance measurement can be performed with high resolution.

[0108] In Figure 9 this, in one sub-frame, the start timing of the exposure period is shifted at a constant speed with respect to the light emission timing of the light source, but the structure is not limited thereto. For example, the start timing of the exposure period can be irregularly shifted with respect to the light emission timing of the light source.

[0109] For example, the first sub-frame may further include a third exposure period and a fourth exposure period, and the time differences between the start timings of the first to fourth exposure periods may be the same, or the time differences may be different. The time difference between the start timing of the first exposure period P1 and the start timing of the second exposure period P2 is referred to as the first time difference D1. The time difference between the start timing of the second exposure period P2 and the start timing of the third exposure period is referred to as the second time difference. The time difference between the start timing of the third exposure period and the start timing of the fourth exposure period is referred to as the third time difference. In the present exemplary embodiment, the first time difference, the second time difference, and the third time difference are set to the same length.

[0110] A histogram is generated by counting the cumulative signal amount obtained by accumulating the total signal amounts of the light amount values obtained in the exposure periods in the sub-frame. In the present exemplary embodiment, information such as the rise, fall, mode value, or centroid of the histogram is used to calculate the time difference from light emission to light reception. Therefore, the distance to the object can be measured.

[0111] As Figure 9 shown, in the present exemplary embodiment, a plurality of frames include a first sub-frame and a second sub-frame. In the first sub-frame, the exposure period starts after the first period from the light emission of the light-emitting element, and in the second sub-frame, the exposure period starts after the second period longer than the first period from the light emission of the light-emitting element.

[0112] Then, any exposure period among the plurality of exposure periods in the first sub-frame partially overlaps with any exposure period among the plurality of exposure periods in the second sub-frame. The period from the light emission of the light-emitting element until the end timing of at least one of the plurality of exposure periods in the first sub-frame is longer than the period from the light emission of the light-emitting element until the start timing of at least one of the plurality of exposure periods in the second sub-frame. With this structure, when generating a histogram, the cumulative signal amount of the first sub-frame and the cumulative signal amount of the second sub-frame can overlap. The first period is an integer including 0, and the second period is an integer not including 0. In other words, between the sub-frames, while shifting the start timing of the exposure period relative to the light emission timing of the light source, the light amount is measured. Therefore, compared with the case where the exposure periods do not overlap, the distance resolution can be improved.

[0113] Figure 10A Shows the gate profile and the dwell period distribution in each sub-frame according to the present exemplary embodiment. The gate profile is a function representing the time variation of the sensitivity to incident photons in a single light amount measurement of a predetermined sub-frame, and in the present exemplary embodiment, the gate profile corresponds to the period during which the gate is turned on. In Figure 10A order to facilitate description, the gate profile is represented by a rectangular function. In reality, due to pulse delay or finite reset time in the sensor, distortion, overshoot, or ringing may occur in the rising and falling waveforms. AsFigure 10A As shown, according to this exemplary embodiment, since the light reception timing is shifted at a constant speed in a sub-frame while accumulating, the accumulated residence time distribution is represented by a rectangular function. According to this exemplary embodiment, compared with the residence time distribution in the comparative exemplary embodiment, the width of the residence time distribution can be widened.

[0114] Figure 10B Shows the effective gate profile obtained by the convolution of the gate profile and the residence time distribution according to this exemplary embodiment. In this exemplary embodiment, since, as Figure 10A shown, the width of the residence time distribution can be made wider, the effective gate profile obtained by convolution has a shape such as a trapezoid.

[0115] According to this exemplary embodiment, the effective gate profile can adopt an intermediate value instead of a binary value, and the period adopting the intermediate value can overlap with a part of the effective gate profile of the immediately preceding sub-frame or the immediately following sub-frame. With this structure, based on the interpolation ratio of the accumulated output in the immediately preceding sub-frame or the immediately following sub-frame, a resolution finer than the gate shift interval can be obtained. Therefore, a signal generation device capable of ensuring an appropriate distance resolution without increasing the distance measurement time can be provided.

[0116] Reference will be made to Figure 11A and Figure 11B to describe the second exemplary embodiment. Figure 11A and Figure 11B Show graphs indicating the gate drive per unit time and the effective gate profile in the drive according to this exemplary embodiment. Figure 11A Show the gate profile and the residence time distribution in each sub-frame, and Figure 11B show the effective gate profile.

[0117] In the first exemplary embodiment, the start times of a plurality of exposure periods are shifted at a constant speed in one sub-frame. The difference between this exemplary embodiment and the first exemplary embodiment is that: in one sub-frame, the exposure periods are shifted in a binary manner at different timings, and exposure is performed multiple times at each timing. Except for this point and the points to be described below, the structure of this exemplary embodiment is basically the same as the structure of the first exemplary embodiment. Therefore, the description of other points will be omitted.

[0118] In this exemplary embodiment, in one sub-frame, the first exposure period for exposure starting after a first period has elapsed since light emission from the light emitting element is repeated multiple times, and the second exposure period for exposure starting after a second exposure period has elapsed since light emission from the light emitting element is repeated multiple times. Then, the cumulative signal amount is calculated using the signals obtained in the multiple first exposure periods and the signals obtained in the multiple second exposure periods.

[0119] In the present exemplary embodiment, the number of repetitions of the first exposure period and the number of repetitions of the second exposure period are set to the same number. For example, the first exposure period is repeated ten times, and the second exposure period is repeated ten times. The number is not limited to this. For example, the number of repetitions of the first exposure period and the number of repetitions of the second exposure period may be different. In this case, the distance to the object is measured based on a histogram taking into account that the number of repetitions of the first exposure period and the number of repetitions of the second exposure period are different. In the present exemplary embodiment, as Figure 11B shown, the effective gate profile has an inverted T shape. In other words, the effective gate profile has a convex shape.

[0120] According to the present exemplary embodiment, similarly to the first exemplary embodiment, a signal generation device capable of ensuring an appropriate distance resolution without increasing the distance measurement time can be provided. In addition, a pulse generator simpler than the pulse generator in the first exemplary embodiment can be used. Thereby, a reduction in the cost of the distance information generation device can be achieved.

[0121] In the present exemplary embodiment, as a modification, the lengths of the first exposure period and the second exposure period may vary. Figure 12A shows the gate profile 1 in the first exposure period, and Figure 12B shows the gate profile 2 in the second exposure period according to the modification. In the modification, the first exposure period is longer than the second exposure period.

[0122] Figure 12C shows the effective gate profile obtained by the synthetic product (convolution) of the gate profile 1, the gate profile 2, and the dwell period distribution. In this way, by accumulating the gate profile 1 and the gate profile 2 having different widths of the exposure period and different start timings of the exposure period multiple times, an effective gate profile similar to the Figure 11B shown effective gate profile can also be obtained. Therefore, also in the modification, a signal generation device capable of ensuring an appropriate distance resolution without increasing the distance measurement time can be provided.

[0123] Reference will be made to Figure 13A and Figure 13B to describe the third exemplary embodiment. Figure 13A and Figure 13B show graphs indicating the gate drive per unit time and the effective gate profile in the indication drive according to the present exemplary embodiment. Figure 13A show the gate profile and the dwell period distribution in each sub-frame, and Figure 13B show the effective gate profile.

[0124] In the present exemplary embodiment, the exposure periods in the first sub-frame and the exposure periods in the second sub-frame are shifted. Except for this point and the points to be described below, the present exemplary embodiment is substantially the same as the first exemplary embodiment. Therefore, the description will be omitted.

[0125] In the present exemplary embodiment, control is performed in such a manner that the end timing of the exposure periods in the first sub-frame becomes equal to or later than the start timing of the exposure periods in the second sub-frame. In addition, the timing difference between the sub-frames is made equal to or less than the width of the gate profile. With this configuration, compared with the first exemplary embodiment, a smaller number of sub-frames can be used to cover the distance measurement range to be observed. Therefore, compared with the first exemplary embodiment, the distance measurement time can be shortened. In particular, by making the timing difference almost equal to the width of the gate profile, the increase / decrease relationship of the effective gate profile can be reversed in the immediately preceding sub-frame or the immediately following sub-frame. More specifically, as Figure 13B shown, the region where the inclination of the effective gate profile of the first sub-frame has a negative value can correspond to the region where the inclination of the effective gate profile of the second sub-frame has a positive value.

[0126] In the present exemplary embodiment, similar to the first exemplary embodiment, a signal generation device capable of ensuring an appropriate distance resolution without increasing the distance measurement time can be provided. In addition, by acquiring data of a plurality of sub-frames while maintaining the reverse relationship of the increase / decrease relationship of the effective gate profile, high resolution can be obtained regardless of the distance to the object.

[0127] Reference will be made to Figure 14A and Figure 14B to describe the fourth exemplary embodiment. Figure 14A and Figure 14B show graphs indicating the gate drive per unit time and the effective gate profile in the indication drive according to the present exemplary embodiment. Figure 14A show the gate profile and the dwell period distribution in each sub-frame, and Figure 14B show the effective gate profile.

[0128] The difference between the present exemplary embodiment and the first exemplary embodiment is that, in one sub-frame, the amount of shift of the start timing of the exposure period gradually decreases as the period elapses, and thereafter, the amount of shift of the start timing of the exposure period gradually increases as the period elapses. Except for this point and the points to be described below, the present exemplary embodiment is substantially the same as the first exemplary embodiment. Therefore, the description of other points will be omitted.

[0129] In the present exemplary embodiment, the dwell period distribution in each sub-frame has a bilaterally symmetric triangular shape instead of a rectangular shape. For example, the first period difference and the third period difference are the same, and the second period difference is smaller than the first period difference. Therefore, as Figure 14BAs shown, compared with the first exemplary embodiment, the effective gate profile has a steeper slope. Therefore, compared with the first exemplary embodiment, by causing the light reflected from the object to span the immediately preceding sub-frame or the immediately following sub-frame, the distance measurement accuracy can be improved.

[0130] In this exemplary embodiment, similarly to the first exemplary embodiment, a signal generation device that can ensure an appropriate distance resolution without increasing the distance measurement time can be provided. In addition, compared with the first exemplary embodiment, the distance measurement accuracy can be improved.

[0131] In this exemplary embodiment, as Modification 1, as Figure 15A shown, the dwell period distribution in each sub-frame can have an asymmetric triangular shape. For example, it can be accumulated by gradually decreasing the shift amount of the start timing of the exposure period as time passes.

[0132] In other words, in one sub-frame, the overlap between multiple exposure periods can be increased in the second half of the sub-frame. The third time difference is less than the second time difference, and the second time difference is less than the first time difference. In this case, as Figure 15B shown, compared with the first exemplary embodiment, the effective gate profile also has a steeper slope. Therefore, in Modification 1, it is possible to further improve the distance measurement accuracy. Instead of gradually decreasing the shift amount of the start timing of the exposure period as time passes, a similar dwell period distribution can be achieved by changing the number of accumulations for each start timing of the exposure period while keeping the shift amount constant. In this case, the number of accumulations in the first exposure period and the number of accumulations in the second exposure period are different. For example, the number of accumulations in the second exposure period can be greater than the number of accumulations in the first exposure period.

[0133] In this exemplary embodiment, as Modification 2, as Figure 16A shown, the dwell period distribution in each sub-frame can have an M shape. For example, in one sub-frame, the shift amount of the start timing of the exposure period can gradually increase as time passes, and from a certain midpoint, the shift amount of the start timing of the exposure period can gradually decrease as time passes.

[0134] In other words, in one sub-frame, the overlap between multiple exposure periods can be increased in the first half and the second half of the sub-frame, and the overlap between multiple exposure periods can be decreased near the center of the sub-frame. Also in Modification 2, as Figure 16B shown, compared with the first exemplary embodiment, the effective gate profile has a steeper slope. Therefore, in Modification 2, it is possible to further improve the distance measurement accuracy.

[0135] In the present exemplary embodiment, as Modification 3, as Figure 17A shown, the residence period distribution in each sub-frame may have a parabolic shape. For example, in one sub-frame, the overlap between multiple exposure periods may increase near the center of the sub-frame, and the overlap between multiple exposure periods may decrease in the first half and the second half of the sub-frame. For example, the second time difference may be less than the first time difference, and the third time difference may be less than the second time difference. In this case, as Figure 17B shown, compared with the first exemplary embodiment, the effective gate profile also has a steeper slope. Therefore, in Modification 3, it is possible to further improve the distance measurement accuracy.

[0136] Reference will be made to Figure 18A and Figure 18B to describe the fifth exemplary embodiment. Figure 18A and Figure 18B show graphs indicating the gate drive per unit time and the effective gate profile in the indication drive according to the present exemplary embodiment. Figure 18A show the gate profile and the residence period distribution in each sub-frame, and Figure 18B show the effective gate profile.

[0137] The difference between the present exemplary embodiment and the first exemplary embodiment is that the counter included in the photoelectric conversion element is an up / down counter. Except for this point and the points to be described below, the present exemplary embodiment is basically the same as the first exemplary embodiment. Therefore, the description of other points will be omitted.

[0138] In the present exemplary embodiment, an up / down counter is used as the counter. With this configuration, it is possible to switch between up counting and down counting in one sub-frame. Therefore, as Figure 18A shown, negative values can be effectively taken in the residence period distribution. Negative values can also be taken in the effective gate profile obtained by convolution. Therefore, it is possible to perform highly accurate distance measurement while eliminating the influence of external light.

[0139] In the present exemplary embodiment, similar to the first exemplary embodiment, a signal generation device capable of ensuring an appropriate distance resolution without increasing the distance measurement time can be provided. In addition, compared with the first exemplary embodiment, the influence of external light can be reduced.

[0140] In the present exemplary embodiment, as a modification, as Figure 19A shown, the residence period distribution in each sub-frame may have a triangular shape instead of a rectangular shape. For example, in one sub-frame, the shift amount of the start timing of the exposure period may gradually decrease as the period elapses, and thereafter, the shift amount of the start timing of the exposure period may gradually increase as the period elapses. In this case, as Figure 19BAs shown, the effective gate profile may have an approximately sinusoidal shape. Therefore, in the case of performing processing using principles such as the Fourier transform, subsequent computational processing can be simplified.

[0141] Reference will be made to Figure 20 describe the signal generation system according to the sixth exemplary embodiment. Figure 20 is a block diagram showing the schematic structure of the signal generation system according to this exemplary embodiment.

[0142] The signal generation device (photoelectric conversion device) described in the above exemplary embodiment can be applied to various signal generation systems. Examples of signal generation systems (photoelectric conversion systems) to which the signal generation device can be applied include digital still cameras, digital video cameras, surveillance cameras, copiers, fax machines, mobile phones, in-vehicle cameras, and observation satellites. The signal generation system also includes a camera module having an optical system such as a lens and an imaging device. As an example, among these signal generation systems, Figure 20 shows a block diagram of a digital still camera.

[0143] Figure 20 The signal generation system exemplified in includes an imaging device 1004 serving as an example of a signal generation device and a lens 1002 for forming an optical image of a subject on the imaging device 1004. The signal generation system further includes an aperture 1003 for changing the amount of light passing through the lens 1002 and a shutter 1001 for protecting the lens 1002. The lens 1002 and the aperture 1003 serve as an optical system for focusing light onto the imaging device 1004. The imaging device 1004 is a signal generation device (imaging device) according to any of the above exemplary embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0144] The signal generation system further includes a signal processing unit 1007 serving as an image generation unit for generating an image by processing the electrical signal output from the imaging device 1004. The signal processing unit 1007 performs operations for appropriately performing various types of correction and compression and outputs image data. The signal processing unit 1007 can be formed on a semiconductor substrate provided with the imaging device 1004, or can be formed on another semiconductor substrate different from the semiconductor substrate provided with the imaging device 1004. The imaging device 1004 and the signal processing unit 1007 can be formed on the same semiconductor substrate.

[0145] The signal generation system further includes a memory unit 1010 for temporarily storing image data and an external interface unit (external I / F unit) 1013 for communicating with an external computer. The signal generation system further includes a recording medium 1012 such as a semiconductor memory for recording or reading out the captured image data and a recording medium control interface unit (recording medium control I / F unit) 1011 for performing recording onto or reading from the recording medium 1012. The recording medium 1012 may be built into the signal generation system or may be detachably attached to the signal generation system.

[0146] The signal generation system further includes an overall control / calculation unit 1009 for controlling various types of calculations and the entire digital still camera and a timing signal generation unit 1008 for outputting various timing signals to the imaging device 1004 and the signal processing unit 1007. A timing signal may be input from the outside of the signal generation system. In one embodiment, the signal generation system includes at least the imaging device 1004 and a signal processing unit 1007 for processing an electrical signal output from the imaging device 1004.

[0147] The imaging device 1004 outputs an imaging signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004 and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0148] In this way, according to this exemplary embodiment, a signal generation system applying the signal generation device (imaging device) according to any of the above exemplary embodiments can be realized.

[0149] Refer to Figure 21A and Figure 21B to describe the signal generation system and the movable body according to the seventh exemplary embodiment. Figure 21A and Figure 21B are diagrams showing the structures of the signal generation system and the movable body according to this exemplary embodiment.

[0150] Figure 21AAn example of a signal generation system related to an in-vehicle camera is shown. The signal generation system 1300 includes a signal generation device 1310. The signal generation device 1310 is the signal generation device described in any of the above typical embodiments. The signal generation system 1300 includes an image processing unit 1312 for performing image processing on a plurality of image data acquired by the signal generation device 1310. The signal generation system 1300 further includes a distance acquisition unit 1316 for calculating the distance to an object and a collision determination unit 1318 for determining whether a collision is likely based on the calculated distance. In this example, the distance acquisition unit 1316 can acquire distance information related to the distance to a ToF object, or can use parallax information to acquire distance information. More specifically, the distance information is information related to parallax, defocus amount, or the distance to an object. The collision determination unit 1318 can use any of this distance information to determine the likelihood of a collision. The distance information acquisition unit can be implemented by specially designed hardware, or can be implemented by a software module. Alternatively, the distance information acquisition unit can be implemented by a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), or can be implemented by a combination of these.

[0151] The signal generation system 1300 is connected to a vehicle information acquisition device 1320 and can acquire vehicle information such as vehicle speed, yaw rate, rudder angle, etc. The signal generation system 1300 is also connected to a control electronic control unit (ECU) 1330. The control ECU 1330 serves as a control device for outputting a control signal for generating the braking force of the vehicle based on the determination result obtained by the collision determination unit 1318. The signal generation system 1300 is also connected to an alarm device 1340, and the alarm device 1340 gives an alarm to the driver based on the determination result obtained by the collision determination unit 1318. For example, when the determination result obtained by the collision determination unit 1318 indicates a high likelihood of a collision, the control ECU 1330 performs vehicle control to avoid the collision and reduce damage by braking, releasing the accelerator, or reducing the engine output. The alarm device 1340 gives an alarm to the user by emitting an alarm such as a sound, displaying a warning message on the screen of the car navigation system, or vibrating the seat belt or the steering wheel.

[0152] In this typical embodiment, for example, the signal generation system 1300 captures images of the surroundings of the vehicle such as the front side or the rear side of the vehicle. Figure 21B The signal generation system 1300 for capturing an image of the front side of the vehicle (imaging range 1350) is shown. The vehicle information acquisition device 1320 issues an instruction to the signal generation system 1300 or the signal generation device 1310. With this structure, the accuracy of distance measurement can be further improved.

[0153] The description of the example of control in a manner such as not colliding with other vehicles has been given above. The signal generation system can also be applied to cases of performing automatic operation control for a vehicle to follow other vehicles or performing automatic operation control for preventing a vehicle from deviating from a lane. In addition to vehicles such as automobiles, the signal generation system can also be applied to movable bodies (movable devices) such as ships, aircraft, or industrial robots, for example. Such movable bodies include one or both of a driving force generation unit for generating a driving force mainly for the movement of the movable body and a rotating body mainly for the movement of the movable body. The driving force generation unit can be an engine or a motor, etc. The rotating body can be a tire, a wheel, a propeller of a ship, or a thruster of an aircraft, etc. In addition, in addition to movable bodies, the signal generation system can also be applied to various devices using object recognition, such as an intelligent transportation system (ITS).

[0154] Refer to Figure 22A and Figure 22B to describe the signal generation system according to the eighth exemplary embodiment. Refer to Figure 22A , glasses 1600 (smart glasses) used as the signal generation system according to the present exemplary embodiment will be described. The glasses 1600 include a signal generation device 1602. The signal generation device 1602 is the signal generation device described in each of the above exemplary embodiments. A display device including a light-emitting device such as an organic light-emitting diode (OLED) or a light-emitting diode (LED) can be provided on the back surface of each lens 1601. The number of signal generation devices 1602 can be one or more than one. Multiple types of signal generation devices can be used in combination. The configuration position of the signal generation device 1602 is not limited to Figure 22A the position shown.

[0155] The glasses 1600 also include a control device 1603. The control device 1603 serves as a power source for supplying power to the signal generation device 1602 and the above display device. The control device 1603 controls the operations of the signal generation device 1602 and the display device. In the lens 1601, an optical system for focusing light onto the signal generation device 1602 is formed.

[0156] Figure 22BFig. 1610 (smart glasses) according to an application example is shown. The glasses 1610 include a control device 1612, and the control device 1612 is equipped with a signal generation device equivalent to the signal generation device 1602 and a display device. In each lens 1611, an optical system for projecting the light emitted from the signal generation device and the display device of the control device 1612 is formed, and an image is projected onto the lens 1611. The control device 1612 serves as a power source for supplying power to the signal generation device and the display device, and controls the operations of the signal generation device and the display device. The control device 1612 may include a line-of-sight detection unit for detecting the line of sight of the wearer. Infrared light can be used to detect the line of sight. The infrared light emission unit emits infrared light onto the eyeball of the user who is looking at the displayed image. The imaging unit including a light receiving element detects the reflected light of the emitted infrared light that has been reflected by the eyeball. Thus, a captured image of the eyeball is obtained. A reduction unit for reducing the light from the infrared light emission unit to the display unit in the plan view is provided, which suppresses the degradation of the image quality.

[0157] From the captured image of the eyeball obtained by imaging using infrared light, the line of sight of the user with respect to the displayed image is detected. Any known method can be applied to the line-of-sight detection using the captured image of the eyeball. As an example, a line-of-sight detection method based on the Purkinje image obtained by reflecting the light emitted onto the cornea can be used.

[0158] More specifically, a line-of-sight detection process based on the corneal reflection of the pupil center is performed. Using the corneal reflection of the pupil center, based on the image of the pupil and the Purkinje image included in the captured image of the eyeball, a line-of-sight vector representing the direction (rotation angle) of the eyeball is calculated, and thus the line of sight of the user is detected.

[0159] The display device according to this exemplary embodiment may include a signal generation device having a light receiving element, and may control the display image on the display device based on the line-of-sight information related to the user from the signal generation device.

[0160] Specifically, in the display device, a first field-of-view region that the user wants to look at and a second field-of-view region other than the first field-of-view region are determined based on the line-of-sight information. The first field-of-view region and the second field-of-view region may be determined by the control device of the display device, or the first field-of-view region and the second field-of-view region determined by an external control device may be received. In the display area of the display device, the display resolution of the first field-of-view region can be controlled to be higher than that of the second field-of-view region. In other words, the resolution of the second field-of-view region can be made lower than that of the first field-of-view region.

[0161] The display area includes a first display area and a second display area different from the first display area. Based on the line-of-sight information, the area with a higher priority can be determined from the first display area and the second display area. The first display area and the second display area can be determined by the control device of the display device, or the first display area and the second display area determined by an external control device can be received. The resolution of the area with a higher priority can be controlled to be higher than the resolution of the areas other than the area with a higher priority. In other words, the resolution of the area with a relatively lower priority can be set to a low resolution.

[0162] Artificial intelligence (AI) can be used to determine the first field-of-view area and the area with a higher priority. The AI can be a model configured to use the image of the eyeball and the actual direction of gaze of the eyeball in the image as teacher data to estimate the angle of the line of sight and the distance to the object present in the line of sight from the image of the eyeball. The AI program can be included in the display device, can be included in the signal generation device, or can be included in an external device. In the case where the external device includes the AI program, the AI program is sent to the display device via communication.

[0163] In the case of performing display control based on line-of-sight detection, this exemplary embodiment can be applied to smart glasses that further include a signal generation device for capturing an image of the outside. The smart glasses can display the external information obtained by imaging in real time.

[0164] The above-mentioned signal generation device and signal generation system can be applied to electronic devices such as smart phones and tablet computers, for example.

[0165] Figure 23A and Figure 23B FIG. 15 is a diagram showing an example of an electronic device 1500 equipped with a signal generation device. Figure 23A shows the front side of the electronic device 1500, and Figure 23B shows the back side of the electronic device 1500.

[0166] As Figure 23A shown, a display 1510 for displaying an image is arranged at the center of the front of the electronic device 1500. Then, along the upper side of the front of the electronic device 1500, a front camera 1521 and 1522 using the signal generation device, an IR light source 1530 for emitting infrared light, and a visible light source 1540 for emitting visible light are arranged.

[0167] As Figure 23B shown, along the upper side of the back of the electronic device 1500, a rear camera 1551 and 1552 using the signal generation device, an IR light source 1560 for emitting infrared light, and a visible light source 1570 for emitting visible light are arranged.

[0168] In the electronic device 1500 having such a structure, the above-described signal generation device is used, which makes it possible to capture a higher-quality image while taking into account, for example, the distance to the object.

[0169] In addition to the above, the signal generation device can be applied to electronic devices such as infrared sensors, distance measurement sensors using active infrared light sources, security cameras, or human or biometric authentication cameras. The signal generation device can improve the accuracy and performance of these electronic devices.

[0170] In this specification, unless otherwise clearly defined, terms such as "A or B" and "at least one of A and B" can include all possible combinations of the listed items. In addition, unless otherwise clearly defined, terms such as "at least one of A or / and B" and "one or more than one of A or / and B" can include all possible combinations of the listed items. That is, the above terms are interpreted as disclosing all cases including the case of including at least one A, the case of including at least one B, and the case of including both at least one A and at least one B. This also applies to combinations of three or more components.

[0171] The above-described exemplary embodiments can be appropriately changed without departing from the technical idea. The disclosure in this specification is not limited to the matters described in this specification, and includes all matters that can be conceived from this specification and the accompanying drawings of this specification. The disclosure in this specification includes a complementary set of individual concepts described in this specification. More specifically, when "A is greater than B" is described in this specification, even if the description of "B is not greater than A" is omitted, it is considered that this specification discloses "B is not greater than A". This is because when "A is greater than B" is described, it is premised on the case of "B is not greater than A".

[0172] The disclosure of the above exemplary embodiments includes the following structures and methods.

[0173] (Structure 1)

[0174] A signal generation device, comprising:

[0175] A plurality of photoelectric conversion elements,

[0176] wherein the signal generation device uses signals from the plurality of photoelectric conversion elements to generate a plurality of sub-frames, and uses the signals of the plurality of sub-frames to generate a distance measurement frame,

[0177] Among them, the multiple sub-frames at least include a first sub-frame and a second sub-frame. In the first sub-frame, the exposure period starts after a first period has elapsed since the light-emitting element emits light. And in the second sub-frame, the exposure period starts after a second period longer than the first period has elapsed since the light-emitting element emits light.

[0178] Among them, the first sub-frame at least includes a first exposure period and a second exposure period.

[0179] Among them, the start timing of the first exposure period and the start timing of the second exposure period are different.

[0180] (Structure 2)

[0181] The signal generation device according to Structure 1, wherein the period from when the light-emitting element emits light until the start of the second exposure period is shorter than the period from when the light-emitting element emits light until the end of the first exposure period.

[0182] (Structure 3)

[0183] The signal generation device according to Structure 1 or 2,

[0184] wherein the multiple sub-frames are generated by a first generation unit, and

[0185] wherein the distance measurement frame is generated by a second generation unit.

[0186] (Structure 4)

[0187] The signal generation device according to any one of Structures 1 to 3,

[0188] wherein the first sub-frame includes multiple exposure periods, the multiple exposure periods include the first exposure period and the second exposure period, and

[0189] wherein, in the first sub-frame, the start timing of the multiple exposure periods is shifted at a constant speed.

[0190] (Structure 5)

[0191] The signal generation device according to any one of Structures 1 to 3, wherein, in the first sub-frame, the first exposure period and the second exposure period each repeat multiple times.

[0192] (Structure 6)

[0193] The signal generation device according to Structure 5, wherein a cumulative signal amount is counted, the cumulative signal amount is obtained by accumulating the light amount values obtained by repeating the first exposure period multiple times and the light amount values obtained by repeating the second exposure period multiple times.

[0194] (Structure 7)

[0195] The signal generation device according to any one of Structures 1 to 6, wherein a period from when light is emitted from the light-emitting element until an end timing of the exposure period in the first sub-frame is longer than a period from when light is emitted from the light-emitting element until a start timing of the exposure period in the second sub-frame.

[0196] (Structure 8)

[0197] The signal generation device according to any one of Structures 1 to 7, wherein in at least two or more than two of the plurality of photoelectric conversion elements, the second exposure periods start simultaneously.

[0198] (Structure 9)

[0199] The signal generation device according to any one of Structures 1 to 8,

[0200] wherein the first sub-frame includes a third exposure period and a fourth exposure period,

[0201] wherein the first sub-frame includes a first time difference between a start timing of the first exposure period and a start timing of the second exposure period, a second time difference between a start timing of the second exposure period and a start timing of the third exposure period, and a third time difference between a start timing of the third exposure period and a start timing of the fourth exposure period, and

[0202] wherein the first time difference and the third time difference are the same, and the second time difference is less than the first time difference.

[0203] (Structure 10)

[0204] The signal generation device according to any one of Structures 1 to 8,

[0205] wherein the first sub-frame includes a third exposure period and a fourth exposure period,

[0206] wherein the first sub-frame includes a first time difference between a start timing of the first exposure period and a start timing of the second exposure period, a second time difference between a start timing of the second exposure period and a start timing of the third exposure period, and a third time difference between a start timing of the third exposure period and a start timing of the fourth exposure period, and

[0207] wherein the first time difference is greater than the second time difference, and the second time difference is greater than the third time difference.

[0208] (Structure 11)

[0209] The signal generation device according to any one of Structures 1 to 8,

[0210] wherein the first sub-frame includes a third exposure period and a fourth exposure period,

[0211] wherein the first sub-frame includes a first time difference between the start timing of the first exposure period and the start timing of the second exposure period, a second time difference between the start timing of the second exposure period and the start timing of the third exposure period, and a third time difference between the start timing of the third exposure period and the start timing of the fourth exposure period, and

[0212] wherein the third time difference is less than the second time difference, and the second time difference is less than the first time difference.

[0213] (Structure 12)

[0214] The signal generation device according to any one of Structures 1 to 11,

[0215] wherein each of the plurality of photoelectric conversion elements includes an avalanche photodiode and a gate circuit, and the gate circuit is configured to control whether to output the signal of the avalanche photodiode, and

[0216] wherein the first exposure period and the second exposure period are set by controlling the signal to be supplied to the gate circuit.

[0217] (Structure 13)

[0218] The signal generation device according to Structure 12,

[0219] wherein the gate circuit is an AND circuit, and

[0220] wherein the signal from the avalanche photodiode and the signal from the drive line are input to the AND circuit.

[0221] (Structure 14)

[0222] The signal generation device according to Structure 12 or 13, wherein each of the plurality of photoelectric conversion elements includes a counter.

[0223] (Structure 15)

[0224] The signal generation device according to Structure 14, wherein the counter is an up / down counter.

[0225] (Structure 16)

[0226] The signal generation device according to any one of Structures 1 to 15,

[0227] Among them, the cumulative sum of the first exposure periods for multiple times, and the cumulative sum of the second exposure periods for multiple times, and

[0228] Among them, the cumulative number of times of the first exposure period is different from the cumulative number of times of the second exposure period.

[0229] (Structure 17)

[0230] The signal generation device according to Structure 16, wherein the cumulative number of times of the second exposure period is greater than the cumulative number of times of the first exposure period.

[0231] (Structure 18)

[0232] A signal generation system, comprising:

[0233] A light-emitting device, which includes a light-emitting element; and

[0234] The signal generation device according to any one of Structures 1 to 17,

[0235] wherein the signal generation device detects the reflected light of the light emitted from the light-emitting device and reflected by an object.

[0236] According to the present disclosure, a signal generation device can be provided that can ensure an appropriate distance resolution without increasing the distance measurement time.

[0237] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims is to be accorded the broadest interpretation so as to encompass all such modifications as well as equivalent structures and functions.

Claims

1. A signal generating device, comprising: A plurality of photoelectric conversion elements, wherein the signal generating device generates a plurality of sub-frames using signals from the plurality of photoelectric conversion elements, and generates a distance measurement frame using the signals of the plurality of sub-frames, The plurality of subframes include at least a first subframe and a second subframe, wherein in the first subframe, the exposure period starts after a first period of time has passed since the light emitting element emits light, and in the second subframe, the exposure period starts after a second period of time longer than the first period of time has passed since the light emitting element emits light, The first subframe includes at least a first exposure period and a second exposure period. wherein the start timing of the first exposure period is different from the start timing of the second exposure period, and Here, a period from when the light-emitting element emits light until the second exposure period starts is shorter than a period from when the light-emitting element emits light until the first exposure period ends.

2. The signal generating device according to claim 1, in, The plurality of subframes are generated by a first generating unit, and The distance measurement frame is generated by a second generating unit.

3. The signal generating device according to claim 1, in, The first subframe includes a plurality of exposure periods, the plurality of exposure periods include the first exposure period and the second exposure period, and Herein, in the first subframe, the start timings of the plurality of exposure periods are shifted at a constant speed.

4. The signal generating device according to claim 1, wherein: In the first subframe, the first exposure period and the second exposure period are each repeated a plurality of times.

5. The signal generating device according to claim 4, wherein: An accumulated signal amount obtained by integrating a light amount value obtained by repeating the first exposure period a plurality of times and a light amount value obtained by repeating the second exposure period a plurality of times is counted.

6. The signal generating device according to claim 1, wherein: A period from when the light emitting element emits light until an end timing of the exposure period in the first subframe is longer than a period from when the light emitting element emits light until a start timing of the exposure period in the second subframe.

7. The signal generating device according to claim 6, wherein: In at least two or more than two photoelectric conversion elements among the plurality of photoelectric conversion elements, the second exposure period starts simultaneously.

8. The signal generating device according to claim 1, in, The first subframe includes a third exposure period and a fourth exposure period, wherein the first subframe includes a first time difference between the start timing of the first exposure period and the start timing of the second exposure period, a second time difference between the start timing of the second exposure period and the start timing of the third exposure period, and a third time difference between the start timing of the third exposure period and the start timing of the fourth exposure period, and The first time period difference is the same as the third time period difference, and the second time period difference is smaller than the first time period difference.

9. The signal generating device according to claim 1, in, The first subframe includes a third exposure period and a fourth exposure period, wherein the first subframe includes a first time difference between the start timing of the first exposure period and the start timing of the second exposure period, a second time difference between the start timing of the second exposure period and the start timing of the third exposure period, and a third time difference between the start timing of the third exposure period and the start timing of the fourth exposure period, and The third time period difference is smaller than the second time period difference, and the second time period difference is smaller than the first time period difference.

10. The signal generating device according to claim 1, in, The first subframe includes a third exposure period and a fourth exposure period, wherein the first subframe includes a first time difference between the start timing of the first exposure period and the start timing of the second exposure period, a second time difference between the start timing of the second exposure period and the start timing of the third exposure period, and a third time difference between the start timing of the third exposure period and the start timing of the fourth exposure period, and The first time period difference is greater than the second time period difference, and the second time period difference is less than the third time period difference.

11. The signal generating device according to claim 1, in, The plurality of photoelectric conversion elements each include an avalanche photodiode and a gate circuit, the gate circuit being configured to control whether to output a signal of the avalanche photodiode, and The first exposure period and the second exposure period are set by controlling a signal to be supplied to the gate circuit.

12. The signal generating device according to claim 11, in, The gate circuit is an AND circuit, and The signal from the avalanche photodiode and the signal from the driving line are input to the AND circuit.

13. The signal generating device according to claim 11, wherein: Each of the plurality of photoelectric conversion elements includes a counter.

14. The signal generating device according to claim 13, wherein: The counter is an up / down counter.

15. The signal generating device according to claim 1, in, Accumulating multiple first exposure periods, accumulating multiple second exposure periods, and The accumulated number of times in the first exposure period is different from the accumulated number of times in the second exposure period.

16. The signal generating device according to claim 15, wherein: The accumulated number of times of the second exposure period is greater than the accumulated number of times of the first exposure period.

17. A signal generating device, comprising: A plurality of photoelectric conversion elements, wherein the signal generating device generates a plurality of sub-frames using signals from the plurality of photoelectric conversion elements, and generates a distance measurement frame using the signals of the plurality of sub-frames, The plurality of subframes include at least a first subframe and a second subframe, wherein in the first subframe, the exposure period starts after a first period of time has passed since the light emitting element emits light, and in the second subframe, the exposure period starts after a second period of time longer than the first period of time has passed since the light emitting element emits light, The first subframe includes at least a first exposure period and a second exposure period. The start timing of the first exposure period is different from the start timing of the second exposure period. The plurality of photoelectric conversion elements each include an avalanche photodiode and a gate circuit, the gate circuit being configured to control whether to output a signal of the avalanche photodiode, and The first exposure period and the second exposure period are set by controlling a signal to be supplied to the gate circuit.

18. The signal generating device according to claim 17, in, The first subframe includes a plurality of exposure periods, the plurality of exposure periods include the first exposure period and the second exposure period, and Herein, in the first subframe, the start timings of the plurality of exposure periods are shifted at a constant speed.

19. The signal generating device according to claim 17, wherein: A period from when the light emitting element emits light until an end timing of the exposure period in the first subframe is longer than a period from when the light emitting element emits light until a start timing of the exposure period in the second subframe.

20. The signal generating device according to claim 19, wherein: In at least two or more than two photoelectric conversion elements among the plurality of photoelectric conversion elements, the second exposure period starts simultaneously.

21. The signal generating device according to claim 17, in, The gate circuit is an AND circuit, and The signal from the avalanche photodiode and the signal from the driving line are input to the AND circuit.

22. The signal generating device according to claim 17, wherein: Each of the plurality of photoelectric conversion elements includes a counter.

23. The signal generating device according to claim 17, in, Accumulating multiple first exposure periods, accumulating multiple second exposure periods, and The accumulated number of times in the first exposure period is different from the accumulated number of times in the second exposure period.

24. The signal generating device according to claim 23, wherein: The accumulated number of times of the second exposure period is greater than the accumulated number of times of the first exposure period.

25. A signal generation system, comprising: A light emitting device comprising a light emitting element; as well as The signal generating device according to any one of claims 1 to 24, The signal generating device detects reflected light of light emitted from the light emitting device and reflected by an object.

Images