Signal generation device, signal generation system

Abstract

The objective is to provide a signal generation device that ensures appropriate distance resolution without increasing the distance measurement time. [Solution] A signal generation device that generates multiple subframes using signals from multiple photoelectric conversion elements to generate a distance measuring frame, wherein the multiple subframes include a first subframe whose exposure period starts after a first period from a first light emission, and a second subframe whose exposure period starts after a second period from a second light emission, and the first subframe has at least a first exposure period and a second exposure period, the period from the third light emission to the start of the second exposure period is shorter than the period from the first light emission to the end of the first exposure period, and the period from the first light emission to the end of the exposure period is longer than the period from the second light emission to the start of the exposure period.

Description

Technical Field

[0001] The present invention relates to a signal generation device and a signal generation system.

Background Art

[0002] Patent Document 1 discloses a distance measuring device that measures the distance to an object by emitting light from a light source and receiving the light including the reflected light from the object by a light receiving element. In the distance measuring device of Patent Document 1, a SPAD (Single Photon Avalanche Diode) element that multiplies electrons generated by photoelectric conversion to acquire a signal is used as the light receiving element. Patent Document 1 discloses a distance measuring method in which measurement is repeatedly performed while changing the start timing of the exposure period (Gating interval) for each sub-frame in which photon detection is performed in the SPAD element.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the distance measuring method as disclosed in Patent Document 1, there is a trade-off relationship between the distance resolution and the shortening of the distance measuring time. That is, when the exposure period in the sub-frame is lengthened, the distance measuring time is shortened, but the distance resolution is lowered. Conversely, when the exposure period in the sub-frame is shortened, the distance resolution is improved, but the distance measuring time becomes longer. However, in order to improve the distance measuring performance, there may be a case where it is required to ensure an appropriate distance resolution without lengthening the distance measuring time.

[0005] Therefore, an object of the present invention is to provide a signal generation device that ensures an appropriate distance resolution without lengthening the distance measuring time. [Means for solving the problem]

[0006] A signal generation device according to one aspect of the present invention has a plurality of photoelectric conversion elements, generates a plurality of subframes using signals from the plurality of photoelectric conversion elements, and generates a distance measuring frame using signals from the plurality of subframes, wherein the plurality of subframes include at least a first subframe whose exposure period starts after a first period from a first emission from a light-emitting element, and a second subframe whose exposure period starts after a second period longer than the first period from a second emission from the light-emitting element, and the first subframe includes at least a first exposure period and a second exposure period, and the light-emitting element A signal generating device characterized in that the period from the first emission of light to the start of the first exposure period is different from the period from the third emission of light from the light-emitting element to the start of the second exposure period, the period from the third emission of light from the light-emitting element to the start of the second exposure period is shorter than the period from the first emission of light from the light-emitting element to the end of the first exposure period, and the period from the first emission of light from the light-emitting element to the end of the exposure period in the first subframe is longer than the period from the second emission of light from the light-emitting element to the start of the exposure period in the second subframe. [Effects of the Invention]

[0007] According to the present invention, a signal generation device is provided that ensures appropriate distance resolution without increasing the distance measurement time. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing a schematic configuration of a distance information generation device according to an embodiment. [Figure 2] This is a schematic diagram of a photoelectric converter included in the signal generation device according to the embodiment. [Figure 3] This figure shows the arrangement of the sensor substrate of the photoelectric conversion device according to the embodiment. [Figure 4] This is a diagram showing the configuration of the circuit board of the photoelectric conversion device according to the embodiment. [Figure 5] This is an example of the configuration of a pixel circuit in a photoelectric conversion device according to an embodiment. [Figure 6] This is a schematic diagram showing the operation of the pixel circuit of the photoelectric converter according to the embodiment. [Figure 7] This is a timing diagram of the drive in the comparative configuration. [Figure 8] This graph shows the gate drive and effective gate profile per unit of time in the comparative drive configuration. [Figure 9] This is a timing diagram of the drive according to Embodiment 1. [Figure 10] This graph shows the gate drive and effective gate profile per unit of time in the drive according to Embodiment 1. [Figure 11] This graph shows the gate drive and effective gate profile per unit of time in the drive according to Embodiment 2. [Figure 12] This graph shows the gate drive and effective gate profile per unit of time in the drive according to a modified example of Embodiment 2. [Figure 13] This graph shows the gate drive and effective gate profile per unit of time in the drive according to Embodiment 3. [Figure 14] This graph shows the gate drive and effective gate profile per unit of time in the drive according to Embodiment 4. [Figure 15] This graph shows the gate drive and effective gate profile per unit of time in the drive according to Modification 1 of Embodiment 4. [Figure 16] This graph shows the gate drive and effective gate profile per unit of time in the drive according to Modification 2 of Embodiment 4. [Figure 17] This graph shows the gate drive and effective gate profile per unit of time in the drive according to Modification 3 of Embodiment 4. [Figure 18]A graph showing the driving of a gate per unit time and the effective gate profile in the driving according to Embodiment 5. [Figure 19] A graph showing the driving of a gate per unit time and the effective gate profile in the driving according to a modified example of Embodiment 5. [Figure 20] A functional block diagram of a signal generation system according to Embodiment 6. [Figure 21] A functional block diagram of a signal generation system according to Embodiment 7. [Figure 22] A functional block diagram of a signal generation system according to Embodiment 8. [Figure 23] A functional block diagram of a signal generation system according to Embodiment 9.

Embodiments for Carrying Out the Invention

[0009] The following embodiments are for embodying the technical idea of the present invention and do not limit the present invention. The sizes and positional relationships of the members shown in each drawing may be exaggerated for clarity of explanation. In the following description, the same components may be given the same numbers and the description may be omitted.

[0010] Hereinafter, embodiments of the present invention will be described in detail based on the drawings. In the following description, terms indicating a specific direction or position (for example, "upper", "lower", "right", "left", and other terms including those terms) are used as necessary. The use of those terms is for facilitating the understanding of the embodiments with reference to the drawings, and the technical scope of the present invention is not limited by the meanings of those terms.

[0011] In this specification, the plane refers to the plane viewed from the direction perpendicular to the light incident surface of the semiconductor layer. The cross-section refers to the plane in the direction perpendicular to the light incident surface of the semiconductor layer. When the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the plane is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0012] In the following description, the anode of the avalanche photodiode (APD) is set to a fixed potential, and the signal is taken from the cathode side. Therefore, the semiconductor region of the first conductivity type, where the majority carriers are charges of the same polarity as the signal charge, is an N-type semiconductor region, and the semiconductor region of the second conductivity type, where the majority carriers are charges of a different polarity than the signal charge, is a P-type semiconductor region. Note that the present invention also applies when the cathode of the APD is set to a fixed potential and the signal is taken from the anode side. In this case, the semiconductor region of the first conductivity type, where the majority carriers are charges of the same polarity as the signal charge, is a P-type semiconductor region, and the semiconductor region of the second conductivity type, where the majority carriers are charges of a different polarity than the signal charge, is an N-type semiconductor region. The following description will focus on the case where one node of the APD is set to a fixed potential, but the potentials of both nodes may fluctuate.

[0013] In the following embodiments, connections between circuit elements may be described. In this case, even if another element is interposed between the elements of interest, unless otherwise specified, the elements of interest will be treated as connected. For example, suppose element A is connected to one node of a capacitive element C having multiple nodes, and element B is connected to the other node. Even in such a case, elements A and B will be treated as connected unless otherwise specified.

[0014] Figure 1 is a hardware block diagram showing a schematic configuration example of a distance information generation device 30 according to an embodiment. The distance information generation device 30 includes a light-emitting device 31, a signal generation device 32, and a signal processing circuit 33. The signal generation device 32 may include a photoelectric converter 100 and a signal processing circuit 33. Note that the configuration of the distance information generation device 30 shown in Figure 1 is just one example and is not limited to the illustrated configuration. For example, the photoelectric converter 100 may include a first frame generation unit 37 and a second frame generation unit 38 included in the signal processing circuit 33.

[0015] The distance information generation device 30 is a device that measures the distance to an object X using technologies such as LiDAR (Light Detection and Ranging). The distance information generation device 30 measures the distance from the device 30 to the object X based on the time difference between when the light emitted from the light-emitting device 31 is reflected by the object X and when it is received by the photoelectric converter 100. The distance information generation device 30 can also measure distances at multiple points in two dimensions by emitting laser light into a predetermined distance measurement range including the object X and receiving the reflected light with a pixel array. As a result, the distance information generation device 30 can output distance information. Alternatively, it may output image information based on the distance information (image information with differences in color or contrast according to the distance information).

[0016] The light received by the signal generator 32 includes ambient light such as sunlight in addition to reflected light from the object X. Therefore, the distance information generator 30 measures the amount of incident light during each of several periods (bin periods) and uses a method that determines that reflected light was incident during the period when the light amount is at its peak, thereby reducing the influence of ambient light during distance measurement.

[0017] The light-emitting device 31 is a device that emits light, such as laser light, to the outside of the distance information generation device 30. For example, a VCSEL (Vertical Cavity Surface Emitting Laser), which is easy to array in two dimensions, may be used as the laser light.

[0018] The signal processing circuit 33 may include a processor that performs arithmetic operations on digital signals, a memory for storing digital signals, and the like. For example, a semiconductor memory can be used as the memory. The distance information generation device 30 does not necessarily have a signal processing circuit 33. In this case, at least a part of the components included in the signal processing circuit 33 is arranged inside the photoelectric converter 100. In this case, the signal generation device 32 and the photoelectric converter 100 are the same.

[0019] The signal generator 32 generates a pulse signal that includes a pulse based on the incident light. In this embodiment, the pulse signal is generated by the photoelectric converter 100 included in the signal generator 32. For example, a photoelectric converter that includes an APD as a photoelectric conversion element can be used. In this case, when one photon is incident on the APD and a charge is generated, one pulse is generated by avalanche multiplication. The photoelectric converter 100 included in the signal generator 32 is not limited to one that uses an APD as a photoelectric conversion element, but may also use a photoelectric conversion element that uses other photodiodes.

[0020] In this embodiment, the photoelectric converter 100 includes a pixel array in which multiple photoelectric conversion elements (pixels) are arranged across multiple rows and multiple columns. The photoelectric converter 100 will now be described with reference to Figures 2 to 6. The configuration example of the photoelectric converter described below is just one example. The photoelectric converter is not limited to this and can be any device capable of realizing the functions of each embodiment described later.

[0021] Figure 2 shows the configuration of a stacked photoelectric converter 100 included in the distance information generation device 30. The photoelectric converter 100 is constructed by stacking two substrates, a sensor substrate 11 (first substrate) and a circuit board 21 (second substrate), and electrically connecting them. The sensor substrate 11 has a first semiconductor layer having a photoelectric conversion unit 102 (described later) and a first wiring structure. The circuit board 21 has a second semiconductor layer having a signal detection circuit such as a signal processing unit 103 (described later) and a second wiring structure. The photoelectric converter 100 is constructed by stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in that order. The photoelectric converter 100 described in each embodiment is a back-illuminated type photoelectric converter in which light is incident from the first surface side of the first semiconductor layer of the sensor substrate 11, and the circuit board is arranged on the second surface facing the first surface of the first semiconductor layer of the sensor substrate 11.

[0022] In the following description, the sensor substrate 11 and the circuit board 21 are explained using diced chips, but they are not limited to chips. For example, each substrate may be a wafer. Furthermore, each substrate may be stacked in wafer form and then diced, or the wafers may be formed into chips, and then the chips may be stacked and bonded together.

[0023] The sensor substrate 11 has a photoelectric conversion region 12 in which multiple photoelectric conversion elements are arranged in a two-dimensional array, and the circuit substrate 21 has a circuit region 22 that processes the signals detected in the photoelectric conversion region 12.

[0024] Figure 3 shows an example of the arrangement of the sensor substrate 11. Photoelectric conversion elements 101, each having a photoelectric conversion unit 102 including an APD, are arranged in a two-dimensional array in a plan view, forming a photoelectric conversion region 12.

[0025] The photoelectric conversion element 101 only needs to be used to measure the time and amount of light that arrives. For example, when used in TOF (Time of Flight), it does not necessarily need to form an image. However, the photoelectric conversion element 101 may also be a pixel that forms an image.

[0026] Figure 4 is a diagram of the circuit board 21. It includes a signal processing unit 103 for processing the charge photoelectrically converted by the photoelectric conversion unit 102 in Figure 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, a signal line 113, a vertical scanning circuit unit 110, an output circuit 114, and a drive line 116.

[0027] The photoelectric conversion unit 102 in Figure 3 and the signal processing unit 103 in Figure 4 are electrically connected via connecting wiring provided for each photoelectric conversion element.

[0028] The vertical scanning circuit section 110 receives control pulses supplied from the control pulse generation section 115 and supplies control pulses to each photoelectric conversion element via the drive line 116. Logic circuits such as a shift register and an address decoder are used in the vertical scanning circuit section 110.

[0029] The signal output from the photoelectric conversion unit 102 of the photoelectric conversion element is processed by the signal processing unit 103. The signal processing unit 103 is equipped with a counter and memory, and digital values ​​are stored in the memory.

[0030] The horizontal scanning circuit unit 111 inputs control pulses to the signal processing unit 103 to sequentially select each column in order to read a signal from the memory of each photoelectric conversion element in which a digital signal is held.

[0031] Signals are output to signal line 113 from the signal processing unit 103 of the photoelectric conversion element selected by the vertical scanning circuit unit 110 for the selected column.

[0032] The signal output to signal line 113 is output via output circuit 114 to an external recording unit or signal processing unit of the photoelectric converter 100.

[0033] In Figure 2, the arrangement of photoelectric conversion units in the photoelectric conversion region may be one-dimensional. The function of the signal processing unit does not necessarily need to be provided for each photoelectric conversion unit; for example, a single signal processing unit may be shared by multiple photoelectric conversion units, and signal processing may be performed sequentially.

[0034] As shown in Figures 3 and 4, multiple signal processing units 103 are arranged in the region that overlaps with the photoelectric conversion region 12 in a plan view. Then, in a plan view, the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the readout circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged so as to overlap between the edge of the sensor substrate 11 and the edge of the photoelectric conversion region 12. In other words, the sensor substrate 11 has a photoelectric conversion region 12 and a non-photoelectric conversion region arranged around the photoelectric conversion region 12. Then, in a plan view, the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the readout circuit 112, the output circuit 114, and the control pulse generation unit 115 are arranged in the region that overlaps with the non-photoelectric conversion region.

[0035] Figure 5 is an example of a block diagram including the equivalent circuits of Figures 3 and 4. Figure 5 shows a block diagram of a photoelectric converter with a typical APD.

[0036] In Figure 5, the photoelectric conversion unit 102 having the APD201 is provided on the sensor substrate 11, and the other components are provided on the circuit board 21.

[0037] The APD201 generates charge pairs corresponding to incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD201. In addition, a voltage VH (second voltage), which is higher than the voltage VL supplied to the anode, is supplied to the cathode of the APD201. A reverse bias voltage is supplied to the anode and cathode such that the APD201 performs avalanche multiplication. By supplying these voltages, the charge generated by the incident light undergoes avalanche multiplication, and an avalanche current is generated.

[0038] Furthermore, when a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the anode and cathode operate with a potential difference greater than the breakdown voltage, and linear mode, in which the anode and cathode operate with a potential difference near or below the breakdown voltage.

[0039] An APD operating in Geiger mode is called a SPAD (Single Photon Avalanche Diode). For example, the voltage VL is -30V and the voltage VH is 1V. The APD201 can be operated in linear mode or Geiger mode.

[0040] The quench element 202 is connected to the power supply that provides voltage VH and to the APD201. When the signal is amplified by avalanche multiplication, the quench element 202 functions as a load circuit (quench circuit), suppressing the voltage supplied to the APD201 and thereby suppressing avalanche multiplication (quench operation). In addition, the quench element 202 also works to restore the voltage supplied to the APD201 to voltage VH by flowing the current that compensates for the voltage drop caused by the quench operation (recharge operation).

[0041] In Figure 5, the quench element 202 is composed of a transistor, and a potential is supplied to the gate of the quench element via the drive line VR. The potential supplied from the drive line VR is a reset signal that resets the cathode potential nodeA of the APD201 by switching the resistance value of the quench element 202.

[0042] The signal processing unit 103 includes a waveform shaping unit 210, a gate circuit 321 (first selection circuit), a counter 211, and an output circuit 306 (second selection circuit). In this specification, the signal processing unit 103 may include any of the waveform shaping unit 210, the gate circuit 321, the counter 211, or the output circuit 306.

[0043] The waveform shaping unit 210 shapes the cathode potential change of the APD201 obtained during photon detection and outputs a pulse signal. For example, an inverter circuit can be used as the waveform shaping unit 210. Figure 5 shows an example in which one inverter is used as the waveform shaping unit 210, but a circuit in which multiple inverters are connected in series may be used, or other circuits that have a waveform shaping effect may be used.

[0044] The gate circuit 321 can be configured, for example, by an AND gate. One input terminal of the AND gate is connected to the waveform shaping unit 210, and the other input terminal is connected to the drive line GATE. The exposure period, described later, is set by controlling the signal supply to the gate circuit 321. The gate circuit 321 outputs the output signal of the waveform shaping unit 210 to the counter 211 during periods when the gate signal input from outside the photoelectric conversion element via the drive line GATE is at a High (H) level. On the other hand, the gate circuit 321 does not output the output signal of the waveform shaping unit 210 to the counter 211 during periods when the gate signal is at a Low (L) level. As the gate signal, for example, by inputting a pulse with a High (H) level period at the nanosecond or picosecond level, it is possible to selectively detect only the photon signals incident during the subframe period, which is the period of interest.

[0045] The counter 211 counts the number (number of times) of pulse signals output from the waveform shaping unit 210 and stores the count value. The counter 211 measures the amount of light incident on the photoelectric conversion element. The counter 211 switches between performing the counting operation and stopping the counting operation in response to a control signal input from outside the photoelectric conversion element via the drive line CTRL. An example of the counter 211 is a multi-bit digital counter, but a 1-bit digital memory or an analog memory using a capacitive element may also be used. The counter 211 and the output circuit 306 are connected by a number of wires corresponding to the number of bits in the counter 211. Also, when the control pulse pRES is supplied via the drive line RES, the signal held in the counter 211 is reset.

[0046] The output circuit 306 receives a selection signal input from outside the photoelectric conversion element via the drive line SEL, and outputs the signal output from the counter 211 to the signal line 113. In this embodiment, a control pulse pSEL is supplied from the vertical scanning circuit section 110 in Figure 4 via the drive line SEL in Figure 5, switching the electrical connection and disconnection between the counter 211 and the signal line 113. The output circuit 306 includes, for example, a buffer circuit for outputting signals, and for example, a 3-state buffer can be used.

[0047] A switch such as a transistor may be placed between the quench element 202 and the APD201, or between the photoelectric conversion unit 102 and the signal processing unit 103, to switch the electrical connection. Alternatively, the supply of voltage VH or voltage VL to the photoelectric conversion unit 102 may be electrically switched using a switch such as a transistor. Furthermore, the above electrical connection may be switched by switching the voltage input to the gate of the transistor constituting the quench element 202 without providing an additional switch.

[0048] Figure 6 schematically illustrates the relationship between the operation of the APD and the output signal.

[0049] Figure 6(a) is an excerpt of the APD201, quench element 202, and waveform shaping unit 210 from Figure 5. Here, the input side of the waveform shaping unit 210 is denoted as nodeA and the output side as nodeB. Figure 6(b) shows the waveform change at nodeA in Figure 6(a), and Figure 6(c) shows the waveform change at nodeB in Figure 6(a).

[0050] Between time t0 and time t1, a potential difference of VH-VL is applied to APD201 in Figure 6(a). When a photon is incident on APD201 at time t1, avalanche multiplication occurs in APD201, an avalanche multiplication current flows through the quench element 202, and the voltage at nodeA drops. As the voltage drop increases further and the potential difference applied to APD201 decreases, the avalanche multiplication of APD201 stops as at time t2, and the voltage level at nodeA stops dropping below a certain value. Subsequently, between time t2 and time t3, a current flows through nodeA to compensate for the voltage drop from voltage VL, and at time t3, nodeA settles to its original potential level. At this time, any portion of the output waveform at nodeA that exceeds a certain threshold is waveform-shaped by the waveform shaping unit 210 and output as a signal at nodeB.

[0051] Note that the arrangement of the signal line 113, the read circuit 112, and the output circuit 114 is not limited to Figure 5. For example, the signal line 113 may be arranged extending in the row direction, and the read circuit 112 may be located at the end of the signal line 113.

[0052] (Embodiment 1) The photoelectric conversion device according to Embodiment 1 will be described using Figures 9 and 10. Here, with reference to Figures 7 and 8 as comparative examples, the principle that ensures appropriate distance resolution without increasing the distance measurement time, which is an effect of the present invention, will be explained using Figures 9 and 10.

[0053] Figure 7 is a timing diagram of the drive for the comparative configuration, and Figure 8 is a graph showing the gate drive per unit of time in the drive for the comparative configuration.

[0054] In Figure 7, the emitted light indicates the timing of light emission from the light-emitting element. The reflected light indicates the timing of light emitted from the light-emitting element being reflected after irradiating an object and detected by the photoelectric converter, which is a signal detection device.

[0055] A single distance measuring frame is generated using the signals from N subframes (where N is an integer greater than or equal to 2), including the first subframe and the second subframe. In each subframe, the H-level period is the period during which light from the light-emitting element can be detected. In each subframe, the L-level period is the period during which light from the light-emitting element cannot be detected. In this embodiment, the H-level period corresponds to the exposure period during which light can be detected by the multiple photoelectric conversion elements arranged in the photoelectric conversion region, and the L-level period corresponds to the non-exposure period during which light cannot be detected by the multiple photoelectric conversion elements. The exposure period refers to, for example, the period during which the photoelectric conversion unit 102 is active and the signal from the photoelectric conversion unit 102 is read out by a signal detection circuit such as a counter circuit. The non-exposure period refers to the period during which the signal from the photoelectric conversion unit 102 is not read out by the signal detection circuit such as a counter circuit.

[0056] For example, the exposure period is the period during which the gate signal input to the gate circuit via the drive line GATE is at the H level. Also, for example, the non-exposure period is the period during which the gate signal input to the gate circuit via the drive line GATE is at the L level. In a drive that resets the quench element 202 by applying periodic pulses to VR, the exposure period may be defined as the period from when VR switches to the H level until the gate signal switches to the L level.

[0057] The exposure and non-exposure periods are not limited to the examples above. The exposure period may be the period during which a reverse bias potential capable of avalanche multiplication is applied to the APD, the quench element is in a non-quenched state, and a signal detection circuit such as a counter can read the signal from the APD. The non-exposure period may be the period during which the quench element is in a quenched state and a signal from the APD is not read through the quench element. As another example, the non-exposure period may be the period during which the potential difference applied to the APD is reduced so that avalanche multiplication does not occur in the APD, and the exposure period may be the period during which a potential difference is applied that causes the APD to perform avalanche multiplication. Furthermore, the non-exposure period may be the period during which a signal detection circuit such as a counter is controlled not to be driven, and the exposure period may be the period during which a signal detection circuit such as a counter is controlled to be driven.

[0058] Multiple subframes are generated, for example, by the first frame generation unit 37 shown in Figure 1, and the distance measurement frame is generated by the second frame generation unit 38. The first frame generation unit 37 and the second frame generation unit 38 may also be located in the photoelectric converter 100. In this case, for example, the first frame generation unit 37 and the second frame generation unit 38 can be located in the readout circuit 112 shown in Figure 4, and distance information and the like can be calculated based on the signal output from the APD. In this case, frame timing generation can be performed by the control pulse generation unit 115.

[0059] In Figure 7, for the sake of clarity, the subframes are shown in order from the emitted light and reflected light. Although there are multiple timings for the emitted light, only one timing for the emitted light in each subframe is shown in the diagram. Specifically, in actual operation, the first light intensity measurement of the first subframe is performed with the first emitted light. The second light intensity measurement of the first subframe is performed with the second emitted light. The mth (where m is an integer greater than or equal to 2)th emitted light is used for the mth light intensity measurement of the first subframe, and the m+1th emitted light is used for the first light intensity measurement of the second subframe. Light intensity measurements are performed similarly up to the Nth subframe.

[0060] As shown in Figure 7, during the period for generating the first subframe, the light intensity of the first subframe is measured, including multiple repeated emissions, without changing the timing from emission to the start of the exposure period of the first subframe. Subsequently, the light intensity of the second subframe is measured. The period from emission to the start of the exposure period in the second subframe is set to be longer than the period from emission to the start of the exposure period in the first subframe. Similar to the light intensity measurement of the first subframe, the light intensity of the second subframe is measured, including multiple repeated emissions, without changing the timing from emission to the start of the exposure period of the second subframe. Then, the light intensity of the Nth subframe is measured, and in the Nth subframe as with each subframe, the light intensity of the subframe is measured multiple times without changing the timing from emission to the start of the exposure period. Based on the light intensity measurement results of multiple subframes, including the first to the Nth subframes, histogram information of reflected light is generated. The distance to the object is calculated based on the time information corresponding to the class with the highest number of detected photons (frequency).

[0061] In the comparison method, the start and end timings of the exposure period are detected multiple times in the same subframe, and then in the next subframe, the relative timing of the start and end timings of the exposure period is shifted relative to the previous subframe.

[0062] Figure 8(a) shows the gate profile and residence time distribution in each subframe in the comparison configuration. The gate profile is a function that represents the time change in sensitivity to incident photons in one light intensity measurement within a given subframe, and in the comparison configuration, it corresponds to the period during which the gate is ON. In Figure 8(a), the gate profile is represented by a rectangular function for simplicity of explanation, but in reality, waveform distortion, overshoot, ringing, etc. may occur in the rising and falling edges due to pulse delays within the sensor and a finite reset time. The residence time distribution is a distribution that represents how many times each light reception timing is integrated when light emission and reception are repeatedly measured in a given subframe. In the comparison configuration, since the reflected light is measured and integrated multiple times while keeping the period from light emission to the start of the exposure period the same in each subframe, it becomes a delta function.

[0063] Furthermore, Figure 8(b) shows the effective gate profile obtained by the combined product (convolution) of the gate profile and the residence duration distribution in the comparative configuration.

[0064] Next, this embodiment will be described. Figure 9 is a timing diagram of the drive according to this embodiment, and Figure 10 is a graph showing the gate drive per unit of time in the drive according to this embodiment.

[0065] In Figure 9, the emitted light, reflected light, each subframe, and histogram represent the same objects as in Figure 7, so an explanation is omitted.

[0066] In Figure 9, during the light intensity measurement of each subframe, the exposure periods of multiple photoelectric conversion elements arranged in a matrix within the signal generation device are controlled simultaneously. For example, in the photoelectric conversion region, the exposure periods of multiple photoelectric conversion elements arranged in a matrix within the array are controlled simultaneously. Specifically, at least two of the multiple photoelectric conversion elements simultaneously begin a first exposure period, and then simultaneously begin a second exposure period. The multiple photoelectric conversion elements simultaneously control the exposure periods of multiple photoelectric conversion elements positioned in the region where signal generation is desired.

[0067] In this embodiment, a single subframe has a first exposure period P1 and a second exposure period P2, with different timings for starting the first exposure period P1 and the second exposure period P2. The first subframe has a first exposure period that begins exposure a predetermined period after emission from the light-emitting element, and a second exposure period that begins exposure a period longer than the predetermined period after emission from the light-emitting element. Here, the predetermined period includes 0. In this embodiment, the period from emission from the light-emitting element to the start of the second exposure period is shorter than the period from emission from the light-emitting element to the end of the first exposure period. In other words, within a single subframe, the second exposure period has both a period that overlaps with the first exposure period and a period that does not overlap with the first exposure period. This allows exposure periods to overlap within a single subframe, making it possible to perform distance measurement with high resolution when generating a histogram.

[0068] In Figure 9, the start timing of the exposure period is shifted at a constant velocity relative to the light emission timing of the light source within a single subframe, but this is not limited to this. For example, the start timing of the exposure period may be shifted irregularly relative to the photoelectric light emission timing.

[0069] For example, the first subframe further includes a third exposure period and a fourth exposure period, and the time difference between the start timings of the first to fourth exposure periods may be the same or different. The time difference between the start timing of the first exposure period P1 and the start timing of the second exposure period P2 is defined as the first time difference D1. The time difference between the start timing of the second exposure period P2 and the start timing of the third exposure period is defined as the second time difference. The time difference between the start timing of the third exposure period and the start timing of the fourth exposure period is defined as the third time difference. In this embodiment, the first, second, and third time differences are the same.

[0070] A histogram is generated by accumulating the light intensity values ​​obtained during each exposure period in each subframe and counting the accumulated signal amount to obtain the total signal amount. In this embodiment, the time difference from light emission to light reception is calculated using information such as the rise, fall, mode, or centroid of the histogram. This makes it possible to measure the distance to the object.

[0071] As shown in Figure 9, in this embodiment, the first subframe has a first subframe in which the exposure period begins after a first period from the emission of light from the light-emitting element, and a second subframe in which the exposure period begins after a second period longer than the first period from the emission of light from the light-emitting element. The exposure period in the second subframe partially overlaps with one of the multiple exposure periods in the first subframe. The period from the emission of light from the light-emitting element in the first subframe to the end timing of at least one of the multiple exposure periods is longer than the period from the emission of light from the light-emitting element in the second subframe to the start timing of at least one of the multiple exposure periods. This allows the integrated signal amount of the first subframe and the integrated signal amount of the second subframe to be superimposed when generating the histogram. The first period is an integer including 0, and the second period is an integer not including 0. In other words, light intensity measurement is performed between subframes while shifting the start timing of the exposure period relative to the emission timing of the light source. Therefore, the distance resolution can be increased compared to when they are not superimposed.

[0072] Figure 10(a) shows the gate profile and the residence period distribution in each subframe in this embodiment. The gate profile is a function that represents the time change in sensitivity to incident photons in one light intensity measurement within a predetermined subframe, and in this embodiment it corresponds to the period during which the gate is ON. In Figure 10(a), the gate profile is represented by a rectangular function for simplicity of explanation, but in reality, waveform distortion, overshoot, ringing, etc. may occur at the rising and falling edges due to pulse delays within the sensor and a finite reset time. As shown in Figure 10(a), according to this embodiment, since the light reception timing is shifted at a constant velocity within the subframe and integrated, the integrated residence period distribution becomes a rectangular function. According to this embodiment, the width of the residence period distribution can be widened compared to the comparative form.

[0073] Furthermore, Figure 10(b) shows the effective gate profile obtained by the combined product (convolution) of the gate profile and the residence period distribution in this embodiment. In this embodiment, as shown in Figure 9(a), the width of the residence period distribution can be made wide, so the effective gate profile obtained by convolution has a trapezoidal shape.

[0074] According to this embodiment, the effective gate profile can take an intermediate value instead of being binary, and the period for taking the intermediate value can overlap with a part of the effective gate profile of the immediately preceding or succeeding subframe. As a result, it is possible to obtain a resolution finer than the gate shift interval based on the internal division ratio of the integrated power in the immediately preceding or succeeding subframe. Therefore, it is possible to provide a signal generation device that ensures appropriate distance resolution without increasing the distance measurement time.

[0075] (Embodiment 2) This embodiment will be described with reference to Figure 11. Figure 11 is a graph showing the gate drive and effective gate profile per unit of time in the drive according to this embodiment. Figure 11(a) shows the gate profile and the distribution of residence time in each subframe, and Figure 11(b) shows the effective gate profile.

[0076] In Embodiment 1, the start times of multiple exposure periods were shifted at a constant rate within a single subframe. In this embodiment, however, the exposure periods are shifted binaryly at different timings within a single subframe, and multiple exposures are performed at each timing. Aside from this point and the points described below, the configuration is substantially the same as Embodiment 1, so the explanation is omitted.

[0077] In this embodiment, within a single subframe, a first exposure period is repeated multiple times, starting after a first period has elapsed since the emission of light from the light-emitting element. A second exposure period is repeated multiple times, starting after a second exposure period has elapsed since the emission of light from the light-emitting element. The integrated signal amount is then calculated using the signals obtained from the multiple first exposure periods and the signals obtained from the multiple second exposure periods.

[0078] In this embodiment, the number of times the first exposure period is repeated is the same as the number of times the second exposure period is repeated. For example, the first exposure period is repeated 10 times, and the second exposure period is repeated 10 times. However, this is not limited to this, and the number of times the first exposure period is repeated and the number of times the second exposure period is repeated may be different. In this case, when measuring the distance to the object based on the histogram, the difference in the number of repetitions should be taken into consideration. In this embodiment, as shown in Figure 11(b), the effective gate profile is in the shape of an inverted T. That is, the effective gate profile is convex.

[0079] According to this embodiment, similar to Embodiment 1, it is possible to provide a signal generation device that ensures appropriate distance resolution without increasing the distance measurement time. Furthermore, a simpler pulse generator can be used compared to Embodiment 1. Therefore, it is possible to reduce the cost of the distance information generation device.

[0080] Furthermore, in this embodiment, as a modification, the lengths of the first exposure period and the second exposure period may be changed. Figure 12(a) shows gate profile 1 during the first exposure period, and Figure 12(b) shows gate profile 2 during the second exposure period. In the modification, the first exposure period is longer than the second exposure period.

[0081] Figure 12(c) shows the effective gate profile obtained by the combined product (convolution) of gate profile 1, gate profile 2, and residence period distribution. In this way, even when integrating gate profile 1 and gate profile 2, which have different exposure period widths and start timings, multiple times, the same effective gate profile as in Figure 11(b) can be obtained. Therefore, even in the modified example, a signal generation device that ensures appropriate distance resolution can be provided without increasing the distance measurement time.

[0082] (Embodiment 3) This embodiment will be described with reference to Figure 13. Figure 13 is a graph showing the gate drive and effective gate profile per unit of time in the drive according to this embodiment. Figure 13(a) shows the gate profile and the distribution of residence time in each subframe, and Figure 13(b) shows the effective gate profile.

[0083] In this embodiment, the multiple exposure periods in the first subframe and the multiple exposure periods in the second subframe are staggered. Aside from this point and the points described below, this embodiment is substantially the same as Embodiment 1, and therefore the explanation is omitted.

[0084] In this embodiment, the end timing of multiple exposure periods in the first subframe is controlled to be the same as or later than the start timing of multiple exposure periods in the second subframe. Furthermore, the timing difference between subframes is set to be the same as or smaller than the width of the gate profile. As a result, the distance measurement range of interest can be covered with fewer subframes compared to Embodiment 1. Therefore, the distance measurement time can be shortened compared to Embodiment 1. In particular, by making the timing difference approximately equal to the width of the gate profile, the increase / decrease relationship of the effective gate profile in the immediately preceding and succeeding subframes can be reversed. More specifically, as shown in Figure 13(b), in the region where the slope of the effective gate profile is negative in the first subframe, the slope of the effective gate profile can be set to a positive region in the second subframe.

[0085] In this embodiment, similar to Embodiment 1, a signal generation device that ensures appropriate distance resolution without increasing the distance measurement time can be provided. Furthermore, by acquiring data in multiple subframes while maintaining a relationship inverse to the increase / decrease relationship of the effective gate profile, it is possible to obtain high resolution regardless of the distance to the object.

[0086] (Embodiment 4) This embodiment will be described with reference to Figure 14. Figure 14 is a graph showing the gate drive and effective gate profile per unit of time in the drive according to this embodiment. Figure 14(a) shows the gate profile and the distribution of residence time in each subframe, and Figure 14(b) shows the effective gate profile.

[0087] In this embodiment, the shift amount of the exposure period start timing is gradually decreased over time within a single subframe, and then gradually increased over time. Aside from this point and the points described below, this embodiment is substantially the same as Embodiment 1, and therefore the explanation is omitted.

[0088] In this embodiment, the dwell time distribution in each subframe is not a rectangle, but a symmetrical triangle. For example, the first and third time differences are the same, and the second time difference is smaller than the first time difference. Therefore, the effective gate profile has a steeper slope compared to Embodiment 1, as shown in Figure 14(b). Consequently, by ensuring that the light reflected from the object spans the preceding or succeeding subframe, the distance measurement accuracy can be improved compared to Embodiment 1.

[0089] In this embodiment, similar to Embodiment 1, it is possible to provide a signal generation device that ensures appropriate distance resolution without increasing the distance measurement time. Furthermore, it is possible to improve the distance measurement accuracy compared to Embodiment 1.

[0090] Furthermore, in this embodiment, as a modification 1, the residence period distribution in each subframe may be an asymmetric triangle, as shown in Figure 15(a). For example, the amount of shift in the start timing of the exposure period may be gradually reduced over time and then integrated. In other words, in a single subframe, the overlap of multiple exposure periods may increase towards the latter half of the subframe. The third period difference is smaller than the second period difference, and the second period difference is smaller than the first period difference. In this case as well, as shown in Figure 15(b), the slope of the effective gate profile becomes steeper compared to Embodiment 1. Therefore, in Modification 1, it is possible to further improve the distance measurement accuracy. Alternatively, instead of reducing the amount of shift in the start timing of the exposure period over time, a similar residence period distribution may be achieved by changing the number of times integration is performed for each start timing of the exposure period while keeping the shift amount constant. In this case, the number of integrations for the first exposure period and the number of integrations for the second exposure period are different. For example, the number of integrations for the second exposure period can be greater than the number of integrations for the first exposure period.

[0091] Furthermore, in this embodiment, as a modification 2, the residence period distribution in each subframe may be M-shaped, as shown in Figure 16(a). For example, in one subframe, the amount of shift in the start timing of the exposure period may be gradually increased as time progresses, and then gradually decreased as time progresses from the middle of the subframe. In other words, in one subframe, there may be a greater overlap of multiple exposure periods in the first half and second half of the subframe, and a smaller overlap of multiple exposure periods near the center of the subframe. In modification 2 as well, as shown in Figure 15(b), the slope of the effective gate profile becomes steeper compared to embodiment 1. Therefore, in modification 2, it is possible to further improve the distance measurement accuracy.

[0092] Furthermore, in this embodiment, as a third modification, the residence period distribution in each subframe may be parabolic, as shown in Figure 17(a). For example, in one subframe, the overlap of multiple exposure periods may be increased near the center of the subframe, and the overlap of multiple exposure periods may be decreased in the first half and second half of the subframe. For example, the second period difference may be smaller than the first period difference, and the third period difference may be smaller than the second period difference. In this case as well, as shown in Figure 17(b), the slope of the effective gate profile becomes steeper compared to Embodiment 1. Therefore, in the third modification, it is possible to further improve the distance measurement accuracy.

[0093] (Embodiment 5) This embodiment will be described with reference to Figure 18. Figure 18 is a graph showing the gate drive per unit of time in the drive according to this embodiment. Figure 18(a) shows the gate profile and the distribution of residence time in each subframe, and Figure 18(b) shows the effective gate profile.

[0094] In this embodiment, the counter included in the photoelectric conversion element is an up / down counter, which is different from Embodiment 1. Aside from this point and the points described below, it is substantially the same as Embodiment 1, and therefore the explanation is omitted.

[0095] In this embodiment, an up / down counter is used as the counter. This makes it possible to switch between counting up and counting down within a single subframe. Therefore, as shown in Figure 18(a), it becomes possible to effectively take negative values ​​in the dwell time distribution. Negative values ​​can also be realized in the convolved effective gate profile. Therefore, high-precision distance measurement becomes possible while canceling the effects of ambient light.

[0096] In this embodiment, similar to Embodiment 1, a signal generation device that ensures appropriate distance resolution without increasing the distance measurement time can be provided. Furthermore, the influence of ambient light can be reduced compared to Embodiment 1.

[0097] Furthermore, in this embodiment, as a modification, the residence period distribution in each subframe may be a triangle instead of a rectangle, as shown in Figure 19(a). For example, in one subframe, the shift amount of the exposure period start timing may be gradually decreased over time, and then the shift amount of the exposure period start timing may be gradually increased over time. In this case, as shown in Figure 19(b), the effective gate profile can take a shape close to a sine wave. Therefore, when processing using principles such as Fourier transform, the subsequent calculation processing can be simplified.

[0098] (Embodiment 6) The signal generation system according to this embodiment will be described with reference to Figure 20. Figure 20 is a block diagram showing the schematic configuration of the signal generation system according to this embodiment.

[0099] The signal generation device (photoelectric converter) described in the above embodiment is applicable to various signal generation systems. Examples of applicable signal generation systems (photoelectric converter systems) include digital still cameras, digital camcorders, surveillance cameras, photocopiers, fax machines, mobile phones, in-vehicle cameras, and observation satellites. Camera modules, which include optical systems such as lenses and imaging devices, are also included in signal generation systems. Figure 20 shows a block diagram of a digital still camera as an example of these.

[0100] The signal generation system illustrated in Figure 20 includes an imaging device 1004, which is an example of a signal generation device, and a lens 1002 that forms an optical image of a subject onto the imaging device 1004. Furthermore, it has an aperture 1003 for varying the amount of light passing through the lens 1002, and a barrier 1001 for protecting the lens 1002. The lens 1002 and aperture 1003 are an optical system that focuses light onto the imaging device 1004. The imaging device 1004 is a signal generation device (imaging device) according to any of the above embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0101] The signal generation system also includes a signal processing unit 1007, which is an image generation unit that generates an image by processing the output signal output from the imaging device 1004. The signal processing unit 1007 performs various corrections and compressions as needed and outputs the image data. The signal processing unit 1007 may be formed on the semiconductor substrate on which the imaging device 1004 is mounted, or it may be formed on a semiconductor substrate separate from the imaging device 1004. Alternatively, the imaging device 1004 and the signal processing unit 1007 may be formed on the same semiconductor substrate.

[0102] The signal generation system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. Furthermore, the signal generation system includes a recording medium 1012 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading data from the recording medium 1012. The recording medium 1012 may be built into the signal generation system or may be detachable.

[0103] Furthermore, the signal generation system includes an overall control / calculation unit 1009 that controls various calculations and the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the imaging device 1004 and the signal processing unit 1007. Here, the timing signals and the like may be input from an external source, and the signal generation system only needs to include at least the imaging device 1004 and the signal processing unit 1007 that processes the output signals output from the imaging device 1004.

[0104] The imaging device 1004 outputs the imaging signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004 and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0105] Thus, according to this embodiment, a signal generation system can be realized by applying a signal generation device (imaging device) of any of the above embodiments.

[0106] (Embodiment 7) The signal generation system and mobile unit of this embodiment will be described with reference to Figure 21. Figure 21 is a diagram showing the configuration of the signal generation system and mobile unit of this embodiment.

[0107] Figure 21(a) shows an example of a signal generation system related to an in-vehicle camera. The signal generation system 1300 has a signal generation device 1310. The signal generation device 1310 is the signal generation device described in any of the embodiments described above. The signal generation system 1300 has an image processing unit 1312 that performs image processing on a plurality of image data acquired by the signal generation device 1310. The signal generation system 1300 also has a distance acquisition unit 1316 that calculates the distance to an object, and a collision determination unit 1318 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the distance acquisition unit 1316 may acquire Time of Flight (Tof) distance information to the object, or it may acquire distance information using parallax information, etc. That is, distance information is information related to parallax, defocus amount, distance to the object, etc. The collision determination unit 1318 may use any of this distance information to determine the possibility of collision. The distance information acquisition means may be implemented by specially designed hardware or by a software module. Furthermore, it may be implemented using FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits), or a combination thereof.

[0108] The signal generation system 1300 is connected to the vehicle information acquisition device 1320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The signal generation system 1300 is also connected to the control ECU 1330, which is a control device that outputs a control signal to generate braking force on the vehicle based on the judgment result of the collision judgment unit 1318. The signal generation system 1300 is also connected to the warning device 1340, which issues a warning to the driver based on the judgment result of the collision judgment unit 1318. For example, if the collision judgment unit 1318 determines that there is a high probability of collision, the control ECU 1330 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 1340 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seat belt or steering wheel.

[0109] In this embodiment, the signal generation system 1300 images the area around the vehicle, for example, in front of or behind it. Figure 21(b) shows the signal generation system when imaging the area in front of the vehicle (image range 1350). The vehicle information acquisition device 1320 sends instructions to the signal generation system 1300 or the signal generation device 1310. This configuration can further improve the accuracy of distance measurement.

[0110] The above example illustrates control to prevent collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles or automatically prevent vehicles from straying from their lanes. Furthermore, the signal generation system can be applied not only to vehicles such as automobiles, but also to mobile bodies (mobile devices) such as ships, aircraft, or industrial robots. This mobile body mainly includes a drive force generation unit that generates the driving force used for the movement of the mobile body, and one or both of the rotating body mainly used for the movement of the mobile body. The drive force generation unit can be an engine, motor, etc. The rotating body can be a tire, wheel, ship's propeller, aircraft propeller, etc. In addition, it can be applied not only to mobile bodies, but also to a wide range of devices that utilize object recognition, such as intelligent transportation systems (ITS).

[0111] (Embodiment 8) The signal generation system of this embodiment will be described using Figures 23(a) and (b). Figure 23(a) illustrates the eyeglasses 1600 (smart glasses), which are the signal generation system of this embodiment. The eyeglasses 1600 have a signal generation device 1602. The signal generation device 1602 is the signal generation device described in each of the embodiments described above. In addition, a display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. There may be one signal generation device 1602 or multiple signal generation devices. In addition, multiple types of signal generation devices may be used in combination. The arrangement position of the signal generation device 1602 is not limited to that shown in Figure 23(a).

[0112] The eyeglasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the signal generator 1602 and the display device. The control device 1603 also controls the operation of the signal generator 1602 and the display device. The lens 1601 has an optical system formed therein for focusing light onto the signal generator 1602.

[0113] Figure 23(b) illustrates a pair of glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, which is equipped with a signal generator corresponding to a signal generator 1602 and a display device. The lens 1611 has an optical system formed to project the light emitted from the signal generator in the control device 1612 and the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power supply to supply power to the signal generator and the display device, and also controls the operation of the signal generator and the display device. The control device may have a gaze detection unit that detects the wearer's gaze. Gaze detection may use infrared light. The infrared light emitter emits infrared light towards the eyeball of the user who is fixating on the displayed image. An imaging unit having a photodetector detects the reflected light from the eyeball of the emitted infrared light, thereby obtaining an image of the eyeball. By having a reduction means that reduces the light from the infrared light emitter to the display unit in planar view, the deterioration of image quality is reduced.

[0114] The user's gaze towards the displayed image is detected from an image of the eyeball obtained by imaging with infrared light. Any known method can be applied to gaze detection using an image of the eyeball. For example, a gaze detection method based on the Purkinje image obtained by the reflection of the irradiated light from the cornea can be used.

[0115] More specifically, gaze detection processing is performed based on the pupil-corneal reflection method. Using the pupil-corneal reflection method, a gaze vector representing the orientation (rotation angle) of the eyeball is calculated based on the pupil image and Purkinje image contained in the captured image of the eyeball, thereby detecting the user's gaze.

[0116] The display device of this embodiment may have a signal generating device having a light-receiving element, and may control the display image of the display device based on the user's gaze information from the signal generating device.

[0117] Specifically, the display device determines a first field of view that the user is fixated on, and a second field of view other than the first field of view, based on gaze information. The first and second field of view may be determined by the control device of the display device, or they may be determined by an external control device and received. Within the display area of ​​the display device, the display resolution of the first field of view may be controlled to be higher than that of the second field of view. In other words, the resolution of the second field of view may be lower than that of the first field of view.

[0118] Furthermore, the display area may have a first display area and a second display area different from the first display area, and a higher priority area may be determined from the first and second display areas based on line-of-sight information. The first and second field-of-sight areas may be determined by the control device of the display device, or they may be determined by an external control device and received. The resolution of the higher priority area may be controlled to be higher than the resolution of the areas other than the higher priority area. In other words, the resolution of areas with relatively lower priority may be set lower.

[0119] AI may be used to determine the first field of view area and the areas with higher priority. The AI ​​may be a model configured to estimate the angle of gaze and the distance to the target object at the end of the line of sight from the image of the eye, using the image of the eye and the direction the eye was actually looking in that image as training data. The AI ​​program may be installed in the display device, the signal generator, or an external device. If installed in an external device, it will be transmitted to the display device via communication.

[0120] When display control is based on visual detection, this method is preferably applicable to smart glasses that further include a signal generation device for capturing external images. The smart glasses can display the captured external information in real time.

[0121] (Embodiment 9) The signal generation device and signal generation system described above may be applied to electronic devices such as smartphones and tablets.

[0122] Figures 23(a) and 23(b) show an example of an electronic device 1500 equipped with a signal generator. Figure 23(a) shows the front side of the electronic device 1500, and Figure 23(b) shows the back side of the electronic device 1500.

[0123] As shown in Figure 23(a), a display 1510 for displaying images is positioned in the center of the surface of the electronic device 1500. Along the top edge of the surface of the electronic device 1500, front cameras 1521 and 1522, which use signal generating devices, an IR light source 1530 that emits infrared light, and a visible light source 1540 that emits visible light are positioned.

[0124] Furthermore, as shown in Figure 23(b), rear cameras 1551 and 1552, which use signal generating devices, an IR light source 1560 that emits infrared light, and a visible light source 1570 that emits visible light are arranged along the upper edge of the back of the electronic device 1500.

[0125] In the electronic device 1500 configured in this way, by applying the signal generation device described above, it is possible to capture higher-quality images that take into account, for example, the distance to the object. The signal generation device can also be applied to other electronic devices such as infrared sensors, distance measuring sensors using active infrared light sources, security cameras, and personal or biometric authentication cameras. This can improve the accuracy and performance of these electronic devices.

[0126] In this specification, the expressions "A or B" and "at least one of A and B" can include all possible combinations of the enumerated items unless otherwise explicitly defined. Similarly, expressions such as "at least one of A and / or B" and "one or more of A and / or B" can also include all possible combinations of the enumerated items unless otherwise explicitly defined. That is, the above expressions are understood to disclose all cases: including at least one A, including at least one B, and including both at least one A and at least one B. This applies equally to combinations of three or more elements.

[0127] The embodiments described above can be modified as appropriate without departing from the technical concept. Furthermore, the disclosures in this specification include not only what is described herein, but also all matters that can be understood from this specification and the drawings attached thereto. The disclosures in this specification also include the complement of the concepts described herein. That is, if this specification states, for example, "A is greater than B," then even if the statement "A is not greater than B" is omitted, this specification can be said to disclose that "A is not greater than B." This is because the statement "A is greater than B" presupposes that the case where "A is not greater than B" is being considered.

[0128] This embodiment includes the following configurations and methods.

[0129] (Composition 1) It has multiple photoelectric conversion elements, This signal generation device generates multiple subframes using signals from the multiple photoelectric conversion elements, and generates a distance measurement frame using the signals from the multiple subframes. The plurality of subframes each include at least a first subframe whose exposure period begins after a first period following the emission of light from the light-emitting element, and a second subframe whose exposure period begins after a second period longer than the first period following the emission of light from the light-emitting element. The first subframe has at least a first exposure period and a second exposure period, A signal generation device characterized in that the timing for starting the first exposure period and the timing for starting the second exposure period are different.

[0130] (Configuration 2) The signal generation device according to configuration 1, characterized in that the period from the emission of light from the light-emitting element to the start of the second exposure period is shorter than the period from the emission of light from the light-emitting element to the end of the first exposure period.

[0131] (Composition 3) The aforementioned multiple subframes are generated by the first frame generation unit, The signal generation device according to configuration 1 or 2, characterized in that the distance measurement frame is generated by a second frame generation unit.

[0132] (Composition 4) The first subframe has a plurality of exposure periods, including the first exposure period and the second exposure period. The signal generation apparatus according to any one of configurations 1 to 3, characterized in that, in the first subframe, the timing for starting the plurality of exposure periods is shifted at a constant velocity.

[0133] (Composition 5) A signal generation apparatus according to any one of configurations 1 to 3, characterized in that the first exposure period and the second exposure period are repeated multiple times in the first subframe.

[0134] (Composition 6) The signal generation device according to configuration 5, characterized in that it calculates the total signal amount by integrating the light intensity values ​​obtained in multiple first exposure periods and the light intensity values ​​obtained in multiple second exposure periods.

[0135] (Composition 7) The signal generation apparatus according to any one of configurations 1 to 6, characterized in that the period from the emission of light from the light-emitting element in the first subframe to the end of the exposure period is longer than the period from the emission of light from the light-emitting element in the second subframe to the start of the exposure period.

[0136] (Composition 8) The signal generation apparatus according to any one of configurations 1 to 7, characterized in that the second exposure period is started simultaneously in at least two of the plurality of photoelectric conversion elements.

[0137] (Composition 9) The first subframe has a third exposure period and a fourth exposure period, The timing of the start of the first exposure period is a first time difference between the timing of the start of the second exposure period and the timing of the start of the second exposure period, the timing of the start of the second exposure period is a second time difference between the timing of the start of the third exposure period and the timing of the start of the third exposure period and the timing of the start of the fourth exposure period. A signal generating device according to any one of configurations 1 to 8, characterized in that the first period difference and the third period difference are the same, and the second period difference is smaller than the first period difference.

[0138] (Composition 10) The first subframe has a third exposure period and a fourth exposure period, The timing of the start of the first exposure period is a first time difference between the timing of the start of the second exposure period and the timing of the start of the second exposure period, the timing of the start of the second exposure period is a second time difference between the timing of the start of the third exposure period and the timing of the start of the third exposure period and the timing of the start of the fourth exposure period. A signal generating device according to any one of configurations 1 to 8, characterized in that the first period difference is greater than the second period difference, and the second period difference is greater than the third period difference.

[0139] (Composition 11) The first subframe has a third exposure period and a fourth exposure period, The timing of the start of the first exposure period is a first time difference between the timing of the start of the second exposure period and the timing of the start of the second exposure period, the timing of the start of the second exposure period is a second time difference between the timing of the start of the third exposure period and the timing of the start of the third exposure period and the timing of the start of the fourth exposure period. A signal generating device according to any one of configurations 1 to 8, characterized in that the third period difference is smaller than the second period difference, and the second period difference is smaller than the first period difference.

[0140] (Composition 12) The photoelectric conversion element comprises an avalanche photodiode and a gate circuit that controls whether or not to output a signal from the avalanche photodiode. The signal generating apparatus according to any one of configurations 1 to 11, characterized in that the first exposure period and the second exposure period are set by controlling the supply of a signal to the gate circuit.

[0141] (Composition 13) The aforementioned gate circuit is an AND gate, The signal generation device according to configuration 12, characterized in that the AND circuit receives the signal from the avalanche photodiode and the signal from the drive line.

[0142] (Composition 14) The signal generation device according to configuration 12 or 13, characterized in that the photoelectric conversion element has a counter.

[0143] (Composition 15) The signal generating device according to configuration 14, characterized in that the counter is an up / down counter.

[0144] (Composition 16) Multiple first exposure periods are accumulated, and multiple second exposure periods are accumulated. A signal generating apparatus according to any one of configurations 1 to 15, characterized in that the number of cumulative exposure periods for the first exposure period and the number of cumulative exposure periods for the second exposure period are different.

[0145] (Composition 17) The signal generation apparatus according to configuration 16, characterized in that the number of cumulative exposure periods for the second exposure period is greater than the number of cumulative exposure periods for the first exposure period.

[0146] (Composition 18) A light-emitting device including the aforementioned light-emitting element; a signal-generating device according to any one of configurations 1 to 17; and a signal-generating system that detects reflected light irradiated from the light-emitting device and reflected from an object using the signal-generating device. [Explanation of Symbols]

[0147] 101 Photoelectric conversion element P1 First exposure period P2 Second exposure period

Claims

1. It has multiple photoelectric conversion elements, This signal generation device generates multiple subframes using signals from the multiple photoelectric conversion elements, and generates a distance measurement frame using the signals from the multiple subframes. The plurality of subframes each include at least a first subframe whose exposure period begins after a first period following a first emission from the light-emitting element, and a second subframe whose exposure period begins after a second period longer than the first period following a second emission from the light-emitting element. The first subframe has at least a first exposure period and a second exposure period, The period from the first emission of light from the light-emitting element to the start of the first exposure period is different from the period from the third emission of light from the light-emitting element to the start of the second exposure period. The period from the third emission from the light-emitting element to the start of the second exposure period is shorter than the period from the first emission from the light-emitting element to the end of the first exposure period. A signal generation device characterized in that the period from the first emission of light from the light-emitting element in the first subframe to the end of the exposure period is longer than the period from the second emission of light from the light-emitting element in the second subframe to the start of the exposure period.

2. The aforementioned plurality of subframes are generated by the first frame generation unit, The signal generation device according to claim 1, characterized in that the distance measurement frame is generated by a second frame generation unit.

3. The first subframe has a plurality of exposure periods, including the first exposure period and the second exposure period. The signal generation apparatus according to claim 1, characterized in that, in the first subframe, the timing of starting the plurality of exposure periods is shifted at a constant velocity.

4. The signal generation apparatus according to claim 1, characterized in that in the first subframe, the first exposure period and the second exposure period are each repeated multiple times.

5. The signal generating device according to claim 4, characterized in that it counts an integrated signal amount obtained by integrating the light intensity values ​​obtained in multiple first exposure periods and the light intensity values ​​obtained in multiple second exposure periods.

6. The signal generation apparatus according to claim 1, characterized in that the second exposure period is started simultaneously in at least two of the plurality of photoelectric conversion elements.

7. The first subframe has a third exposure period and a fourth exposure period, The timing of the start of the first exposure period is a first time difference between the timing of the start of the second exposure period and the timing of the start of the second exposure period, the timing of the start of the second exposure period is a second time difference between the timing of the start of the third exposure period and the timing of the start of the third exposure period and the timing of the start of the fourth exposure period. The signal generating apparatus according to claim 1, characterized in that the first period difference and the third period difference are the same, and the second period difference is smaller than the first period difference.

8. The first subframe has a third exposure period and a fourth exposure period, The timing of the start of the first exposure period is a first time difference between the timing of the start of the second exposure period and the timing of the start of the second exposure period, the timing of the start of the second exposure period is a second time difference between the timing of the start of the third exposure period and the timing of the start of the third exposure period and the timing of the start of the fourth exposure period. The signal generating apparatus according to claim 1, characterized in that the third period difference is smaller than the second period difference, and the second period difference is smaller than the first period difference.

9. The first subframe has a third exposure period and a fourth exposure period, The timing of the start of the first exposure period is a first time difference between the timing of the start of the second exposure period and the timing of the start of the second exposure period, the timing of the start of the second exposure period is a second time difference between the timing of the start of the third exposure period and the timing of the start of the third exposure period and the timing of the start of the fourth exposure period. The signal generating apparatus according to claim 1, characterized in that the first period difference is greater than the second period difference, and the second period difference is smaller than the third period difference.

10. The photoelectric conversion element comprises an avalanche photodiode and a selection circuit that controls whether or not to output a signal from the avalanche photodiode. The signal generating apparatus according to claim 1, characterized in that the first exposure period and the second exposure period are set by controlling the supply of a signal to the selection circuit.

11. The signal generating device according to claim 10, characterized in that the selection circuit receives a signal from the avalanche photodiode and a signal from the drive line.

12. The signal generation device according to claim 10, characterized in that the photoelectric conversion element has a counter.

13. The signal generating device according to claim 12, characterized in that the counter is an up / down counter.

14. Multiple first exposure periods are accumulated, and multiple second exposure periods are accumulated. The signal generating apparatus according to claim 1, characterized in that the number of cumulative exposure periods for the first exposure period and the number of cumulative exposure periods for the second exposure period are different.

15. The signal generating apparatus according to claim 14, characterized in that the number of times the second exposure period is accumulated is greater than the number of times the first exposure period is accumulated.

16. It has multiple photoelectric conversion elements, This signal generation device generates multiple subframes using signals from the multiple photoelectric conversion elements, and generates a distance measurement frame using the signals from the multiple subframes. The plurality of subframes each include at least a first subframe whose exposure period begins after a first period following a first emission from the light-emitting element, and a second subframe whose exposure period begins after a second period longer than the first period following a second emission from the light-emitting element. The first subframe has at least a first exposure period and a second exposure period, The period from the first emission of light from the light-emitting element to the start of the first exposure period is different from the period from the third emission of light from the light-emitting element to the start of the second exposure period. The period from the first emission of light from the light-emitting element in the first subframe to the end of the exposure period is longer than the period from the second emission of light from the light-emitting element in the second subframe to the start of the exposure period. The signal generation device is characterized in that the photoelectric conversion element has an avalanche photodiode.

17. The photoelectric conversion element has a selection circuit that controls whether or not to output a signal from the avalanche photodiode. The signal generating apparatus according to claim 16, characterized in that the first exposure period and the second exposure period are set by controlling the signal supply to the selection circuit.

18. The first subframe has a plurality of exposure periods, including the first exposure period and the second exposure period. The signal generation apparatus according to claim 16, characterized in that, in the first subframe, the timing of starting the plurality of exposure periods is shifted at a constant velocity.

19. The signal generation apparatus according to claim 16, characterized in that the second exposure period is started simultaneously in at least two of the plurality of photoelectric conversion elements.

20. The signal generating device according to claim 17, characterized in that the selection circuit receives a signal from the avalanche photodiode and a signal from the drive line.

21. The signal generating device according to claim 16, characterized in that the photoelectric conversion element has a counter.

22. Multiple first exposure periods are accumulated, and multiple second exposure periods are accumulated. The signal generating apparatus according to claim 16, characterized in that the number of cumulative exposure periods for the first exposure period and the number of cumulative exposure periods for the second exposure period are different.

23. The signal generating apparatus according to claim 22, characterized in that the number of times the second exposure period is accumulated is greater than the number of times the first exposure period is accumulated.

24. The light-emitting device includes the light-emitting element, and the signal-generating device according to any one of claims 1 to 23, A signal generation system that detects reflected light emitted from the aforementioned light-emitting device and reflected from an object using a signal generation device.

Images