Distance measurement device and distance measurement method

Abstract

Provided is a distance measurement device (1) for measuring the distance to a target object using indirect TOF, wherein: during a light emission and exposure period, an exposure signal generation unit (112) outputs, to a pixel drive unit (12), a first exposure signal that drives a plurality of gates (33) to transfer a charge generated by a photoelectric conversion unit (31) to any of a plurality of charge storage units (32) during the period of a first exposure packet width; and during a background light adjustment period, the exposure signal generation unit (112) outputs, to the pixel drive unit (12), a second exposure signal that drives the plurality of gates (33) to transfer a charge generated by the photoelectric conversion unit (31) to any of the plurality of charge storage units (32) during the period of a second exposure packet width, which is different from the first exposure packet width.

Description

Distance measuring device and distance measuring method

[0001] The present disclosure relates to a distance measuring device and a distance measuring method.

[0002] A distance measuring device using the pulse TOF method, which is a type of indirect TOF (Time of Flight), can measure the distance to an object by exposing reflected light to multiple exposure pulses (exposure packets) with different timing and calculating the amount of delay in the reflected light from the signal ratio.

[0003] Japanese Patent Application Laid-Open No. 2003-124999 discloses a distance measuring device that has four gates per pixel that are driven at different timings, and acquires reflected light at four different timings.

[0004] Japanese Patent Application Laid-Open No. 2022-188478

[0005] The light incident on a distance measuring device includes light other than reflected light (hereafter referred to as background light). In order to accurately measure the distance to an object, it is necessary to correctly subtract the signal charge due to the background light contained in the incident light that is exposed to the distance measuring device at different times.

[0006] In the technology disclosed in Patent Document 1, in order to correctly subtract the signal charge due to background light, an error is corrected for each timing in the amount of signal charge due to background light contained in incident light that is exposed to the distance measuring device at four different timings. For this correction, a correction coefficient is set for each gate, and information indicating the obtained amount of charge is multiplied by the correction coefficient. However, this requires additional calculation processing after reading out the charge, which results in problems such as increased power consumption.

[0007] An object of the present disclosure is to provide a distance measuring device, a distance measuring method, and a program that can measure the distance to an object more accurately without requiring additional calculation processing.

[0008] One aspect of a distance measuring device according to the present disclosure is a distance measuring device that measures a distance to an object using an indirect TOF (Time of Flight) method, and includes a light source, an emission signal generation unit that generates an emission signal that causes the light source to project light, a pixel array unit having pixels that receive incident light, an exposure signal generation unit that generates an exposure signal that instructs the pixel array unit to perform exposure, a pixel drive unit that drives the pixel array unit based on the exposure signal, and a signal processing unit that derives a distance to the object based on information acquired from the pixel array unit, wherein the pixels each include a photoelectric conversion unit that generates charges from the received incident light, a plurality of charge accumulation units that accumulate the charges, and a signal processing unit that transfers the charges generated in the photoelectric conversion unit during an exposure packet width period based on the exposure signal to any one of the plurality of charge accumulation units. the exposure signal generation unit outputs a first exposure signal to the pixel driving unit to drive the plurality of gates so as to transfer the charge generated in the photoelectric conversion unit during a period of a first exposure packet width to one of the plurality of charge accumulation units, during a light emission exposure period, and outputs a second exposure signal to the pixel driving unit to drive the plurality of gates so as to transfer the charge generated in the photoelectric conversion unit during a period of a second exposure packet width different from the first exposure packet width to one of the plurality of charge accumulation units, during a background light adjustment period, and the pixel array unit outputs information based on the amount of charge accumulated in each of the plurality of charge accumulation units during the light emission exposure period and the background light adjustment period.

[0009] Furthermore, one aspect of the distance measurement method according to the present disclosure is an indirect Time of Flight (TOF) method. a light emitting exposure step for driving the plurality of gates to transfer the charge generated in the photoelectric conversion unit to one of the plurality of charge accumulation units during a period of a first exposure packet width; a background light adjustment step for driving the plurality of gates to transfer the charge generated in the photoelectric conversion unit to one of the plurality of charge accumulation units during a period of a second exposure packet width different from the first exposure packet width; and a signal processing step for deriving the distance to the object based on information based on the amount of charge accumulated in each of the plurality of charge accumulation units during the light emitting exposure step and the background light adjustment step, which is obtained from the pixel array unit.

[0010] The present disclosure can be realized not only as the distance measurement method, but also as a program for causing a computer to execute the distance measurement method, and further as a computer-readable recording medium storing the program.

[0011] The distance measuring device, distance measuring method, and program according to the present disclosure can measure the distance to an object more accurately without requiring additional calculation processing.

[0012] FIG. 1 is a block diagram illustrating a functional configuration of a distance measuring device according to an embodiment. FIG. 2 is a schematic diagram of a pixel array unit according to an embodiment. FIG. 3 is a diagram illustrating an example of a pixel configuration according to an embodiment. FIG. 4 is a circuit diagram of a portion of a distance measuring device according to an embodiment. FIG. 5 is a graph illustrating a relationship between a distance measurement distance and a distance measurement error for each background light amount error. FIG. 6 is a flowchart illustrating an operation of a distance measuring device according to an embodiment. FIG. 7 is a flowchart illustrating an operation of determining an exposure time during a background light adjustment period according to an embodiment. FIG. 8A is a flowchart illustrating another operation of determining an exposure time during a background light adjustment period according to an embodiment. FIG. 8B is a diagram illustrating an example of data used in another operation of determining an exposure time during a background light adjustment period according to an embodiment. FIG. 9 is a diagram illustrating an operation of a light emission exposure period of a distance measuring device according to Example 1. FIG. 10 is a diagram illustrating an operation of a background light adjustment period of a distance measuring device according to Example 1. FIG. 11 is a diagram illustrating an operation of a light emission exposure period of a distance measuring device according to Example 2. FIG. 12 is a diagram illustrating an operation of a background light adjustment period of a distance measuring device according to Example 2. FIG. 13 is a diagram illustrating an operation of a light emission exposure period of a distance measuring device according to Example 3. Fig. 14 is a diagram explaining the operation of the distance measuring device of Example 4 during a light emission exposure period. Fig. 15 is a diagram explaining the operation of the distance measuring device of Example 4 during a background light adjustment period. Fig. 16 is a diagram explaining the operation of the distance measuring device of Example 5 during a light emission exposure period. Fig. 17 is a diagram explaining the operation of the distance measuring device of Example 6 during a light emission exposure period. Fig. 18 is a diagram explaining the operation of the distance measuring device of Example 7 during a light emission exposure period. Fig. 19 is a diagram explaining the operation of the distance measuring device of Example 7 during a background light adjustment period.

[0013] Hereinafter, embodiments of a distance measuring device, a distance measuring method, and a program according to the present disclosure will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a preferred specific example of the present disclosure. The numerical values, components, the arrangement and connection of the components, steps, the order of steps, and the like shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components constituting a preferred embodiment.

[0014] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0015] Furthermore, in this specification, terms indicating the relationship between elements, such as perpendicular, parallel, or coincident, terms indicating the shape of elements, such as circular or rectangular, and numerical ranges are not expressions that only express a strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about a few percent.

[0016] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" in this specification do not refer to the number or order of components, etc., but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0017] (Embodiment) [Configuration] First, the configuration of a distance measuring device according to this embodiment will be described. Fig. 1 is a block diagram showing the functional configuration of a distance measuring device 1 according to this embodiment.

[0018] The distance measuring device 1 is a distance measuring device that can measure the distance to an object using indirect TOF. The distance measuring device 1 generates, for example, a distance image that indicates the distance to a subject, which is an example of the object. Specifically, the distance measuring device 1 includes an image sensor 10, a light source 20, and an information processing device 40.

[0019] The imaging element 10 is, for example, a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor, etc. Specifically, the imaging element 10 includes a drive signal generation unit 11, a pixel drive unit 12, a pixel array unit 13, an AD conversion unit 14, and a signal processing unit 15.

[0020] The drive signal generation unit 11 generates a signal for driving the distance measuring device 1 based on information acquired from the information processing device 40. The drive signal generation unit 11 generates at least one of a signal for causing the distance measuring device 1 to measure distance using the pulse TOF method and a signal for causing the distance measuring device 1 to measure distance using the CW-TOF method. The drive signal generation unit 11 is a processing circuit realized by a memory for storing a program and a processor for executing the program, etc. The drive signal generation unit 11 may also be a dedicated logic circuit for performing predetermined processing. Specifically, the drive signal generation unit 11 includes a light emission signal generation unit 111 and an exposure signal generation unit 112.

[0021] The light emission signal generation unit 111 generates and outputs a light emission control signal for controlling the light source 20. More specifically, the light emission signal generation unit 111 outputs, as the light emission control signal, a light emission control pulse that causes the light source 20 to irradiate projection light at a predetermined pulse width. The light emission control pulse includes a plurality of pulses that cause the light source 20 to repeatedly irradiate pulsed light. The light source 20 projects a plurality of pulsed lights as projection light at a timing according to the input light emission control pulse.

[0022] The exposure signal generation unit 112 generates and outputs an exposure control signal for controlling the pixel drive unit 12. More specifically, the exposure signal generation unit 112 outputs, as the exposure control signal, an exposure control pulse to the pixel drive unit 12, which causes the pixels 30 to expose at a predetermined exposure packet width. Each of the pixels 30 is exposed to light at a timing according to the exposure control pulse, and accumulates signal charge.

[0023] In this specification, the exposure period refers to a period during which signal charges used for reading out pixel signals are accumulated. Therefore, even if light is incident on a pixel 30 and signal charges are generated, the period is considered non-exposure if the signal charges are not used for reading out pixel signals.

[0024] The pixel driving section 12 drives the pixel array section 13 , more specifically, the gate 33 of the pixel 30 , based on the exposure control signal input from the exposure signal generating section 112 .

[0025] The pixel array unit 13 is a light receiving unit configured with a plurality of pixels 30 arranged two-dimensionally. Fig. 2 is a schematic diagram of the pixel array unit 13 according to the embodiment. As shown in Fig. 2, the pixel array unit 13 is configured with a plurality of pixels 30 arranged two-dimensionally. For the sake of explanation, Fig. 2 shows a configuration of 4 pixels horizontally and 4 pixels vertically, for a total of 16 pixels, but the number of pixels 30 included in the pixel array unit 13 is not particularly limited. The plurality of pixels 30 have, for example, the same configuration as each other.

[0026] Each pixel 30 generates a pixel signal based on incident light. Specifically, each pixel 30 converts the incident light into signal charges and generates a pixel signal based on the converted signal charges. Specifically, each pixel 30 includes a photoelectric conversion unit 31, a plurality of charge accumulation units 32, and a plurality of gates 33.

[0027] 3 is a diagram showing an example of the configuration of a pixel 30 according to an embodiment. As shown in Fig. 3, the pixel 30 includes, for example, a photoelectric conversion unit 31, a plurality of charge accumulation units 32a and 32b, a plurality of gates 33a and 33b, a charge discharge unit 34, and a discharge gate 35. The photoelectric conversion unit 31, the plurality of charge accumulation units 32a and 32b, the plurality of gates 33a and 33b, the charge discharge unit 34, and the discharge gate 35 are provided on, for example, a semiconductor substrate.

[0028] The photoelectric conversion unit 31 generates signal charges by converting incident light incident on the pixel 30 into signal charges. The incident light incident on the pixel 30 includes, for example, light projected from the light source 20 and reflected by an object. The incident light incident on the pixel 30 also includes light emitted from another light source, for example, the sun. The photoelectric conversion unit 31 is realized by, for example, a photoelectric conversion element such as a photodiode.

[0029] The charge accumulation unit 32 accumulates the signal charge converted by the photoelectric conversion unit 31. The charge accumulation unit 32 is, for example, an impurity region that functions as a source or drain of a FET (Field Effect Transistor) formed on a semiconductor substrate. The number of the charge accumulation units 32 included in each pixel 30 is two, but may be three or four.

[0030] The gate 33 is electrically connected to the photoelectric conversion unit 31 and transfers the signal charge converted by the photoelectric conversion unit 31 from the photoelectric conversion unit 31 to the charge accumulation unit 32. The multiple gates 33 are provided in one-to-one correspondence with the multiple charge accumulation units 32. In other words, the gate 33 distributes the signal charge generated by the photoelectric conversion unit 31 to the corresponding charge accumulation unit 32.

[0031] The charge discharging unit 34 discharges the signal charge converted by the photoelectric conversion unit 31. For example, a predetermined reset voltage is applied to the charge discharging unit 34. The reset voltage may be a ground voltage. The charge discharging unit 34 is, for example, an impurity region that functions as a source or a drain of the FET.

[0032] The discharge gate 35 is electrically connected to the photoelectric conversion unit 31, and transfers the signal charge converted by the photoelectric conversion unit 31 from the photoelectric conversion unit 31 to the charge discharge unit 34. The discharge gate 35 resets the charge in the photoelectric conversion unit 31 by causing the charge discharge unit 34 to discharge the signal charge.

[0033] That is, the signal charge generated by the photoelectric conversion unit 31 in response to light incident on the pixel 30 is transferred to the charge accumulation unit 32 by the gate 33 and accumulated in the charge accumulation unit 32. Alternatively, the signal charge is transferred to the charge discharge unit 34 by the discharge gate 35 and discharged from the pixel 30. In the pixel 30, the signal charge is processed in one of the two ways described above.

[0034] The AD conversion unit 14 is a signal detection circuit that reads out the signal charges accumulated in the charge accumulation units 32 as pixel signals. The AD conversion unit 14 reads out, for example, pixel signals corresponding to the potentials of the respective charge accumulation units 32. The pixel signals read out from the respective pixels 30 include signals indicating signal values ​​based on the amounts of signal charges accumulated in the respective charge accumulation units 32.

[0035] The signal processing unit 15 derives the distance to the object and generates a distance image based on the pixel signals acquired from the AD conversion unit 14. The signal processing unit 15 is a processing circuit realized by a memory that stores a program and a processor that executes the program, etc. The signal processing unit 15 may also be a dedicated logic circuit that performs predetermined processing.

[0036] The inspection data storage unit 16 stores data based on the results of inspecting the image sensor 10 in the state of a semiconductor wafer, and transmits information relating to manufacturing variations of the image sensor 10 to an information processing device 40, which will be described later.

[0037] The sensor 17 is a temperature sensor provided in the image sensor 10. In this embodiment, the sensor 17 transmits the temperature of the image sensor 10 to the information processing device 40. The light source 20 may also be provided with a temperature sensor so that the temperature of the light source 20 is transmitted to the information processing device 40.

[0038] The light source 20 projects light in accordance with a light emission control signal from the light emission signal generation unit 111. For example, the light source 20 projects pulsed light that repeats at a predetermined duty ratio in accordance with the timing indicated by a light emission control pulse included in the input light emission control signal. The light projected by the light source 20 is reflected by an object and received by the pixels 30. The light source 20 outputs light in a wavelength range to which the pixels 30 are sensitive, such as infrared light. The light source 20 is realized by, for example, an element that has a relatively fast response speed and is capable of high-speed blinking, such as a light-emitting diode (LED) or a laser diode, and an optical system that receives light from the light-emitting element and controls the light distribution from the light-emitting element.

[0039] The information processing device 40 performs information processing to control the drive signal generation unit 11 based on information acquired from the sensor 17. The information processing device 40 also stores information for causing the drive signal generation unit 11 to generate signals, such as information related to the timing of light emission signals and exposure signals. The information processing device 40 also acquires the distance image generated by the signal processing unit 15 and performs various information processing. For example, the information processing device 40 performs processing to control the movement of the mobile object on which the distance measuring device 1 is mounted, based on the distance image generated by the signal processing unit 15. The information processing device 40 is realized by, for example, a microcomputer or a processor. Note that the distance measuring device 1 does not necessarily have to include the information processing device 40. For example, the information processing device 40 may be another device provided in the mobile object on which the distance measuring device 1 is mounted.

[0040] [Basic Operation] Figure 4 is a circuit diagram of a portion of a distance measuring device 1 according to an embodiment. Figure 4 shows a circuit diagram of a portion of the pixel driving unit 12 and pixel array unit 13 in a distance measuring device 1 in which each pixel 30 has two types of gates 33 called TG1 gates and TG2 gates, and a discharge gate 35 called a PRS gate. As shown in Figure 4, the TG1 gate, TG2 gate, and PRS gate provided in the distance measuring device 1 according to an embodiment are all driven by the same power supply and share the same power supply wiring (VDD wiring) and ground wiring (GNDP wiring). In other words, a voltage from the same power supply is applied to the two gates 33 provided in each pixel 30. This allows for a reduction in the number of components, thereby enabling a reduction in the size of the distance measuring device 1 and a reduction in manufacturing costs.

[0041] However, because a voltage from the same power supply is applied to the multiple gates 33, the resistance components and inductance components of the wiring are also shared among the multiple gates 33. Furthermore, the pixel drive unit 12, which is a circuit that drives each of the multiple gates 33, shares the power supply wiring (VDD wiring) and ground wiring (GND wiring) that supply power to the circuit. This can cause interference between signals that drive pixels via the resistance components and inductance components of the power supply wiring and GND wiring that supply power to the circuit, resulting in deviation from the intended timing.

[0042] If the timing at which the pixel driving unit 12 drives the gate 33 deviates from the intended timing, the period during which the photoelectric conversion unit 31 generates signal charges to be accumulated in the plurality of charge accumulation units 32 will differ among the charge accumulation units 32. Furthermore, the signal charges generated in the photoelectric conversion unit 31 include signal charges generated by light reflected from the object and signal charges generated by background light. However, if the timing at which the gate 33 is driven deviates, the amount of signal charge generated by background light will differ among the charge accumulation units 32.

[0043] Next, we will explain the degree of difference in the amount of signal charge generated by background light that the distance measuring device 1 according to this embodiment should achieve. Fig. 5 is a graph showing the relationship between the measured distance and the distance measuring error for each background light amount error. The background light amount error is the difference in the amount of signal charge generated by background light. The distance measuring error is the ratio of the difference between the actual distance to the object and the distance to the object measured by the distance measuring device 1 to the actual distance to the object. The horizontal axis of the graph in Fig. 5 represents the distance to the object, and the vertical axis represents the distance measuring error. Fig. 5 shows the distance measuring error when the background light amount error is 0.5%, 1%, and 1.5%.

[0044] Here, the background light illuminance is 10 klx, which corresponds to the illuminance outdoors on a cloudy day. It is generally known that the signal charge due to reflected light decreases in inverse proportion to the square of the distance to the target object.

[0045] When the distance is short, the signal charge due to the reflected light is sufficiently large compared to the signal charge due to the background light, so the influence of the background light amount subtraction error due to the exposure width deviation is small.

[0046] When the distance is about five times the closest distance, the signal charge due to the reflected light and the signal charge due to the background light are approximately the same in magnitude.

[0047] Furthermore, when the distance is about 10 times the closest distance, the signal charge due to background light is several times larger than the signal charge due to reflected light, and it is found that a distance measurement error of about 1% occurs.

[0048] Therefore, to keep the distance measurement error to about 1%, the difference in signal charge due to background light must be kept to about 1%. Furthermore, when the background light illuminance is 100 klx (equivalent to sunlight in the daytime), the ratio of signal charge due to background light to signal charge due to reflected light becomes larger, so the difference in signal charge due to background light must be kept to 0.1% or less.

[0049] The distance measuring device 1 of this embodiment can reduce the difference in signal charge due to background light by operating during the light emission exposure period and the background light adjustment period for correcting the difference in charge due to background light that occurs during the background light adjustment period before reading out the signal charge.

[0050] The exposure signal generation unit 112 outputs to the pixel driving unit 12 a first exposure signal and a second exposure signal such that the difference in the amount of charge generated from the background light accumulated in each of the plurality of charge accumulation units 32 during the light emission exposure period and the background light adjustment period is 1% or less of the difference in the amount of charge generated from the incident light accumulated in each of the plurality of charge accumulation units 32.

[0051] The operation of the distance measuring device 1 according to this embodiment will be described below with reference to a flowchart of FIG.

[0052] First, the distance measuring device 1 performs an operation during the light emission exposure period (S11). The light source 20 emits light based on the light emission control signal obtained from the light emission signal generation unit 111. The pixel driving unit 12 drives the gate 33 for a period of the first exposure packet width based on the exposure control signal obtained from the exposure signal generation unit 112, and transfers the signal charge generated by exposing the photoelectric conversion unit 31 to reflected light and background light to the charge accumulation unit 32. The exposure control signal generated by the exposure signal generation unit 112 during the light emission exposure period is an example of a first exposure signal.

[0053] When the light emission exposure period ends, the distance measuring device 1 performs the operation of the background light adjustment period (S12). The pixel driving unit 12 drives the gate 33 for a period of the second exposure packet width based on the exposure control signal obtained from the exposure signal generation unit 112, and transfers the signal charge generated by exposing the photoelectric conversion unit 31 to background light to the charge accumulation unit 32. The charge generated during the light emission exposure period and the charge generated during the background light adjustment period are accumulated and added together in the same charge accumulation unit 32. The exposure control signal generated by the exposure signal generation unit 112 during the background light adjustment period is an example of a second exposure signal. The second exposure packet width is a different period from the first exposure packet width. A method for determining the second exposure packet width will be described in detail later.

[0054] When the background light adjustment period ends, the distance measuring device 1 reads out the charge amount (S13). The AD conversion unit 14 reads out, as a pixel signal, information indicating the charge amount of the signal charge accumulated in the plurality of charge accumulation units 32 during the light emission exposure period and the background light adjustment period.

[0055] Finally, the distance measuring device 1 creates a distance image (S14). The signal processing unit 15 creates the distance image based on the plurality of pixel signals acquired from the AD conversion unit 14.

[0056] According to the above-described operation, the distance measuring device 1 performs exposure during the background light adjustment period so as to compensate for the difference between the period of the exposure packet in which reflected light is exposed and the period of the exposure packet in which only background light is exposed during the light emission exposure period. In other words, the difference in the amount of signal charge generated by background light for each exposure packet during the light emission exposure period is adjusted during the background light adjustment period before the signal charge is output from the pixel array unit 13, so that the amount of background light contained in each packet becomes equal. According to the distance measuring device 1, background light is correctly subtracted through simple processing without requiring additional calculation processing, and the distance to the target can be measured more accurately.

[0057] It may be desirable to simplify the configuration of the image sensor 10 by arranging the signal processing unit 15 in the information processing device 40 rather than in the image sensor 10. In this case, the image sensor 10 needs to transmit each signal charge to the information processing device 40. However, since the signal charge due to background light contained in each signal charge is equal, the image sensor 10 only needs to transmit information indicating the difference in the amount of charge accumulated in the charge accumulation units 32, rather than information indicating the amount of charge accumulated in each charge accumulation unit 32. This makes it possible to reduce the communication volume and communication speed by half.

[0058] [Method for Determining Exposure Time During Background Light Adjustment Period] A method for determining the exposure time during the background light adjustment period in the process of step S12 in Figure 6 will be described. As described above, during the light emission exposure period, the driving of the multiple gates 33 deviates from the target timing due to pulse interference. This causes a problem in that the amount of charge generated by background light among the charges accumulated in the multiple charge accumulation units 32 during the light emission exposure period differs for each charge accumulation unit 32. During the background light adjustment period, the charge accumulation units 32 are caused to accumulate charge generated by background light for a period of the second exposure packet width so as to cancel out the difference in charge generated by background light that differs for each charge accumulation unit 32 during the light emission exposure period.

[0059] The following describes the operation of the information processing device 40 to determine the exposure time during the background light adjustment period, i.e., the period of the second exposure packet width. Fig. 7 is a flowchart illustrating the operation to determine the exposure time during the background light adjustment period according to the embodiment.

[0060] First, the information processing device 40 sets the timing of the light emission exposure period (S21). The timing of the light emission exposure period is the timing at which the pixel driving unit 12 drives the multiple gates 33 during the light emission exposure period. The exposure signal generating unit 112 generates a first exposure signal based on the timing of the light emission exposure period determined by the information processing device 40.

[0061] Next, the information processing device 40 sets the exposure width of each signal during the background light adjustment period to be the same (S22). That is, during the background light adjustment period, the information processing device 40 drives the multiple gates 33 so that the same amount of signal charge due to background light is accumulated in each of the multiple charge accumulation units 32.

[0062] The information processing device 40 operates the image sensor 10 by receiving only background light (S23). The information processing device 40 controls the light source 20 not to emit light, for example, by controlling the light emission signal generation unit 111 not to generate a light emission control signal, thereby creating a state in which only background light is incident on the distance measuring device 1, and operates the image sensor 10 under such a state.

[0063] The information processing device 40 checks the output difference between the signals (S24), that is, checks the difference in the amount of signal charge accumulated in the plurality of charge accumulation units 32. Because this signal charge is generated only by background light, the difference in the amount of signal charge between the signals indicates the difference in exposure width between the signals.

[0064] Next, the information processing device 40 determines whether the output difference between the signals is within an allowable range (S25). The allowable range is, for example, 1% or less, as described above. The allowable range may also be 0.1% or less. The allowable range may be set by the user as appropriate depending on the operation environment of the distance measuring device 1.

[0065] If it is determined that the output difference between the signals is outside the allowable range (No in S25), the information processing device 40 adjusts the exposure width of each signal during the background light adjustment period (S26). The information processing device 40 adjusts the timing at which the second exposure signal instructs exposure, thereby adjusting the exposure width of each signal during the background light adjustment period. After the information processing device 40 adjusts the timing at which the second exposure signal instructs exposure, the process returns to step S23, and the information processing device 40 operates in a state where only background light is received.

[0066] If it is determined that the output difference between the signals is within the allowable range (Yes in S25), the information processing device 40 stores the timing at which the second exposure signal instructs exposure, and ends the adjustment of the second exposure signal (S27).

[0067] According to the above-described operation, it is possible to determine the length of the exposure time for adjusting the background light. This operation is performed during calibration when a user uses the distance measuring device 1. Note that this operation may also be performed in a factory where the distance measuring device 1 is manufactured.

[0068] Furthermore, the amount of exposure width deviation due to the multiple gates 33 during the light emission exposure period of the image sensor 10 is temperature dependent. Therefore, the exposure time during the background light adjustment period is further adjusted according to the temperature of the distance measuring device 1, more specifically, the temperature of the image sensor 10. Fig. 8A is a flowchart illustrating another operation for determining the exposure time during the background light adjustment period according to the embodiment. Fig. 8B is a diagram showing an example of data used in another operation for determining the exposure time during the background light adjustment period according to the embodiment.

[0069] The relationship between temperature and background light adjustment amount is measured in advance (S31). The method for determining the exposure time during the background light adjustment period shown in FIG. 7 is performed in advance at a plurality of temperatures to create the data shown in FIG. 8B. This data is stored in the information processing device 40. FIG. 8B shows the duration of the second exposure packet width for each exposure packet during the background light adjustment period at temperatures of -20°C, 0°C, 35°C, 60°C, and 85°C. The duration of the second exposure packet width is measured in nanoseconds.

[0070] First, the information processing device 40 acquires the temperature of the image sensor 10 (S32). The information processing device 40 acquires the temperature of the image sensor 10 measured by the sensor 17.

[0071] Next, the information processing device 40 calculates the background light adjustment amount depending on the temperature (S33). Based on the temperature of the image sensor 10 acquired from the sensor 17, the information processing device 40 refers to the data shown in Fig. 8B stored in the information processing device 40 and calculates the background light adjustment amount for the temperature of the image sensor 10.

[0072] Finally, the information processing device 40 corrects the exposure time during the background light adjustment period based on the calculated background light adjustment amount.

[0073] According to the above operation, when the amount of exposure width deviation due to the multiple gates 33 during the light emission exposure period of the image sensor 10 depends on the temperature of the image sensor 10, the exposure time for adjusting the background light can be determined.

[0074] The amount of exposure width deviation during the light emission exposure period may vary due to variations in the manufacturing process. Temperature dependency may also vary from device to device. While the exposure time correction using the above method could be performed for each individual rangefinder, adjusting for each temperature increases manufacturing costs.

[0075] Therefore, by determining the exposure time for the background light adjustment period shown in Figure 7 at two different temperatures during wafer inspection, the susceptibility to temperature change for each individual is measured and stored in the inspection data storage unit 16 within the image sensor 10. During actual use, temperature correction is performed by correcting the data shown in Figure 8B using the susceptibility to temperature change for each individual. Although the exposure width during wafer inspection and the exposure width during actual use do not necessarily match, by storing the susceptibility to temperature change rather than the adjustment amount itself, it is possible to correct the exposure time for adjusting the background light taking into account individual variations.

[0076] According to this operation, when there is individual variation in the exposure width deviation amount due to the plurality of gates 33 during the light emission exposure period of the image sensor 10, it is possible to determine the exposure time for adjusting the background light.

[0077] [1. Pulse TOF Method] There are two types of indirect TOF: a pulse drive method (pulse TOF method) and a continuous wave drive method (CW-TOF method). First, distance calculation in pulse TOF will be described.

[0078] The light emission signal generation unit 111 generates a light emission control signal and outputs it to the light source 20. The light source 20 projects a light emission pulse (i.e., irradiation light) based on the input light emission control signal. The light emission pulse is reflected by an object and received by the pixel 30 as a reflected pulse (i.e., reflected light).

[0079] At the same time, the exposure signal generation unit 112 generates an exposure control signal and outputs it to the pixel drive unit 12. The pixel drive unit 12 drives the gates 33 based on the input exposure control signal, thereby causing the pixels 30 to perform exposure during exposure periods with four timings.

[0080] Of the four exposure periods performed by pixel 30, the first exposure period, which is the earliest exposure period, is the same timing as the irradiation of the light emission pulse. The second exposure period is the timing immediately after the first exposure period. The third exposure period is the timing immediately after the second exposure period, and the fourth exposure period is the timing immediately after the third exposure period. The amount of signal charge received during the first exposure period is A0, the amount of signal charge received during the second exposure period is A1, the amount of signal charge received during the third exposure period is A2, and the amount of signal charge received during the fourth exposure period is A3.

[0081] For example, if the reflected light from the object is received over the first exposure period and the second exposure period, the amounts of signal charge will be A0 > A2 and A1 > A3. The amount of signal charge received in each exposure period includes not only the signal charge due to reflected light but also the signal charge due to background light such as sunlight or indoor lighting, but the signal charge due to background light can be excluded by subtracting A0 from A2 and A1 from A3.

[0082] If the pulse width is T, the delay Δt can be calculated as Δt = T × (A1 - A3) / (A0 - A2 + A1 - A3), and the distance D to the object can be calculated as D = Δt / 2 × c (c: speed of light). The denominator, A0 - A2 + A1 - A3, corresponds to the amount of signal charge generated by the light reflected from the object.

[0083] When reflected light is detected across A1 and A2, A2 > A0 and A1 > A3, and the delay Δt is calculated as Δt = T × (A1 - A3) / (A2 - A0 + A1 - A3). Similarly, when reflected light is detected across A2 and A3, A2 > A0 and A3 > A1, and the delay Δt is calculated as Δt = T × (A3 - A1) / (A2 - A0 + A3 - A1).

[0084] Furthermore, if reflected light is not detected in any of the exposure periods, if it is detected only in the first exposure period (A0), or if it is detected only in the fourth exposure period (A3), distance measurement is not possible.

[0085] Hereinafter, the operation of the distance measuring device 1 according to the embodiment when it operates using pulse TOF will be described using Examples 1 to 5 according to implementation forms.

[0086] [1-1. Example 1] In the distance measuring device 1 of Example 1, each of the multiple pixels 30 included in the pixel array unit 13 has two gates 33 and two charge storage units 32, and can store two signal charges in one frame of operation. The distance measuring device 1 measures the distance to the object based on the amount of four accumulated signal charges by performing two operations in the first and second frames. During the light emission exposure period, the distance measuring device 1 performs exposure for a total of four periods of the first exposure packet width, two in each frame. In other words, the distance measuring device 1 is set to have four periods for storing signal charges used to read out pixel signals.

[0087] The operation of the distance measuring device 1 of Example 1 will be described below. Fig. 9 is a diagram illustrating the operation of the distance measuring device of Example 1 during the light emission exposure period. Fig. 10 is a diagram illustrating the operation of the distance measuring device of Example 1 during the background light adjustment period. Fig. 10(a) is a diagram illustrating the background light adjustment period of the first frame of the distance measuring device of Example 1. Fig. 10(b) is a diagram illustrating the exposure control signal during the background light adjustment period of the first frame of the distance measuring device of Example 1.

[0088] Regarding the exposure control signal that controls the PRS gate (discharge gate 35), when the exposure control signal is High, the PRS gate is on, and the charges generated in the photoelectric conversion unit 31 are transferred to and discharged by the charge discharge unit 34. When the exposure control signal is Low, the PRS gate is off, and the charges generated in the photoelectric conversion unit 31 are accumulated without being discharged.

[0089] Regarding the exposure control signal that controls the TG1 gate (gate 33a) and the TG2 gate (gate 33b), when the exposure control signal is High, the TG1 gate and the TG2 gate are turned on, and after the other gates are turned off, the charges generated in the photoelectric conversion unit 31 are transferred to the corresponding charge accumulation units 32a and 32b, respectively. In FIG. 10(b), A0 is the amount of signal charge generated in the photoelectric conversion unit 31 from the time when the PRS gate is turned off to the time when the TG1 gate changes from on to off. A2 is the amount of signal charge generated in the photoelectric conversion unit 31 from the time when the TG1 gate is turned off to the time when the TG2 gate changes from on to off.

[0090] In the first frame of Figure 9, A0 is the amount of signal charge generated in the photoelectric conversion unit 31 from the time when the PRS gate is turned off for the first time to the time when the TG1 gate changes from on to off. A2 is the amount of signal charge generated in the photoelectric conversion unit 31 from the time when the PRS gate is turned off for the second time to the time when the TG2 gate changes from on to off. The exposure widths of A0 and A2 are both the period of the first exposure packet width. The period of the first exposure packet width is, for example, 11 nsec.

[0091] Because the reflected light from the object is received at a timing spanning A0 and A1, A0 represents the amount of signal charge generated by the reflected light and background light, and A2 represents the amount of signal charge generated by the background light. However, as mentioned above, pulse interference causes the exposure widths of A0 and A2 to differ. For example, the PRS gate has two LOW timings, but if the interval between them is narrow, they may interfere with each other, causing a delay in the second OFF timing. For example, the exposure width of A0 may be 11 nsec, while the exposure width of A2 may be 10.9 nsec.

[0092] In such a case, the amount of signal charge generated by the background light contained in A0 is different from the amount of signal charge in A2, so even if background light subtraction is performed on A0-A2, the amount of signal charge generated by the reflected light cannot be obtained.

[0093] 9, A1 represents the amount of signal charge generated by reflected light and background light, and A3 represents the amount of signal charge generated by background light. Here, the amount of signal charge generated by background light included in A1 differs from the amount of signal charge in A3, so even if background light subtraction is performed on A1-A3, the accurate amount of signal charge generated by reflected light cannot be obtained.

[0094] The exposure control signal in the first frame and the exposure control signal in the second frame drive the gates at different timings.

[0095] During the light emission exposure period, this operation in each frame is repeated, for example, 10,000 times, and charge is accumulated in the charge accumulation section 32. In other words, the amount of A0 during the entire light emission exposure period is approximately 10,000 times the signal charge accumulated in one exposure shown in FIG.

[0096] 10A, a background light adjustment period is set after the light emission exposure period of the first frame. During the background light adjustment period, no light emission control signal is generated. That is, during the background light adjustment period, the light emission signal generation unit 111 does not output a light emission signal to the light source 20. During the readout period set after the light emission exposure period and background light adjustment period end in the first frame, the AD conversion unit 14 reads out the signal charge accumulated in the charge accumulation unit 32 during the light emission exposure period and background light adjustment period.

[0097] As shown in FIG. 10(b), the exposure control signal during the background light adjustment period turns off the PRS gate once for a period of the second exposure packet width.

[0098] The period of the second exposure packet width is, for example, 10 usec. The period of the second exposure packet width is not particularly limited as long as it is long enough to reduce the deviation of the exposure width that occurs during the light emission exposure period to within an allowable range.

[0099] The TG1 gate and the TG2 gate are turned on at the predetermined timing as described above, and the signal charge generated by the background light is further accumulated in A0 and A2 accumulated during the light emission exposure period. The predetermined timing is determined so that the difference between the amount of signal charge due to background light contained in A0 and the amount of signal charge due to background light in A2 throughout the light emission exposure period and the background light adjustment period becomes smaller. The distance measuring device 1 performs the same operation for the second frame as for the first frame.

[0100] According to the above-described operation, by calculating A0-A2 and A1-A3, the signal charge due to background light can be more accurately removed. Therefore, the value of the delay amount Δt is calculated more accurately, and the distance measuring device 1 of the first embodiment can measure the distance to the object more accurately.

[0101] Furthermore, during the light emission exposure period and the background light adjustment period, the amount of signal charge generated by the background light is adjusted so that A0 and A2, and A1 and A3 are approximately equal, so the distance measuring device 1 of Example 1 does not require additional calculation processing in the signal processing unit 15.

[0102] [1-2. Example 2] The distance measuring device 1 of Example 2 has the same configuration as the distance measuring device 1 of Example 1. The distance measuring device 1 of Example 2 performs exposure during two periods of the first exposure packet width at the same timing in the first frame and the second frame during the light emission exposure period. In other words, the distance measuring device 1 has two periods set for accumulating signal charges used to read out pixel signals. Figure 11 is a diagram explaining the operation of the light emission exposure period of the distance measuring device of Example 2. Figure 12 is a diagram explaining the operation of the background light adjustment period of the distance measuring device of Example 2.

[0103] 11, during the light emission exposure period, the light emission signal generation unit 111 generates a light emission control signal in the first frame but does not generate a light emission control signal in the second frame. In other words, reflected light is received only in the first frame.

[0104] Therefore, A0 and A2 represent the amounts of signal charges generated by the reflected light and background light, and A1 and A3 represent the amounts of signal charges generated by the background light. Background light subtraction is performed by A0-A1 and A2-A3.

[0105] Here, the exposure control signal in the first frame and the exposure control signal in the second frame are exposure control signals that drive the gates at the same timing. However, because there is a difference between the first frame and the second frame in that there is or is not a light emission control signal, if there is interference between the light emission control signal and the exposure control signal through the power supply wiring or ground wiring, an error may occur in the timing at which the exposure control signal drives the gates.

[0106] For example, if an error occurs such that the exposure width of A0 < the exposure width of A1, the exposure width of A2 > the exposure width of A3, then during the background light adjustment period, the exposure signal generation unit 112 generates an exposure control signal such that the exposure width of A0 > the exposure width of A1 and the exposure width of A2 < the exposure width of A3, as shown in Fig. 12. This makes it possible to further reduce the difference between the amount of signal charge due to background light contained in A0 and the amount of signal charge due to background light contained in A1, and the difference between the amount of signal charge due to background light contained in A2 and the amount of signal charge due to background light contained in A3, during the entire light emission exposure period and background light adjustment period.

[0107] According to the above-described operation, by calculating A0-A1 and A2-A3, the signal charge due to background light can be more accurately removed. Therefore, the value of the delay amount Δt is calculated more accurately, and the distance measuring device 1 of the second embodiment can measure the distance to the object more accurately.

[0108] Furthermore, during the light emission exposure period and the background light adjustment period, the amount of signal charge generated by the background light is adjusted so that A0 and A1, and A2 and A3 are approximately equal, so the distance measuring device 1 of Example 2 does not require additional calculation processing in the signal processing unit 15.

[0109] In addition, in the distance measuring device 1 of Example 2, the exposure control signal during the light emission exposure period is a control signal that drives the gate at the same timing in the first frame and the second frame, so the distance measuring range is narrower than that of the distance measuring device 1 of Example 1, but since the light emission control signal is not generated in the second frame, the number of times light is emitted is halved, so the power required for the light source 20 can be reduced.

[0110] Furthermore, in the distance measuring device 1 of Example 2, since the exposure control signal that drives the PRS gate has one pulse, less power is required to drive the pixel 30, and there is no need to consider timing deviations due to interference between PRS pulses.

[0111] [1-3. Example 3] The distance measuring device 1 of Example 3 has a configuration similar to that of the distance measuring device 1 of Example 1, and performs exposure during the period of the first exposure packet width at four timings, as with the distance measuring device 1 of Example 1, but the order in which the exposure packets are acquired is different from that of the distance measuring device 1 of Example 1. Fig. 13 is a diagram illustrating the operation of the distance measuring device of Example 3 during the light emission exposure period.

[0112] 13, the reflected light is received during periods A0 and A2, so A0 and A2 represent the amounts of signal charge generated by the reflected light and background light, and A1 and A3 represent the amounts of signal charge generated by the background light. Background light subtraction is performed by A0-A1 and A2-A3.

[0113] Here, since the relative relationship between the timing of the light emission control signal and the timing of the exposure control signal is different between the first and second frames, if there is interference between the light emission control signal and the exposure control signal through the power supply wiring or ground wiring, an error may occur in the timing of driving the gate by the exposure control signal. The deviation in the exposure width that occurs during the light emission exposure period due to this error can be reduced by setting the background light adjustment period as shown in Figure 12.

[0114] According to the above-described operation, by calculating A0-A1 and A2-A3, the signal charge due to background light can be more accurately removed. Therefore, the value of the delay amount Δt is calculated more accurately, and the distance measuring device 1 of the third embodiment can measure the distance to the object more accurately.

[0115] Furthermore, during the light emission exposure period and the background light adjustment period, the amount of signal charge generated by the background light is adjusted so that A0 and A1, and A2 and A3 are approximately equal, so the distance measuring device 1 of Example 3 does not require additional calculation processing in the signal processing unit 15.

[0116] Furthermore, in the distance measuring device 1 of Example 3, since the exposure control signal that drives the PRS gate has one pulse, less power is required to drive the pixel 30, and there is no need to consider timing deviations due to interference between PRS pulses.

[0117] [1-4. Example 4] The distance measuring device 1 of Example 4 has a configuration similar to that of the distance measuring device 1 of Example 1 and performs exposure in the same acquisition order as the distance measuring device 1 of Example 3. While the distance measuring device 1 of Example 3 acquires four signal charges in two frames, the first frame and the second frame, the distance measuring device 1 of Example 4 uses two pixels 30 as a pair and acquires four signal charges in one frame by driving one pixel 30 and the other pixel 30 in the pair at different timings. Figure 14 is a diagram explaining the operation of the distance measuring device of Example 4 during the light emission exposure period. Figure 15 is a diagram explaining the operation of the distance measuring device of Example 4 during the background light adjustment period.

[0118] In the distance measuring device 1 of Example 4, for example, of the multiple pixels 30 arranged in a matrix as shown in Figure 2, the pixels 30 in the even columns are driven as a first group, and the pixels 30 in the odd columns are driven as a second group, and two adjacent pixels 30 are used as a pair.

[0119] 14 shows how A0 and A2 are acquired by the pixels 30 of the first group, and A1 and A3 are acquired by the pixels 30 of the second group. A0 and A2 represent the amount of signal charge generated by reflected light and background light, and A1 and A3 represent the amount of signal charge generated by background light. Background light subtraction is performed by A0-A1 and A2-A3. Because four exposure packets are driven continuously within the same cycle, pulse interference is likely to occur, resulting in deviations in the exposure width.

[0120] For example, if an error occurs such that the exposure width of A0 < the exposure width of A1, the exposure width of A2 > the exposure width of A3, then during the background light adjustment period, the exposure signal generation unit 112 generates an exposure control signal such that the exposure width of A0 > the exposure width of A1 and the exposure width of A2 < the exposure width of A3, as shown in Fig. 15. This makes it possible to further reduce the difference between the amount of signal charge due to background light contained in A0 and the amount of signal charge due to background light contained in A1, and the difference between the amount of signal charge due to background light contained in A2 and the amount of signal charge due to background light contained in A3, during the entire light emission exposure period and background light adjustment period.

[0121] According to the above-described operation, by calculating A0-A1 and A2-A3, the signal charge due to background light can be more accurately removed. Therefore, the value of the delay amount Δt is calculated more accurately, and the distance measuring device 1 of the fourth embodiment can measure the distance to the object more accurately.

[0122] Furthermore, during the light emission exposure period and the background light adjustment period, the amount of signal charge generated by the background light is adjusted so that A0 and A1, and A2 and A3 are approximately equal, so the distance measuring device 1 of Example 4 does not require additional calculation processing in the signal processing unit 15.

[0123] The distance measuring device 1 of the fourth embodiment can obtain four signals in one frame, and therefore can generate a distance image with little motion blur.

[0124] Furthermore, in the distance measuring device 1 of Example 4, since the exposure control signal that drives the PRS gate has one pulse, less power is required to drive the pixel 30, and there is no need to consider timing deviations due to interference between PRS pulses.

[0125] [1-5. Example 5] The distance measuring device 1 of Example 5 is a modified example of the driving of the distance measuring device 1 of Example 4. By combining two pixels 30 into one pair, four signal charges can be acquired in one frame, and exposure is performed so that the exposure timing of two of them overlaps. Figure 16 is a diagram explaining the operation of the light emission exposure period of the distance measuring device of Example 5.

[0126] As shown in Figure 16, the exposure of A1 and the exposure of A2 are performed at the same timing with respect to the light emission control signal. In Figure 16, reflected light is detected at a timing that straddles A0 and A2. That is, A0, A1, and A2 represent the amounts of signal charge generated by reflected light and background light, and A3 represents the amount of signal charge generated by background light. Background light subtraction is performed by A0-A3 and A2-A3. Note that when reflected light is detected at a timing that straddles A1 and A3, background light subtraction is performed by A1-A0 and A3-A0.

[0127] Because four exposure packets are driven continuously within the same cycle, pulse interference is likely to occur, resulting in deviations in the exposure width during the light emission exposure period. This deviation can be reduced by setting the background light adjustment period as shown in Figure 15. However, it is necessary to adjust the exposure time during the background light adjustment period so that the amount of background light contained in all signal charges A0, A1, A2, and A3 is the same.

[0128] According to the above-described operation, for example, by calculating A0-A3 and A2-A3, it is possible to more accurately exclude the signal charge due to background light. Therefore, the value of the delay amount Δt is calculated more accurately, and the distance measuring device 1 of the fifth embodiment can measure the distance to the object more accurately.

[0129] Furthermore, during the light emission exposure period and the background light adjustment period, the amount of signal charge generated by the background light is adjusted to be approximately equal for A0, A1, A2, and A3, so the distance measuring device 1 of Example 5 does not require additional calculation processing in the signal processing unit 15.

[0130] The distance measuring device 1 of the fifth embodiment can acquire four signals in one frame, and therefore can generate a distance image with little motion blur.

[0131] Furthermore, in the distance measuring device 1 of Example 5, since the exposure control signal that drives the PRS gate has one pulse, less power is required to drive the pixel 30, and there is no need to consider timing deviations due to interference between PRS pulses.

[0132] [1-6. Modification 1] In the above-described embodiments 1 to 5, the second exposure signal generated by the exposure signal generation unit 112 during the exposure time of the background light adjustment period is an exposure signal that generates one long pulse. Here, the second exposure signal may be an exposure signal that generates a pulse with the same exposure packet width as the light emission exposure period.

[0133] In other words, in the above-mentioned Examples 1 to 5, the period of the first exposure packet width was several nanoseconds, and the period of the second exposure packet width was several usec, but the period of the second exposure packet width may also be several nanoseconds.

[0134] Even in such a case, by repeating the exposure operation multiple times during the background light adjustment period, the influence of the deviation in exposure width that occurs during the light emission exposure period can be reduced, as in Examples 1 to 5.

[0135] [1-7. Modification 2] In the above-described Examples 1 to 5, each of the plurality of pixels 30 of the distance measuring device 1 has two gates 33 and two charge storage units 32, and can store two signal charges in one frame of operation. Here, each of the plurality of pixels 30 may have three gates 33 and three charge storage units 32, and can store three signal charges in one frame of operation.

[0136] In this case, for the three exposure packets A0, A1, and A2, if reflected light is detected at a timing that straddles A0 and A1, background light subtraction is performed by A0-A2 and A1-A2, and if reflected light is detected at a timing that straddles A1 and A2, background light subtraction is performed by A1-A0 and A2-A0.

[0137] Because three exposure packets are driven continuously within the same cycle, pulse interference is likely to occur, causing deviations in the exposure width during the light emission exposure period, but this deviation can be reduced during the background light adjustment period, as in Examples 1 to 5. However, it is necessary to adjust the exposure time during the background light adjustment period so that the amount of background light contained in all of the signal charges A0, A1, and A2 is the same.

[0138] [2. CW-TOF Method] In the first to fifth embodiments, the distance measuring device 1 performs distance measurement using the pulse TOF method, but the distance measuring device 1 may also perform distance measurement using the CW-TOF method.

[0139] First, we will explain the distance measurement method in CW-TOF. In CW-TOF, both the illumination light pulse and the exposure pulse are continuously repeated at a predetermined cycle. The exposure pulses have phases that differ by 90 degrees, and the phase delay of the reflected light is calculated from the four-phase signal output and converted into distance.

[0140] If the frequency of the continuous wave is f [Hz], the delay Δt is Δt = Δt = tan －１ {(A0-A2) / (A1-A3)} / 2πf, and the distance D to the object is calculated as D=Δt / 2×c, where c is the speed of light. －１ Since the (a tan) function can only return a range of ±π / 2, a function called a tan2 can be used to handle the range of ±π. －１ We will express this as (atan).

[0141] Unlike pulse TOF, all timings are covered with four phases, so no matter how delayed the timing of the reflected light is, it will be detected by one of the exposure pulses with a delay of several cycles, making it impossible to determine that it is "outside the ranging range." In other words, with the CW-TOF method, the phase difference repeats from 0° to 360°, causing aliasing of the distance for each ranging range. However, because it is possible to drive with a narrower pulse width than pulse TOF, it is easy to improve ranging accuracy within a specified range.

[0142] Hereinafter, the operation of the distance measuring device 1 according to the embodiment when it operates using CW-TOF will be described using Examples 6 and 7 according to implementation forms.

[0143] [2-1. Sixth Embodiment] In the distance measuring device 1 of the sixth embodiment, each of the plurality of pixels 30 included in the pixel array unit 13 has two gates 33 and two charge storage units 32, and can store two signal charges in one frame of operation. Fig. 17 is a diagram for explaining the operation of the distance measuring device of the sixth embodiment during the light emission exposure period.

[0144] 17, during the light emission exposure period, the PRS gate is always off. That is, the charge generated in the photoelectric conversion unit 31 is not discharged from the pixel 30, so that the signal charge generated in the photoelectric conversion unit 31 in the first frame is transferred to either A0 or A2, and the signal charge generated in the photoelectric conversion unit 31 in the second frame is transferred to either A1 or A3.

[0145] As mentioned above, in the CW-TOF method, the delay amount Δt is Δt = tan －１ It can be calculated by {(A0-A2) / (A1-A3)} / 2πf. Therefore, the pulse widths of A0 and A2, and the pulse widths of A1 and A3, must be the same, but pulse interference may prevent them from matching. Also, as mentioned above, the exposure width in the CW-TOF system is even smaller than that in the pulse TOF system, so it is even more difficult to adjust the exposure widths to be equal during the light emission exposure period than in the pulse TOF system.

[0146] Therefore, by providing a background light adjustment period as shown in Figure 10, the difference between the amount of signal charge due to background light contained in A0 during the entire light emission exposure period and background light adjustment period and the amount of signal charge due to background light contained in A2 can be made smaller.

[0147] According to the above operation, for example, by calculating A0-A2 and A1-A3, it is possible to more accurately remove the signal charge due to background light. －１ Since the value of {(A0-A2) / (A1-A3)} / 2πf is calculated more accurately, the distance measuring device 1 of the sixth embodiment can measure the distance to the object more accurately.

[0148] Furthermore, during the light emission exposure period and the background light adjustment period, the amount of signal charge generated by the background light is adjusted so that A0 and A2, and A1 and A3 are approximately equal, so the distance measuring device 1 of Example 6 does not require additional calculation processing in the signal processing unit 15.

[0149] [2-2. Seventh Embodiment] In a distance measuring device 1 according to a seventh embodiment, each of the plurality of pixels 30 included in the pixel array unit 13 has four gates 33 and four charge storage units 32, and can store four signal charges in one frame of operation. Fig. 18 is a diagram illustrating the operation of the distance measuring device according to the seventh embodiment during the light emission exposure period. Fig. 19 is a diagram illustrating the operation of the distance measuring device according to the seventh embodiment during the background light adjustment period.

[0150] 18, during the light emission exposure period, the PRS gate is always turned off. That is, the signal charge generated in the photoelectric conversion unit 31 is transferred to any one of A0, A1, A2, and A3.

[0151] In the distance measuring device 1 of Example 7, light reception is not performed during one of the four first exposure packet width periods during the light emission exposure period. In FIG. 18, reflected light is detected at timings A1, A2, and A3, and only background light is received at timing A0. Therefore, background light subtraction is performed at A1-A0, A2-A0, and A3-A0. Therefore, the amount of background light contained in all signal charges A0, A1, A2, and A3 must be the same.

[0152] For example, if an error occurs such that the exposure width of A0 < the exposure width of A1, the exposure width of A2 > the exposure width of A3, then during the background light adjustment period, the exposure signal generation unit 112 generates an exposure control signal such that the exposure width of A0 > the exposure width of A1 and the exposure width of A2 < the exposure width of A3, as shown in Fig. 19. This makes it possible to reduce the differences in the amounts of signal charge due to background light contained in A0, A1, A2, and A3 throughout the light emission exposure period and the background light adjustment period.

[0153] According to the above-described operation, for example, by calculating A1-A0, A2-A0, and A3-A0, it is possible to more accurately exclude the signal charge due to background light. Therefore, the value of the delay amount Δt is calculated more accurately, and the distance measuring device 1 of the seventh embodiment can measure the distance to the object more accurately.

[0154] Furthermore, during the light emission exposure period and the background light adjustment period, the amount of signal charge generated by the background light is adjusted to be approximately equal for A0, A1, A2, and A3, so the distance measuring device 1 of Example 7 does not require additional calculation processing in the signal processing unit 15.

[0155] The distance measuring device 1 of the seventh embodiment can obtain four signals in one frame, and therefore can generate a distance image with little motion blur.

[0156] [Effects, etc.] Techniques derived from the contents disclosed in this specification are, for example, the following techniques: Hereinafter, the techniques derived from the contents disclosed in this specification will be described together with the effects, etc. obtained by the techniques.

[0157] The first technology is a distance measuring device 1 for measuring the distance to an object using an indirect TOF (Time of Flight) method, and includes a light source 20, an emission signal generation unit 111 for generating an emission signal for causing the light source 20 to project light, a pixel array unit 13 having pixels 30 for receiving incident light, an exposure signal generation unit 112 for generating an exposure signal for instructing the pixel array unit 13 to perform exposure, a pixel drive unit 12 for driving the pixel array unit 13 based on the exposure signal, and a signal processing unit 15 for deriving the distance to the object based on information acquired from the pixel array unit 13. The pixel 30 includes a photoelectric conversion unit 31 for generating charges from the received incident light, a plurality of charge accumulation units 32 for accumulating the charges, and a signal processing unit 15 for deriving the distance to the object based on information acquired from the pixel array unit 13. and a plurality of gates 33 that transfer charges generated in the photoelectric conversion unit 31 to one of the plurality of charge accumulation units 32, the exposure signal generation unit 112 outputs a first exposure signal to the pixel driving unit 12 to drive the plurality of gates 33 so as to transfer charges generated in the photoelectric conversion unit 31 during a period of a first exposure packet width to one of the plurality of charge accumulation units 32 during the light emission exposure period, and outputs a second exposure signal to the pixel driving unit 12 to drive the plurality of gates 33 so as to transfer charges generated in the photoelectric conversion unit 31 during a period of a second exposure packet width different from the first exposure packet width to one of the plurality of charge accumulation units 32 during the background light adjustment period, and the pixel array unit 13 outputs information based on the amount of charge accumulated in each of the plurality of charge accumulation units 32 during the light emission exposure period and the background light adjustment period.

[0158] Such a distance measuring device 1 performs exposure during the background light adjustment period so as to compensate for the difference between the period of the exposure packet in which reflected light is exposed and the period of the exposure packet in which only background light is exposed during the light emission exposure period. In other words, the difference in the amount of signal charge generated by background light for each exposure packet in the light emission exposure period is adjusted during the background light adjustment period before the signal charge is output from the pixel array unit 13. The charge generated during the light emission exposure period and the charge generated during the background light adjustment period are accumulated and added together in the same charge accumulation unit 32 before the signal charge is output from the pixel array unit 13. Such a distance measuring device 1 can more accurately measure the distance to an object without requiring additional calculation processing.

[0159] Furthermore, before the signal charges are output from the pixel array section 13, the differences in the signal charges due to background light contained in each packet are corrected to be equal, so that the distance can be calculated by simpler processing.

[0160] Technique 2 is the distance measuring device 1 of technique 1, in which the light emission signal generating unit 111 does not output the light emission signal to the light source 20 during the background light adjustment period.

[0161] In such a distance measuring device 1, only background light is incident on the pixel 30 during the background light adjustment period, and therefore, the charge generated only by the background light can be accumulated in the charge accumulation section 32.

[0162] Technique 3 is the distance measuring device 1 of Technique 1 or 2, in which the plurality of gates 33 are supplied with voltage from the same power supply.

[0163] Such a distance measuring device 1 has a reduced number of electronic circuits, which allows the size of the distance measuring device 1 to be reduced and manufacturing costs to be reduced.

[0164] Technology 4 is a distance measuring device 1 according to any one of technologies 1 to 3, further comprising an information processing device 40 that acquires information indicating the difference in the amount of charge accumulated in each of multiple charge accumulation sections 32 during the light emission exposure period and the background light adjustment period, and derives the distance to the target object based on the acquired information.

[0165] In such a distance measuring device 1, the information processing device 40 acquires information indicating the difference between the amounts of charge accumulated in the charge accumulation units 32, rather than information indicating the amounts of charge accumulated in each of the charge accumulation units 32. Therefore, such a distance measuring device 1 can reduce the communication volume by half.

[0166] Technology 5 is a ranging device 1 of any of technologies 1 to 4, further comprising an information processing device 40, which determines the second exposure packet width based on information indicating the difference in the amount of charge accumulated in each of multiple charge accumulation sections 32 when background light among the incident light is incident on pixel 30.

[0167] Such a distance measuring device 1 can determine the exposure time during the background light adjustment period.

[0168] Technique 6 is the distance measuring device 1 of any of techniques 1 to 5, further comprising an information processing device 40 that acquires the temperature of the distance measuring device 1 and determines the second exposure packet width based on the acquired temperature of the distance measuring device 1.

[0169] The distance measuring device 1 can determine the exposure time during the background light adjustment period even when the deviation in the amount of background light is temperature dependent. The information processing device 40 acquires the temperature of the distance measuring device 1, more specifically, the temperature of the image sensor 10, and determines the second exposure packet width based on the acquired temperature of the image sensor 10.

[0170] Technology 7 is a distance measuring device 1 of any of technologies 1 to 6, further including an inspection data storage unit 16 that stores inspection information obtained when performance is inspected during manufacturing, and that determines the second exposure packet width based on the inspection information.

[0171] The distance measuring device 1 can determine the exposure time during the background light adjustment period even when there is manufacturing variation in the deviation of the background light amount. The information processing device 40 reads the inspection information stored in the inspection data storage unit 16 and acquires the inspection information.

[0172] Technology 8 is a distance measuring device 1 according to any of technologies 1 to 7, in which the exposure signal generation unit 112 outputs to the pixel driving unit 12 a first exposure signal and a second exposure signal such that the difference in the amount of charge generated from background light accumulated in each of the multiple charge accumulation units 32 during the light emission exposure period and the background light adjustment period is 1% or less of the difference in the amount of charge generated from incident light accumulated in each of the multiple charge accumulation units 32.

[0173] Such a distance measuring device 1 can suppress distance measurement errors to 1% or less.

[0174] Technology 9 is a distance measuring device 1 according to any of technologies 1 to 8, in which the pixel 30 has two charge storage sections 32, and the signal processing section 15 operates in two frames in which the timing of driving the multiple gates 33 is different from the timing of projecting light from the light source 20 during the light emission exposure period, thereby deriving the distance to the target based on information obtained from the four amounts of charge stored in the two charge storage sections 32 during the light emission exposure period and the background light adjustment period.

[0175] Such a distance measuring device 1 can more accurately eliminate signal charges due to background light, and therefore can more accurately measure the distance to the object.

[0176] Technology 10 is a distance measuring device 1 according to any of technologies 1 to 8, in which the pixel array unit 13 has a first pixel and a second pixel that are driven at different timings, each of the first pixel and the second pixel has two charge accumulation units 32, and the signal processing unit 15 derives the distance to the object based on information obtained from four amounts of charge accumulated in the first pixel and the second pixel during the light emission exposure period and the background light adjustment period.

[0177] Such a distance measuring device 1 can more accurately eliminate signal charges due to background light, and therefore can more accurately measure the distance to the object.

[0178] Technology 11 is a distance measuring device 1 according to any one of technologies 1 to 8, in which the pixel 30 has three charge storage sections 32, and the signal processing section 15 derives the distance to the object based on information obtained from three amounts of charge stored in the three charge storage sections 32 during the light emission exposure period and the background light adjustment period.

[0179] Such a distance measuring device 1 can more accurately eliminate signal charges due to background light, and therefore can more accurately measure the distance to the object.

[0180] Technology 12 is a distance measuring device 1 according to any of technologies 1 to 8, in which the pixel 30 has four charge storage sections 32, and the signal processing section 15 derives the distance to the object based on information obtained from four amounts of charge stored in the four charge storage sections 32 during the light emission exposure period and the background light adjustment period.

[0181] Such a distance measuring device 1 can more accurately eliminate signal charges due to background light, and therefore can more accurately measure the distance to the object.

[0182] Technique 13 is a distance measurement method performed by a distance measurement device 1 that measures the distance to an object using an indirect TOF (Time of Flight) method, the distance measurement device 1 including a light source 20 and a pixel array unit 13 having pixels 30 that receive incident light, each pixel 30 having a photoelectric conversion unit 31 that generates charges from the received incident light, a plurality of charge accumulation units 32 that accumulate the charges, and a plurality of gates 33 that transfer the charges generated in the photoelectric conversion unit 31 to any of the plurality of charge accumulation units 32 during a period of an exposure packet width, and the distance measurement method includes: the light emitting exposure step driving the plurality of gates 33 to transfer the charges generated in the photoelectric conversion unit 31 to one of the plurality of charge storage units 32 during a period of a second exposure packet width different from the first exposure packet width; and a signal processing step deriving the distance to the object based on information based on the amount of charge accumulated in each of the plurality of charge storage units 32 during the light emitting exposure step and the background light adjustment step obtained from the pixel array unit 13.

[0183] This distance measurement method exposes the pixel during the background light adjustment period so as to compensate for the difference between the period of the exposure packet in which the reflected light is exposed and the period of the exposure packet in which only the background light is exposed during the light emission exposure period. In other words, the difference in the amount of signal charge generated by the background light for each exposure packet during the light emission exposure period is adjusted during the background light adjustment period before the signal charge is output from the pixel array unit 13. The charge generated during the light emission exposure period and the charge generated during the background light adjustment period are accumulated in the charge accumulation unit 32 in the same way. With this distance measurement device 1, the difference due to background light for each exposure packet is smaller, so the distance to the object can be measured more accurately without the need for additional calculation processing.

[0184] (Other Embodiments) Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0185] In the above embodiment, during the background light adjustment period, the light emission signal generation unit 111 does not generate a light emission control signal and the light source 20 stops emitting light, but during the background light adjustment period, the light emission signal generation unit 111 may generate a light emission control signal and cause the light source 20 to emit light. In this case, during the background light adjustment period, the signal charge generated by the reflected light received by the pixel 30 can be corrected by a separately performed calibration or the like.

[0186] The communication method between the devices in the above-described embodiment is not particularly limited. Furthermore, a relay device (such as a gateway device) (not shown) may be involved in the communication between the devices.

[0187] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0188] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0189] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0190] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0191] The present disclosure may also be realized as a computer program product for causing a computer to execute the distance measuring method of the above-described embodiments. The present disclosure may also be realized as a computer-readable non-transitory recording medium on which such a program is recorded. The present disclosure also includes forms obtained by applying various modifications to the embodiments that would occur to those skilled in the art, or forms realized by arbitrarily combining the components and functions of the embodiments within the scope of the present disclosure.

[0192] The present disclosure relates to a distance measuring device that measures the distance to an object using an indirect TOF method, and can be applied to devices for detecting objects to be detected, etc.

[0193] REFERENCE SIGNS LIST 1 Range finding device 10 Image pickup element 11 Drive signal generating section 111 Light emission signal generating section 112 Exposure signal generating section 12 Pixel driving section 13 Pixel array section 14 AD conversion section 15 Signal processing section 16 Inspection data holding section 17 Sensor 20 Light source 30 Pixel 31 Photoelectric conversion section 32 Charge accumulation section 33 Gate 40 Information processing device

Claims

1. A distance measuring device that measures the distance to an object using an indirect TOF (Time of Flight) method, comprising: a light source; an emission signal generation unit that generates an emission signal that causes the light source to project light; a pixel array unit having pixels that receive incident light; an exposure signal generation unit that generates an exposure signal that instructs the pixel array unit to expose; a pixel drive unit that drives the pixel array unit based on the exposure signal; and a signal processing unit that derives the distance to the object based on information obtained from the pixel array unit, wherein the pixels have a photoelectric conversion unit that generates charges from the received incident light, a plurality of charge accumulation units that accumulate the charges, and a plurality of gates that transfer the charges generated in the photoelectric conversion unit to any of the plurality of charge accumulation units during a period of an exposure packet width based on the exposure signal, the exposure signal generation unit outputs a first exposure signal to the pixel drive unit during a light emission exposure period, which drives the plurality of gates to transfer charge generated in the photoelectric conversion unit during a period of a first exposure packet width to one of the plurality of charge accumulation units, and outputs a second exposure signal to the pixel drive unit during a background light adjustment period, which drives the plurality of gates to transfer charge generated in the photoelectric conversion unit during a period of a second exposure packet width different from the first exposure packet width to one of the plurality of charge accumulation units; and the pixel array unit outputs information based on the amount of charge accumulated in each of the plurality of charge accumulation units during the light emission exposure period and the background light adjustment period.

2. The distance measuring device according to claim 1, wherein the light emission signal generating section does not output the light emission signal to the light source during the background light adjustment period.

3. A distance measuring device according to claim 1 or 2, wherein a voltage is applied to the plurality of gates from the same power supply.

4. A distance measuring device according to any one of claims 1 to 3, further comprising an information processing device that acquires information indicating the difference in the amount of charge accumulated in each of the plurality of charge accumulation units during the light emission exposure period and the background light adjustment period, and derives the distance to the target object based on the acquired information.

5. A distance measuring device according to any one of claims 1 to 4, further comprising an information processing device, wherein the information processing device determines a second exposure packet width based on information indicating the difference in the amount of charge accumulated in each of the plurality of charge accumulation sections when background light among the incident light is incident on the pixel.

6. The distance measuring device according to any one of claims 1 to 5, further comprising an information processing device that acquires the temperature of the distance measuring device and determines the second exposure packet width based on the acquired temperature of the distance measuring device.

7. A distance measuring device according to any one of claims 1 to 6, further comprising: an inspection data storage unit that stores inspection information obtained when performance is inspected during manufacture; and an information processing device that determines the second exposure packet width based on the inspection information.

8. A distance measuring device according to any one of claims 1 to 7, wherein the exposure signal generation unit outputs the first exposure signal and the second exposure signal to the pixel drive unit such that the difference in the amount of charge generated from background light accumulated in each of the plurality of charge accumulation units during the light emission exposure period and the background light adjustment period is 1% or less of the difference in the amount of charge generated from incident light accumulated in each of the plurality of charge accumulation units.

9. A distance measuring device as described in any one of claims 1 to 8, wherein the pixel has two of the charge storage sections, and the signal processing section operates in two frames in which the timing of driving the multiple gates is different from the timing of projecting light from the light source during the light emission exposure period, thereby deriving the distance to the object based on information obtained from the four amounts of charge stored in the two charge storage sections during the light emission exposure period and the background light adjustment period.

10. A distance measuring device according to any one of claims 1 to 8, wherein the pixel array section has a first pixel and a second pixel that are driven at different timings, each of the first pixel and the second pixel has two of the charge storage sections, and the signal processing section derives the distance to the object based on information obtained from four amounts of charge stored in the first pixel and the second pixel during the light emission exposure period and the background light adjustment period.

11. A distance measuring device according to any one of claims 1 to 8, wherein the pixel has three of the charge storage sections, and the signal processing section derives the distance to the object based on information obtained from three amounts of charge stored in the three charge storage sections during the light emission exposure period and the background light adjustment period.

12. A distance measuring device according to any one of claims 1 to 8, wherein the pixel has four of the charge storage sections, and the signal processing section derives the distance to the object based on information obtained from the four amounts of charge stored in the four charge storage sections during the light emission exposure period and the background light adjustment period.

13. A distance measuring method performed by a distance measuring device that measures the distance to an object using an indirect TOF (Time of Flight) method, the distance measuring device comprising: a light source; and a pixel array unit having pixels that receive incident light, each pixel having a photoelectric conversion unit that generates charge from the received incident light, a plurality of charge accumulation units that accumulate the charge, and a plurality of gates that transfer the charge generated in the photoelectric conversion unit to one of the plurality of charge accumulation units during an exposure packet width period; the distance measuring method comprising: an emission exposure step of driving the plurality of gates to transfer the charge generated in the photoelectric conversion unit to one of the plurality of charge accumulation units during a first exposure packet width period; and a background light adjustment step of driving the plurality of gates to transfer the charge generated in the photoelectric conversion unit to one of the plurality of charge accumulation units during a second exposure packet width period that is different from the first exposure packet width. a signal processing step of deriving the distance to the object based on information based on the amount of charge accumulated in each of the plurality of charge accumulation units in the light emission exposure step and the background light adjustment step, obtained from the pixel array unit.

Images