Phase difference calculation device, phase difference calculation method, and computer readable medium

Abstract

A phase difference calculation device is provided, comprising: a first light quantity acquisition unit, which acquires a first light quantity received by reflected light of light irradiated in a first time window, and a second light quantity received by the reflected light in a second time window; a time window offset control unit, which offsets the first time window, the second time window, and the third time window in the negative direction of the time axis, thereby setting a fourth time window, a fifth time window, and a sixth time window, and offsets the fourth time window, the fifth time window, and the sixth time window in the negative direction of the time axis until the reflected light is no longer received in the fourth time window; a second light quantity acquisition unit, which acquires a third light quantity received by the reflected light in the sixth time window; and a phase difference calculation unit, which calculates the phase difference between the light and the reflected light based on a first corrected light quantity obtained by adding the third light quantity to the first light quantity and a second corrected light quantity obtained by subtracting the third light quantity from the second light quantity.

Description

Technical Field

[0001] The present invention relates to a phase difference calculation device, a phase difference calculation method, and a program. Background Art

[0002] ToF (Time of Flight) sensors, which measure distance based on the time of flight of light, are used, for example, to obtain three-dimensional information of a subject. Technologies related to ToF sensors are described, for example, in Patent Document 1. In the technology described in Patent Document 1, a depth image acquisition device includes: a light-emitting diode that irradiates modulated light toward a detection area; a ToF sensor that outputs a signal for generating a depth image by receiving light reflected from an object located in the detection area; and a filter that allows a relatively large amount of light in a specific wavelength band of the incident light to pass through, and controls at least one of the configurations of the light-emitting diode, the ToF sensor, or the filter according to the temperature of the light-emitting diode or the ToF sensor.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2019-078748 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] However, when using a ToF sensor in an environment where other objects are present, such as indoors, light reflected from the same object enters through multiple reflection paths, potentially reducing measurement accuracy. Technologies for addressing this so-called multipath reflected light are not described in, for example, Patent Document 1 mentioned above.

[0008] Therefore, an object of the present invention is to provide a phase difference calculation device, a phase difference calculation method, and a program that can reduce the influence of multipath reflected light and improve the measurement accuracy of a ToF sensor.

[0009] Solutions to Problems

[0010] According to a certain aspect of the present invention, a phase difference calculation device is provided, which includes: a first light quantity acquisition unit, which acquires, in a first time window, a first light quantity received by reflected light of light irradiated in the first time window, and a second light quantity received by the reflected light in the second time window, in consecutive first, second, and third time windows of the same length; a time window offset control unit, which offsets the first, second, and third time windows in the negative direction of the time axis, thereby setting a fourth, fifth, and sixth time window, and offsets the fourth, fifth, and sixth time windows in the negative direction of the time axis until the reflected light is no longer received in the fourth time window; a second light quantity acquisition unit, which acquires a third light quantity received by the reflected light in the sixth time window; and a phase difference calculation unit, which calculates the phase difference between the light and the reflected light based on a first corrected light quantity obtained by adding the third light quantity to the first light quantity and a second corrected light quantity obtained by subtracting the third light quantity from the second light quantity.

[0011] According to another aspect of the present invention, a phase difference calculation method is provided, which includes the following steps: in a first time window, a second time window, and a third time window of consecutive equal lengths, obtaining a first light quantity received by the reflected light of the light irradiated in the first time window, and a second light quantity received by the reflected light in the second time window; offsetting the first time window, the second time window, and the third time window in the negative direction of the time axis to set a fourth time window, a fifth time window, and a sixth time window, and offsetting the fourth time window, the fifth time window, and the sixth time window in the negative direction of the time axis until the reflected light is no longer received in the fourth time window; obtaining a third light quantity received by the reflected light in the sixth time window; and calculating the phase difference between the light and the reflected light based on a first corrected light quantity obtained by adding the third light quantity to the first light quantity and a second corrected light quantity obtained by subtracting the third light quantity from the second light quantity.

[0012] According to another aspect of the present invention, a program is provided for causing a computer to implement the following functions: in a first time window, a second time window and a third time window of consecutive equal lengths, obtaining a first light quantity received by the reflected light of the light irradiated in the first time window and a second light quantity received by the reflected light in the second time window; a time window offset control unit offsets the first time window, the second time window and the third time window in the negative direction of the time axis to set the fourth time window, the fifth time window and the sixth time window, and offsets the fourth time window, the fifth time window and the sixth time window in the negative direction of the time axis until the reflected light is no longer received in the fourth time window; obtaining a third light quantity received by the reflected light in the sixth time window; and calculating the phase difference between the light and the reflected light based on a first corrected light quantity obtained by adding the third light quantity to the first light quantity and a second corrected light quantity obtained by subtracting the third light quantity from the second light quantity. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a diagram for explaining an overview of measurement performed by a ToF sensor according to one embodiment of the present invention.

[0014] Figure 2 This is a first timing chart showing an example of the operation of the ToF sensor according to one embodiment of the present invention.

[0015] Figure 3 This is a second timing chart showing an example of the operation of the ToF sensor according to one embodiment of the present invention.

[0016] Figure 4 This is a diagram for explaining the principle of removing or reducing the influence of multipath reflected light in one embodiment of the present invention.

[0017] Figure 5 This is a diagram showing a schematic configuration of a ToF sensor according to one embodiment of the present invention.

[0018] Figure 6 It means that it can be applied to Figure 5 FIG. 1 is a diagram showing an example of a circuit configuration of a light-receiving unit of a ToF sensor.

[0019] Figure 7 This is a flowchart showing the outline of the steps of a phase difference calculation method according to one embodiment of the present invention. DETAILED DESCRIPTION

[0020] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, in this specification and the drawings, components having substantially the same functional structure are denoted by the same reference numerals, and repeated descriptions are omitted.

[0021] Figure 1 This figure is used to outline the measurement performed by a ToF sensor according to one embodiment of the present invention. The ToF sensor 100 emits pulsed light PL from a light source. The reflected light from the pulsed light PL reflected from an object obj is received by the light receiving portion of the ToF sensor 100. The distance d from the ToF sensor 100 to the object obj can be calculated based on the phase difference between the pulsed light PL and the reflected light. However, the reflected light received by the ToF sensor 100 includes not only the reflected light RL from the pulsed light PL reflected solely from the object obj, but also multipath reflected light MRL1-MRL3 from the pulsed light PL reflected from, for example, the ceiling or floor, or both, and the object obj. The phase difference between the reflected light RL and the multipath reflected light MRL1-MRL3 differs from that of the pulsed light PL. Therefore, if the multipath reflected light MRL1-MRL3 is mixed with the reflected light RL, it becomes difficult to accurately calculate the distance d.

[0022] Figure 2 This is the first timing diagram illustrating an example of the operation of a ToF sensor according to one embodiment of the present invention. In the illustrated example, the ToF sensor 100 receives light in consecutive time windows of equal length, TW1 to TW3, using a shutter S1 that opens in synchronization with the irradiation time r of pulsed light PL generated by a light source; a shutter S2 that opens only for the same duration of irradiation time r from the end of irradiation of pulsed light PL; and a shutter S3 that opens only for the same duration of irradiation time r from the closing of shutter S2. Pulsed light PL is irradiated in time window TW1, and reflected light from the pulsed light PL irradiated at the beginning of irradiation time r is received in time window TW1. The amount of reflected light received in time window TW1 is represented as light amount Q1. Meanwhile, reflected light from pulsed light PL irradiated at the end of irradiation time r is received in time window TW2. The amount of reflected light received in time window TW2 is represented as light amount Q2. In addition, although the amount of light received also includes the amount of background light BG, for example, the amount of light received in the time window TW3 where no reflected light is received can be subtracted from the amount of light in the time windows TW1 and TW2 as the amount of background light BG, so that the amounts of light Q1 and Q2 of the reflected light without the influence of the background light BG can be calculated.

[0023] Since the reference in the above Figure 1 As the distance d to the object obj increases, the phase delay of the reflected light relative to the pulsed light PL increases, resulting in a smaller proportion of the light quantity Q1 and a larger proportion of the light quantity Q2. Utilizing this phenomenon, the ToF sensor 100 calculates the phase difference between the pulsed light PL and the reflected light from the ratio of the light quantities Q1 and Q2, and further calculates the distance d from the phase difference. However, if multipath reflected light MRL1-MRL3, which is received with a delay greater than that of the reflected light RL, is mixed in, the accuracy of the calculated phase difference and distance d decreases. Specifically, the inclusion of multipath reflected light MRL1-MRL3 causes a discrepancy between the ratio of the light quantities in each time window TW1 and TW2 in the reflected light RL, which accurately reflects the distance d, and the ratio of the light quantities Q1 and Q2 of the reflected light actually observed. Therefore, in this embodiment, the following process is additionally performed to reduce the influence of multipath reflected light on the light quantities of the reflected light received in each time window.

[0024] Figure 3 This is a second timing diagram showing an example of the operation of the ToF sensor according to one embodiment of the present invention. Figure 2As shown, the pulsed light PL is irradiated in time window TW1, while the shutter timings S1-S3 are advanced. As a result, the original time windows TW1-TW3 are shifted in the negative direction of the time axis, becoming time windows TW4-TW6. The shifting of time windows TW4-TW6 is continued until reflected light is no longer received in time window TW4, as shown in the figure. In this case, the timing at which reflected light RL, which is reflected only from object obj, begins to be received coincides with the end point of time window TW4, i.e., the starting point of time window TW5. Since reflected light RL is only received for the same duration as the irradiation time r of pulsed light PL, the timing at which reflected light RL ends to be received coincides with the end point of time window TW5.

[0025] On the other hand, as described above, multipath reflected light beams MRL1-MRL3 are received later than reflected light RL. Therefore, the timing at which they begin receiving light does not coincide with the start of time window TW5, nor does the timing at which they end receiving light coincide with the end of time window TW5. As a result, multipath reflected light beams MRL1-MRL3 are also received after the end of time window TW5, that is, in time window TW6. In other words, the amount of reflected light Q3 received in time window TW6 does not include the amount of reflected light RL, but only includes the amount of multipath reflected light beams MRL1-MRL3.

[0026] Figure 4 This diagram illustrates the principle of eliminating or reducing the influence of multipath reflected light in one embodiment of the present invention. As described above, when time windows TW4 to TW6 are shifted until the amount of reflected light received in time window TW4 is substantially zero, the amount of reflected light Q3 received in time window TW6 excludes the amount of reflected light RL and includes only the amount of multipath reflected light MRL1 to MRL3. Therefore, as shown in the figure, if the amount of light Q3 is added to the amount of reflected light Q1 in time window TW1 to obtain the corrected light amount Qm1, and the amount of light Q3 is subtracted from the amount of reflected light Q2 in time window TW2 to obtain the corrected light amount Qm2, the ratio of the corrected light amounts Qm1 to Qm2 is the same as the ratio of the light amounts in each of the time windows TW1 and TW2 for reflected light RL, which does not include multipath reflected light. By calculating the phase difference and distance d based on these corrected light amounts Qm1 and Qm2, the influence of multipath reflected light can be reduced, thereby improving the measurement accuracy of the ToF sensor 100.

[0027] Figure 5This diagram schematically illustrates the configuration of a ToF sensor according to one embodiment of the present invention. In the illustrated example, the ToF sensor 100 includes a light source 110, a light receiving unit 120, and a control unit 130. The light source 110, for example, comprises an infrared laser light source and emits the aforementioned pulsed light PL. The light receiving unit 120 receives reflected light from the pulsed light PL during a specific time window and includes, for example, a light receiving element such as a photodiode, and shutters S1 to S3, which are arranged for each pixel of the light receiving element and block light incident on the light receiving element at specific time intervals.

[0028] The control unit 130 is implemented, for example, by a computer having a communication interface, a processor, and memory. The processor operates according to a program stored in the memory or received via the communication interface, thereby implementing the functions of the light source control unit 131, shutter timing control unit 132, first light intensity acquisition unit 133, time window shift control unit 134, second light intensity acquisition unit 135, phase difference calculation unit 136, and distance calculation unit 137 in software form. The functions of each unit are described below.

[0029] The light source control unit 131 controls the irradiation timing of the pulse light PL in the light source unit 110. Specifically, the light source control unit 131 controls so that the pulse light PL is irradiated at a specific cycle for an irradiation time r. In the example shown in the figure, the shutter timing control unit 132 controls the light source unit 110 so that the irradiation time of the pulse light PL is the same as the irradiation time of the pulse light PL. Figure 2 The time window TW1 shown coincides.

[0030] The shutter timing control unit 132 controls the timing of opening and closing of the shutters S1 to S3 in the light receiving unit 120 . Figure 2 as well as Figure 3 The shifting of the time windows TW1 to TW3 to the time windows TW4 to TW6 is performed by the time window shift control unit 134 inputting the amount of change in shutter timing to the shutter timing control unit 132 .

[0031] The first light quantity acquisition unit 133 acquires the light quantities Q1 and Q2 of the reflected light of the pulse light PL irradiated in the time window TW1 received in the time windows TW1 and TW2, respectively. Figure 2 The calculation for removing the influence of the background light BG described above can be performed by the first light amount acquisition unit 133 or by the phase difference calculation unit 136 described later. Figure 2 As described above, the light quantities Q1 and Q2 include the light quantities of the reflected light RL reflected only by the object obj and the light quantities of the multipath reflected lights MRL1 to MRL3 .

[0032] The time window shift control unit 134 controls Figure 2 as well as Figure 3The time windows TW1 to TW3 are shifted toward the time windows TW4 to TW6. Specifically, after the first light quantity acquisition unit 133 acquires the light quantities Q1 and Q2, the time window shift control unit 134 inputs the shutter timing change amount to the shutter timing control unit 132, so that the operation timing of the shutters S1, S2, and S3 is advanced by a time corresponding to the shift amount in the negative direction of the time axis toward the time windows TW1 to TW3.

[0033] In this embodiment, the time window shift control unit 134 obtains the amount of reflected light from the pulsed light PL irradiated after the shutter timing is changed, which is received in time windows TW4 to TW6, or a comparison result of the light amounts described later, from the light receiving unit 120. If it is determined based on the obtained light amount or the comparison result that the reflected light is not received in time window TW4, the time window shift control unit 134 terminates the time window shifting and determines time windows TW4 to TW6.

[0034] The second light quantity acquisition unit 135 acquires the light quantity Q3 of the reflected light of the pulse light PL irradiated in the time window TW1 received in the time window TW6. As described above, even if the shutter timing is shifted to match the time windows TW4 to TW6, the irradiation of the pulse light PL is maintained in the time window TW1. Figure 4 As described above, the light quantity Q3 corresponds to the light quantity corresponding to the amount of at least any one of the multipath reflected lights MRL1 to MRL3 received with a delay relative to the reflected light RL.

[0035] The phase difference calculation unit 136 calculates the phase difference between the pulsed light PL and the reflected light based on a corrected light quantity Qm1 obtained by adding the light quantity Q3 obtained by the second light quantity acquisition unit 135 to the light quantity Q1 obtained by the first light quantity acquisition unit 133, and a corrected light quantity Qm2 obtained by similarly subtracting the light quantity Q3 from the light quantity Q2. The distance calculation unit 137 calculates the distance d based on the phase difference. Calculation of the corrected phase difference and calculation of the distance based on the phase difference can be performed using known techniques, and therefore detailed descriptions are omitted.

[0036] According to the ToF sensor 100 having the above-mentioned structure, the influence of multipath reflected light is reduced in the calculation of the phase difference, thereby improving the measurement accuracy of the distance d. Figure 5The control unit 130 may be implemented as a single device with the structure shown, or as separate devices. In this case, as long as the light source unit 110 and the light receiving unit 120 are positioned close to each other, the control unit 130 does not need to be implemented in the same device as the light source unit 110 and the light receiving unit 120. Furthermore, the components of the control unit 130 do not need to be implemented in a single device. For example, the light source control unit 131 may be incorporated into the light source device that includes the light source unit 110, the shutter timing control unit 132 may be incorporated into the light receiving device that includes the light receiving unit 120, and the first light intensity acquisition unit 133, the time window shift control unit 134, the second light intensity acquisition unit 135, and the phase difference calculation unit 136 may be implemented as a phase difference calculation device separate from these. The distance calculation unit 137 may also be implemented as a separate device that calculates the distance d based on the phase difference output by the phase difference calculation device. The aforementioned ToF sensor 100 is also an example of a phase difference calculation device that includes a structure for calculating a phase difference.

[0037] Figure 6 It means that it can be applied to Figure 5 The diagram shows an example circuit configuration of the light-receiving unit of a ToF sensor. In the illustrated example, the circuit configuration includes a photodiode PD, capacitors C1 and C2, a reset switch RSW, an amplifier AMP, a selector switch SSW, a comparator CMP, and a switch SW. It outputs a pixel signal SIG and a comparator output CMP-OUT. The photodiode PD is positioned corresponding to a pixel in the light-receiving unit 120 where shutter S1 is configured, and detects the amount of light incident on that pixel. Before the shutter timing control unit 132, which receives the shutter timing change amount from the time window shift control unit 134, changes the shutter timing, the selector switch SSW is connected to the capacitor C2 side, and a charge corresponding to the amount of light received in the time window TW1 or time window TW4 before the shift is applied is stored in the capacitor C2.

[0038] After the shutter timing control unit 132 changes the shutter timing, the charge in capacitor C1 is reset using the reset switch RSW. The switch SSW is then connected to the comparator CMP side, where the charge corresponding to the amount of light received in the post-shift time window TW4 is compared with the charge corresponding to the amount of light charged to capacitor C2 before the shift. If the output of the comparator CMP indicates that the post-shift charge is smaller than the pre-shift charge, the time window shift control unit 134 further shifts the time windows TW4-TW6 in the negative direction on the time axis. If the output of the comparator CMP indicates that the pre-shift charge and the post-shift charge are the same, the time window shift control unit 134 terminates the time window shift and determines the time windows TW4-TW6. The time window shift control unit 134 may also set the final shift amount, indicating that the post-shift charge is smaller than the pre-shift charge, as the shift amount for time windows TW4-TW6.

[0039] Thus, by providing a circuit configuration that compares the amount of light received before and after the time window shift for each pixel in the light receiving unit 120, the time window shift control unit 134 in the control unit 130 can determine the end of the time window shift based on the change in shutter timing based on the light amount comparison result (comparator output CMP-OUT). Alternatively, the light receiving unit 120 may not include the aforementioned circuit configuration, and the control unit 130 may determine the end of the time window shift by comparing the light amount in time window TW1 or time window TW4 before and after the shift. In the above example, the end of the shift is determined based on the comparison result of the light amount in time window TW1 or time window TW4 before and after the shift. However, in other examples, the time window shift may be terminated when the light amount received in time window TW4 becomes equal to the light amount received in time window TW3 (the amount of background light BG).

[0040] Figure 7This is a flowchart schematically illustrating the steps of a phase difference calculation method according to one embodiment of the present invention. In the illustrated example, each step is executed by the control unit 130 of the ToF sensor 100. Under the control of the light source control unit 131, pulsed light PL is emitted. The shutter timing control unit 132 controls the shutters S1 to S3 so that when reflected light is received in time windows TW1 to TW3, the first light quantity acquisition unit 133 acquires the light quantities Q1 and Q2 in time windows TW1 and TW2 (step S101). As described above, the acquired light quantities Q1 and Q2 include the quantities of reflected light RL and multipath reflected light MRL1 to MRL3. Next, the time window shift control unit 134 shifts the time windows TW1, TW2, and TW3 in the negative direction of the time axis (step S102). For example, the time window shift control unit 134 repeatedly shifts the time windows by a specific amount until no reflected light is received in time window TW4 (step S103).

[0041] After the time window shift is complete, the second light intensity acquisition unit 135 acquires the light intensity Q3 for time window TW6 (step S104). The phase difference calculation unit 136 calculates the phase difference between the pulsed light PL and the reflected light based on the corrected light intensity Qm1 (which is the addition of light intensity Q3 to light intensity Q1) and the corrected light intensity Qm2 (which is the subtraction of light intensity Q3 from light intensity Q2) (step S105). The calculated phase difference can be used to calculate the distance d in the distance calculation unit 137 or stored or output as a phase difference for subsequent use in distance d calculation or in other devices.

[0042] The above, while referring to the attached Figure 1 While preferred embodiments of the present invention have been described in detail, the present invention is not limited to the examples described. It is apparent that anyone with ordinary knowledge in the technical field to which the present invention pertains will be able to conceive of various variations or modifications within the scope of the technical concepts described in the claims, and will understand that such variations or modifications naturally fall within the technical scope of the present invention.

[0043] Description of labels

[0044] 100…ToF sensor, 110…light source unit, 120…light receiving unit, 130…control unit, 131…light source control unit, 132…shutter timing control unit, 133…first light intensity acquisition unit, 134…time window offset control unit, 135…second light intensity acquisition unit, 136…phase difference calculation unit, 137…distance calculation unit, PL…pulsed light, RL…reflected light, MRL1–MRL3…multipath reflected light, Q1–Q3…light intensity, Qm1, Qm2…corrected light intensity, S1–S3…shutter, TW1–TW6…time window.

Claims

1. A phase difference calculation device, comprising: a first light quantity acquisition unit configured to acquire, in a first time window, a second time window, and a third time window having consecutive equal lengths, a first light quantity of reflected light received in the first time window of the light irradiated in the first time window and a second light quantity of the reflected light received in the second time window; a time window offset control unit that offsets the first, second, and third time windows in the negative direction of the time axis and sets a fourth, fifth, and sixth time window, and offsets the fourth, fifth, and sixth time windows in the negative direction of the time axis until the reflected light is no longer received in the fourth time window; a second light quantity acquisition unit configured to acquire a third light quantity of the reflected light received in the sixth time window; as well as The phase difference calculation unit calculates a phase difference between the light and the reflected light based on a first corrected light intensity obtained by adding the third light intensity to the first light intensity and a second corrected light intensity obtained by subtracting the third light intensity from the second light intensity.

2. The phase difference calculation device according to claim 1, wherein: The invention also includes: a light receiving unit, the light receiving unit including: a light receiving element for receiving the reflected light; and a shutter for blocking the light incident on the light receiving element at specific time intervals. The time window shift control unit shifts the fourth time window, the fifth time window, and the sixth time window in the negative direction of the time axis by advancing the operation timing of the shutter.

3. The phase difference calculation device according to claim 2, wherein: The light receiving unit further includes a circuit structure capable of comparing the amount of light received in the fourth time window before and after the operation timing of the shutter is advanced.

4. A phase difference calculation method comprising the following steps: In a first time window, a second time window, and a third time window of the same length, obtaining a first light quantity received in the first time window of reflected light of the light irradiated in the first time window and a second light quantity received in the second time window of the reflected light; offsetting the first time window, the second time window, and the third time window in the negative direction of the time axis to set a fourth time window, a fifth time window, and a sixth time window, and offsetting the fourth time window, the fifth time window, and the sixth time window in the negative direction of the time axis until the reflected light is no longer received in the fourth time window; acquiring a third light quantity of the reflected light received in the sixth time window; and The phase difference between the light and the reflected light is calculated based on a first corrected light amount obtained by adding the third light amount to the first light amount and a second corrected light amount obtained by subtracting the third light amount from the second light amount.

5. A computer-readable medium having a program stored thereon, the program being configured to cause a computer to implement the following functions: In a first time window, a second time window, and a third time window of the same length, obtaining a first light quantity received in the first time window of reflected light of the light irradiated in the first time window and a second light quantity received in the second time window of the reflected light; The time window offset control unit offsets the first time window, the second time window, and the third time window in the negative direction of the time axis to set the fourth time window, the fifth time window, and the sixth time window, and offsets the fourth time window, the fifth time window, and the sixth time window in the negative direction of the time axis until the reflected light is no longer received in the fourth time window; acquiring a third light quantity of the reflected light received in the sixth time window; and The phase difference between the light and the reflected light is calculated based on a first corrected light amount obtained by adding the third light amount to the first light amount and a second corrected light amount obtained by subtracting the third light amount from the second light amount.

Images