Processor, program, and distance measuring system

Abstract

To improve the accuracy of measuring a distance.SOLUTION: A processor includes a distance measurement operation processing unit for calculating the distance to a target object on the basis of the charge amount of each tap obtained from a light reception unit with the taps for each of phases with phase differences. The distance measurement operation processing unit calculates the distance to the target object on the basis of the sum of the charge amounts of the taps for each phase.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a processing device, a program, and a distance measurement system.

Background Art

[0002] Conventionally, as a technique for measuring the distance from an imaging device to an object, a ToF (Time of Flight) method is known. As one of the ToF methods, an indirect ToF method is known in which a laser pulse is periodically irradiated, the reflected light reflected by the object is accumulated, and the distance to the object is calculated based on the phase difference from the light emission (see Patent Document 1 below).

[0003] In the indirect ToF method, at each of a plurality of phases (for example, 0°, 90°, 180°, 270°), the integration time, which is the timing of emitting and receiving the irradiation light, increases the power consumption of the light source and the ToF sensor. Further, in the indirect ToF method, in order to ensure high distance measurement performance at long distances and wide angles of view, it is possible to increase the S / N by increasing the integration time to increase the amount of received light.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technique of Patent Document 1 irradiates an object with two types of irradiation light having a predetermined phase difference, and the charge generated by receiving the reflected light reflected by the object is distributed to a first tap and a second tap according to the distance to the object, and two detection signals are detected for each of the two types of irradiation light. A method is used to calculate a correction parameter for correcting the deviation of the characteristics between the first tap and the second tap.

[0005] As described above, in the indirect ToF method, in order to ensure high distance measurement performance at long distances and wide angles of view, it is important to increase the integration time to increase the S / N, and it is required to measure the distance with high accuracy while ensuring the integration time.

[0006] The present invention has been made in view of such a situation, and an object thereof is to improve ranging accuracy.

Means for Solving the Problems

[0007] In order to solve the above-described problems, a processing device according to an embodiment includes a ranging operation processing unit that calculates the distance to an object based on the charge amount of each of a plurality of taps obtained from a light receiving unit having a plurality of taps for each of a plurality of phases having a phase difference from each other. The ranging operation processing unit calculates the distance to the object based on the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases.

Effects of the Invention

[0008] According to the processing device according to an embodiment, the ranging accuracy can be improved.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

[0010] Hereinafter, an embodiment will be described with reference to the drawings.

[0011] (Configuration of Distance Measurement Module 11) FIG. 1 is a diagram showing the configuration of a distance measurement module 11 according to an embodiment. The distance measurement module 11 shown in FIG. 1 is an example of a "distance measurement system". As shown in FIG. 1, the distance measurement module 11 includes a light emitting unit 12, a light emission control unit 13, a light receiving unit 14, and a distance measurement arithmetic processing unit 15. The distance measurement module 11 irradiates an object with irradiation light, receives the reflected light of the irradiation light by the object, and measures the distance to the object.

[0012] The light emitting unit 12 emits light while modulating according to the timing of a light emission control signal supplied from the light emission control unit 13 in accordance with the control by the light emission control unit 13, and irradiates the object with the irradiation light.

[0013] The light emission control unit 13 supplies a light emission control signal having a predetermined frequency (for example, 20 MHz) to the light emitting unit 12 to control the light emission of the light emitting unit 12. Further, in order to drive the light receiving unit 14 in accordance with the light emission timing in the light emitting unit 12, the light emission control unit 13 also supplies a light emission control signal to the light receiving unit 14.

[0014] The light receiving unit 14 has a sensor surface on which a plurality of pixels are arranged in an array, and receives the reflected light from the object. Then, the light receiving unit 14 supplies image data composed of detection signals corresponding to the amount of received light of the reflected light received by each pixel to the distance measurement arithmetic processing unit 15.

[0015] The distance measurement arithmetic processing unit 15 performs an operation to obtain the distance from the distance measurement module 11 to the object based on the image data supplied from the light receiving unit 14. Then, the distance measurement arithmetic processing unit 15 generates a depth map in which the distance to the object is represented for each pixel and a reliability map in which the reliability for each distance is represented for each pixel, and outputs them to a subsequent control unit (for example, an application processing unit, an operating system processing unit, etc.).

[0016] In addition, the light-receiving unit 14 is provided with a pixel array unit 22 in which a plurality of pixel circuits 21 are arranged in an array, and a drive control circuit 23 is arranged in the peripheral region of the pixel array unit 22. The pixel array unit 22 is a sensor surface that receives reflected light. The drive control circuit 23 outputs control signals (for example, distribution signal DIMIX, selection signal ADDRESSDECODE, reset signal RST, etc.) for controlling the drive of the pixel circuit 21 based on, for example, the light emission control signal supplied from the light emission control unit 13.

[0017] The pixel circuit 21 is configured to distribute the charges generated by the photodiode 31 to taps 32A and 32B. Among the charges generated by the photodiode 31, the charges distributed to the tap 32A (hereinafter referred to as "tapA") are read out from the signal line 33A and used as the detection signal A, and the charges distributed to the tap 32B (hereinafter referred to as "tapB") are read out from the signal line 33B and used as the detection signal B.

[0018] The tap 32A is composed of a transfer transistor 41A, an FD (Floating Diffusion) unit 42A, a selection transistor 43A, and a reset transistor 44A. Similarly, the tap 32B is composed of a transfer transistor 41B, an FD unit 42B, a selection transistor 43B, and a reset transistor 44B.

[0019] The distance measurement calculation processing unit 15 may be provided in the processing unit 3 (see FIG. 17) of the distance measurement module 11, or may be provided in the processing unit 5 (see FIG. 17) of an information processing device such as a terminal device like a PC (Personal Computer) or a server arranged on the cloud. The distance measurement module 11 and the external device can transmit data by means of wired communication, wireless communication, communication via a network, communication using a portable storage medium such as an SD card, etc. Note that the processing unit 5 of the information processing device is configured to include, for example, a processor (such as a CPU (Central Processing Unit) 501), a memory (ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, SSD (Solid State Drive) 505, etc.), and the like. And the processing function of the information processing device is realized, for example, by the processor executing a program stored in the memory.

[0020] By the way, in the indirect ToF method, if the integration time is increased, at least one of the light quantity variation (in many cases, a decrease in light quantity) of the light emitting unit and the light reception sensitivity variation (in many cases, a decrease in sensitivity) of the light receiving unit may occur due to an increase in the amount of heat generated in the light emitting element and the light receiving element. If the light quantity variation of the irradiation light or the light reception variation of the photodiode occurs, the amount of charge accumulated in tapA and tapB varies, resulting in a ranging error and a decrease in ranging accuracy.

[0021] (Method for calculating the distance to an object by the distance measurement calculation processing unit 15) Here, in the distance measurement module 11 according to one embodiment, the distance measurement calculation processing unit 15 calculates the distance to the object based on tapA + tapB (that is, the sum of the charge amounts of each of the plurality of taps). Thereby, even when the light quantity fluctuation of the light emitting unit and the light reception sensitivity fluctuation of the light receiving unit occur due to heat generation, the distance measurement module 11 according to one embodiment can calculate the distance to the object using an equation in which the light quantity fluctuation of the light emitting unit and the light reception sensitivity fluctuation of the light receiving unit are corrected. For this reason, the distance measurement module 11 according to one embodiment can suppress the distance measurement error caused by the light quantity fluctuation of the light emitting unit and the light reception sensitivity fluctuation of the light receiving unit, and thus, it is possible to increase the angle of view, increase the distance, and increase the frame rate.

[0022] Hereinafter, a specific example of the method for calculating the distance to the object by the distance measurement calculation processing unit 15 will be described, assuming that the reflected light incident on 1 pixel is represented by the following mathematical formula (1).

[0023] [Number]

[0024] In the above mathematical formula (1), Ppeak_xx represents the peak light quantity entering 1 pixel in the reflected light in Qx (where xx is 0, 1, 2, 3). Also, T represents the reciprocal of the modulation frequency. Also, t represents time (where the light emission timing at the light source is 0). Also, φ represents the phase difference (the time difference between the light reflected from the light source to the object and returning, converted into phase).

[0025] [Case where the number of phases = 2] As an example, when the number of phases is 2, t = 20 ns (50 MHz), Ppeak = 1, and φ = 10 ns / T * 360 deg = 180 deg, two waveforms, namely the phase Q0 that does not delay the emission signal and the phase Q1 that delays the emission signal by 90 deg, return to the ToF sensor. Inside the ToF sensor, at the timing corresponding to half of the period T of the modulation frequency, the destination of the charge generated by the photodiode 31 switches between tapA and tapB. Since the light that enters the ToF sensor during the period of tapA is accumulated as charge, the following equations (2) and (3) hold. Here, N represents the number of pulses (integration time / T). Also, G0 represents the charge conversion coefficients of tapA and tapB (hereinafter referred to as "gain").

[0026]

Equation

Equation

[0027] The above equations (2) and (3) become the following equations (4) and (5).

[0028]

Equation

Equation

[0029] Similarly, for the phase Q1 with a phase difference of 90°, the following equations (6) and (7) hold.

[0030]

Equation

Equation

[0031] From the above, as an example of the formula for calculating φ corresponding to the target distance, when G0 = G90 and Ppeak0 = Ppeak90, the following mathematical formula (8) holds.

[0032]

Equation

[0033] Therefore, by eliminating tan-1 from the above mathematical formula (8), the following mathematical formula (9) holds, and the phase difference φ corresponding to the distance can be obtained from the following mathematical formula (9).

[0034]

Equation

[0035] Actually, after obtaining tapA0 and tapB0, tapA90 and tapB90 are obtained. Since the phase Q1 has a temperature rise compared to the phase Q0, G0 ≠ Ga90 and Ppeak0 ≠ Ppeak90. Therefore, considering the fluctuations in the peak light amount and the gain, the above mathematical formula (9) becomes the following mathematical formula (10).

[0036]

Equation

[0037] For example, when expressing the fluctuation amounts of Ppeak and G as α and β as in the following mathematical formulas (11) and (12), the above mathematical formula (10) becomes the following mathematical formula (13).

[0038]

Equation

Equation

Equation

[0039] Therefore, it can be seen that for the above formula (9), the phase difference φ varies due to the variation α of the peak light quantity and the variation β of the gain.

[0040] Here, since the sum of the charges of tapA and tapB corresponds to the integrated value obtained by photoelectrically converting the incident light to the sensor, the following formulas (14) and (15) are obtained. Based on the following formulas (14) and (15), the above formula (13) becomes the following formula (16).

[0041]

Equation

Equation

Equation

[0042] Therefore, even when variations in the peak light quantity and gain occur, it is possible to reduce the error in the distance measurement value.

[0043] <When the number of phases = 4> Hereinafter, the case where the number of phases is 4 will be described. In the 4-phase, one frame is composed of four sub-frames. In the 4-phase, tap values of phases Q2 and Q3 are added to the 2-phase.

[0044] The tap value of phase Q2 with a phase difference of 180° is obtained by the following formulas (17) and (18).

[0045]

Equation

Equation

[0046] The tap value of phase Q3 with a phase difference of 270° is obtained by the following formulas (19) and (20).

[0047] [Number] [Number]

[0048] Therefore, the equation considering the fluctuations in the peak light quantity and the gain in the four-phase is represented by the following mathematical formula (21).

[0049] [Number]

[0050] Similar to the two-phase case, if Ppeak_xx * G_xx is not constant, an error will occur in the distance measurement value. Therefore, when the above mathematical formula (21) is transformed using tapA + tapB, it becomes the following mathematical formula (22).

[0051] Therefore, by obtaining tapA + tapB for each phase, even when fluctuations in the peak light quantity and the gain occur and Ppeak_xx * G_xx fluctuates, it is possible to reduce the error in the distance measurement value.

[0052] [Number]

[0053] As described above, in the distance measurement module 11 according to an embodiment, the distance measurement operation processing unit 15 calculates the distance to the object based on tapA + tapB. Therefore, the light receiving unit 14 needs to transmit tapA + tapB, or tapA and tapB, to the distance measurement operation processing unit 15. Hereinafter, a configuration example of the light receiving unit 14 for this purpose will be described.

[0054] (Functional configuration example of the light receiving unit 14 (First example)) FIG. 2 is a diagram showing a functional configuration example (first example) of the light receiving unit 14 included in the distance measuring module 11 according to an embodiment. FIG. 3 is a diagram showing an example (first to third examples) of the image data transmitted by the light receiving unit 14 included in the distance measuring module 11 according to an embodiment. FIG. 4 is a diagram showing an example (fourth example) of the image data transmitted by the light receiving unit 14 included in the distance measuring module 11 according to an embodiment.

[0055] As shown in FIG. 2, the light receiving unit 14 includes an optical system 201, a light receiving circuit unit 202, and an image data transmission unit 203.

[0056] The light receiving circuit unit 202 measures tap A and tap B for each of the phases Q0 (0°), Q1 (90°), Q2 (180°), and A3 (270°).

[0057] The image data transmission unit 203 calculates tap A - tap B and tap A + tap B for each of the phases Q0 (0°), Q1 (90°), Q2 (180°), and A3 (270°).

[0058] Then, the image data transmission unit 203 transmits tap A - tap B and tap A + tap B for each of the phases Q0 (0°), Q1 (90°), Q2 (180°), and A3 (270°) to the subsequent distance measurement arithmetic processing unit 15 (see the first example in FIG. 3).

[0059] However, not limited thereto, for example, the image data transmission unit 203 may transmit tap A and tap B for each of the phases Q0 (0°), Q1 (90°), Q2 (180°), and A3 (270°) to the subsequent distance measurement arithmetic processing unit 15 (see the second example in FIG. 3). In this case, the subsequent distance measurement arithmetic processing unit 15 may calculate tap A - tap B and tap A + tap B, respectively.

[0060] Also, for example, the image data transmission unit 203 may transmit tapA - tapB and tapA + tapB to the subsequent ranging operation processing unit 15 for phases Q0(0°) and A3(270°), and may transmit only tapA - tapB for phases Q1(90°) and Q2(180°). (Refer to the third example in FIG. 3). In this case, the subsequent ranging operation processing unit 15 may estimate tapA - tapB and tapA + tapB for phases Q1(90°) and Q2(180°) based on tapA - tapB and tapA + tapB for phases Q0(0°) and A3(270°) by linear approximation or the like. Thereby, the ranging module 11 according to one embodiment can reduce the data amount of the image data transmitted from the image data transmission unit 203 to the subsequent ranging operation processing unit 15.

[0061] Also, for example, as shown in FIG. 5, the image data transmission unit 203 may transmit tapA + tapB to the ranging operation processing unit 15 once every several frames. In this case, the ranging operation processing unit 15 stores the variation amount of the received tapA + tapB and uses the variation amount in other frames, thereby reducing the data amount of the image data transmitted from the image data transmission unit 203 to the subsequent ranging operation processing unit 15.

[0062] Note that the processing unit of the light receiving unit 14 is configured to include, for example, a processor (such as a CPU), a memory (such as a ROM, a RAM, an SSD, etc.), and the like. And the processing function of the light receiving unit 14 is realized, for example, by the processor executing a program stored in the memory.

[0063] (Functional configuration example of the light receiving unit 14 (second example)) FIG. 5 is a diagram showing a functional configuration example (second example) of the light receiving unit 14 included in the ranging module 11 according to one embodiment.

[0064] In the light receiving unit 14 shown in FIG. 5, the image data transmission unit 203 can change the output of the image data based on external information input from the outside.

[0065] For example, when it is input as external information that the transmission target is a video or the like that prioritizes the frame rate, the image data transmission unit 203 may transmit only tapA - tapB.

[0066] Also, for example, when it is input as external information that the transmission target is a still image, the image data transmission unit 203 may transmit tapA - tapB and tapA + tapB.

[0067] Also, for example, when the temperature of the light emitting unit 12 is input as external information, the image data transmission unit 203 may transmit tapA + tapB only when the temperature of the light emitting unit 12 is equal to or higher than a predetermined threshold value.

[0068] (Functional configuration example of the light receiving unit 14 (third example)) FIG. 6 is a diagram showing a functional configuration example (third example) of the light receiving unit 14 included in the distance measuring module 11 according to an embodiment.

[0069] The light receiving unit 14 shown in FIG. 6 includes a temperature sensor 204, a correction value calculation unit 205, and a storage unit 206.

[0070] In the light receiving unit 14 shown in FIG. 6, the correction value calculation unit 205 can estimate the amount of decrease in the gain of the light receiving element based on the temperature of the light receiving element detected by the temperature sensor 204. Then, the correction value calculation unit 205 can transmit the temperature information indicating the detected temperature of the light receiving element and the correction information indicating the estimated amount of decrease in the gain of the light receiving element to the subsequent distance measurement calculation processing unit 15. Since the storage unit 206 stores in advance the correspondence relationship between the temperature of the light receiving element and the amount of decrease in the gain of the light receiving element, the correction value calculation unit 205 can estimate the amount of decrease in the gain of the light receiving element corresponding to the temperature of the light receiving element by referring to the storage unit 206. As the temperature sensor 204, for example, a semiconductor temperature sensor, a thermocouple, a thermistor, or the like can be used. Also, the temperature sensor 204 is preferably installed near the light receiving element.

[0071] Note that the distance measurement module 11 according to an embodiment may correct tapA + tapB based on the temperature of the light emitting unit 12 and the temperature of the light receiving sensor.

[0072] The value of tapA + tapB at Qx is obtained by the following formula (23).

[0073]

Equation

[0074] In the above formula (1), Ppeakx and Gx vary according to the temperature. For example, assuming that Ppeakx and Gx are linearly proportional to the temperature, the following formulas (24) and (25) hold. However, Ttx represents the temperature of the light emitting unit 12 (laser, laser driver, etc.). Also, Trx represents the temperature of the light receiving sensor. Also, a and a' represent slopes. Also, b and b' represent intercepts.

[0075]

Equation

Equation

[0076] Therefore, the following formula (26) holds, and it is possible to calculate the value of tapA + tapB according to the temperature of the light emitting unit 12 and the temperature of the light receiving sensor.

[0077]

Equation

[0078] Note that the correction of tapA + tapB described above may be performed by the correction value calculation unit 205, or may be performed by the distance measurement calculation processing unit 15 based on the temperature information detected by the temperature sensor 204.

[0079] (Functional configuration example of the light receiving unit 14 (Fourth example)) FIG. 7 is a diagram showing a functional configuration example (fourth example) of the light receiving unit 14 included in the distance measuring module 11 according to an embodiment.

[0080] In the light receiving unit 14 shown in FIG. 7, the image data transmission unit 203 can switch processing according to the temperature of the light receiving element detected by the temperature sensor 204.

[0081] For example, when the temperature of the light receiving element detected by the temperature sensor 204 is higher than a predetermined temperature threshold, the image data transmission unit 203 may determine that "parameter correction is to be performed" and transmit tapA - tapB and tapA + tapB to the distance measurement arithmetic processing unit 15.

[0082] Alternatively, when the temperature of the light receiving element detected by the temperature sensor 204 is higher than a predetermined temperature threshold, the image data transmission unit 203 may determine that "parameter correction is to be performed" and transmit tapA and tapB to the distance measurement arithmetic processing unit 15.

[0083] On the other hand, when the temperature of the light receiving element detected by the temperature sensor 204 is equal to or lower than a predetermined temperature threshold, the image data transmission unit 203 may determine that "parameter correction is not to be performed" and transmit tapA - tapB to the distance measurement arithmetic processing unit 15.

[0084] Thereby, the distance measuring module 11 according to an embodiment pays attention to the fact that the error becomes large when the temperature change of the light receiving element is large, and can achieve both the implementation of correction only when the temperature change of the light receiving element is large and the reduction of the data amount. Also, the distance measuring module 11 according to an embodiment can perform the reduction of the frame rate only when the temperature is equal to or lower than a predetermined temperature threshold by adopting such a method. Note that examples of the predetermined temperature threshold include the temperature at which the linearity of the gain of the light receiving element is lost.

[0085] (Influence of external light) The values of tapA and tapB are also affected by external light Pamb (for example, sunlight). In that case, it can be considered as in the following mathematical formulas (27) to (31).

[0086] [Number] [Number] [Number] [Number] [Number]

[0087] At this time, since tapA + tapB is the following mathematical formula (32), if it is defined as tapABx_amb = N * Gx * t * P_amb, the following mathematical formula (33) is obtained.

[0088] [Number] [Number]

[0089] Therefore, by obtaining tapA + tapB using the above mathematical formula (33) for each phase, even when fluctuations in the peak light amount and fluctuations in gain occur due to the influence of external light, resulting in fluctuations in Ppeak_xx * G_xx, it is possible to reduce the error in the distance measurement value.

[0090] Hereinafter, a method for obtaining tapABxx_amb in the above mathematical formula (33) will be described.

[0091] (Corresponding to external light) FIG. 8 is a diagram showing a configuration example (first example) of a frame in the distance measurement module 11 according to an embodiment. As shown in FIG. 8, in the four-phase, one frame is composed of four sub-frames.

[0092] As shown in FIG. 8, the distance measurement module 11 according to one embodiment can directly obtain tapABxx_amb by obtaining tapA and tapB in a state where the laser light is turned off. Note that the distance measurement module 11 according to one embodiment preferably measures tapABxx_amb for the same time as the integration time of Qx. However, not limited thereto, since tapABxx_amb (N*Gx*T*P_amb) is proportional to the ambient light acquisition time T, the distance measurement module 11 according to one embodiment may measure tapABxx_amb for a time different from the integration time of Qx and perform proportional calculation. For example, when the integration time is 1 ms and the ambient light acquisition is 0.1 ms, tapABxx_amb can be calculated by multiplying tapA + tapB obtained by the ambient light acquisition by 10.

[0093] (Functional configuration example of the light receiving unit 14 (Example 5)) FIG. 9 is a diagram showing a functional configuration example (Example 5) of the light receiving unit 14 included in the distance measurement module 11 according to one embodiment.

[0094] The light receiving unit 14 shown in FIG. 9 includes an ambient light acquisition unit 207, a correction value calculation unit 208, and a storage unit 209.

[0095] In the light receiving unit 14 shown in FIG. 9, the correction value calculation unit 208 can estimate tapABxx_amb based on the amount of ambient light detected by the ambient light acquisition unit 207. Then, the correction value calculation unit 208 can transmit the estimated tapABxx_amb to the subsequent distance measurement operation processing unit 15. Since the correspondence relationship between the amount of ambient light and tapABxx_amb is stored in the storage unit 209 in advance, the correction value calculation unit 208 can estimate tapABxx_amb corresponding to the amount of ambient light by referring to the storage unit 209. Alternatively, tapABxx_amb may be calculated based on the ratio between the amount of ambient light obtained in a state where the laser light is turned off in advance and the amount of ambient light detected by the ambient light acquisition unit 207. As the ambient light acquisition unit 207, for example, an illuminance sensor or the like can be used.

[0096] (Functional Configuration Example of Light Emitting Unit 12 (First Example)) FIG. 10 is a diagram showing a functional configuration example (first example) of a light emitting unit 12 included in the distance measuring module 11 according to an embodiment.

[0097] As shown in FIG. 10, the light emitting unit 12 includes a laser driver 301, a laser 302, and an optical system 303.

[0098] Based on the light emission signal input from the light emission control unit 13, the laser driver 301 controls the current for driving the laser 302. For example, based on the light emission signal input from the light emission control unit 13, the laser driver 301 supplies a current of 10 A to the laser 302, thereby causing the laser 302 to emit light with a burst pulse of 100 MHz and 1 ms. The laser 302 emits irradiation light when current is supplied from the laser driver 301. The light emission amount of the laser 302 is, for example, 10 W or the like. The optical system 303 optically diffuses, condenses, scans, etc. the irradiation light emitted from the laser 302.

[0099] (Functional Configuration Example of Light Emitting Unit 12 (Second Example)) FIG. 11 is a diagram showing a functional configuration example (second example) of a light emitting unit 12 included in the distance measuring module 11 according to an embodiment.

[0100] The light emitting unit 12 shown in FIG. 11 includes a temperature sensor 304 and a correction value calculation unit 305.

[0101] In the light emitting unit 12 shown in FIG. 11, the correction value calculation unit 305 can estimate the amount of decrease in the light amount of the irradiation light based on the temperature of the laser 302 or the laser driver 301 detected by the temperature sensor 304. Then, the correction value calculation unit 305 can transmit the temperature information indicating the detected temperature of the laser 302 or the laser driver 301 and the correction information indicating the estimated amount of decrease in the light amount of the irradiation light to the light emission control unit 13 in the previous stage. As the temperature sensor 304, for example, a semiconductor temperature sensor, a thermocouple, a thermistor, etc. can be used. Further, the temperature sensor 304 is preferably installed in the vicinity of the laser 302 or the laser driver 301.

[0102] (Example of IL characteristics of the light source of the light emitting unit 12) FIG. 12 is a diagram showing an example of the IL characteristics of the light source of the light emitting unit 12 included in the distance measurement module 11 according to an embodiment. As shown in FIG. 12, the IL characteristics of the light source of the light emitting unit 12 vary depending on the temperature. Therefore, it is preferable that the distance measurement module 11 acquires the IL characteristics of each of a plurality of temperatures in advance and stores them in the memory. Thereby, the correction value calculation unit 305 can estimate the current light amount based on the IL characteristics corresponding to the current temperature stored in the memory, and calculate a correction value for the light amount of the irradiation light according to the current light amount. As another example, the distance measurement module 11 may store correction values for each of a plurality of temperatures in the memory in advance. In this case, the correction value calculation unit 305 can calculate the correction value corresponding to the current temperature stored in the memory as the correction value for the light amount of the irradiation light.

[0103] (Example of timing for acquiring temperature information) FIG. 13 is a diagram showing a configuration example (second example) of a frame in the distance measurement module 11 according to an embodiment. As shown in FIG. 13, in the distance measurement module 11 according to an embodiment, temperature information can be acquired by the temperature sensor 304 in each of a plurality of phases Q0 to Q3. Note that, in the distance measurement module 11 according to an embodiment, temperature information may be acquired by the temperature sensor 304 in all of the plurality of phases Q0 to Q3, or temperature information may be acquired by the temperature sensor 304 in a part of the plurality of phases Q0 to Q3. For example, in the distance measurement module 11 according to an embodiment, temperature information may be acquired by the temperature sensor 304 in phases Q0 and Q3, and the temperature information in phases Q1 and Q2 may be estimated by interpolation.

[0104] (Example of functional configuration of the light emitting unit 12 (third example)) FIG. 14 is a diagram showing an example of the functional configuration (third example) of the light emitting unit 12 included in the distance measurement module 11 according to an embodiment.

[0105] The light emitting unit 12 shown in Fig. 14 includes a light receiving circuit 306 and a correction value calculation unit 307. Ppeak used for the calculation of tapA + tapB is proportional to the irradiation light. Therefore, the light emitting unit 12 shown in Fig. 14 can obtain Ptx proportional to Ppeak by measuring the light amount of a part of the irradiation light.

[0106] Specifically, in the light receiving circuit 306 of the light emitting unit 12 shown in Fig. 14, a part of the irradiation light is received by a light receiving element (for example, a photodiode), and the AD converter performs AD conversion on the output value of the light receiving element, so that the peak light amount and the average light amount can be obtained. Then, based on the peak light amount and the average light amount acquired by the light receiving circuit 306, the correction value calculation unit 307 calculates a correction value for the light amount of the irradiation light, and can transmit a light amount signal indicating the peak light amount and the average light amount and correction information indicating the calculated correction value to the subsequent distance measurement arithmetic processing unit 15.

[0107] Fig. 15 is a diagram showing a configuration example of the light emitting unit 12 shown in Fig. 14. In the light emitting unit 12 shown in Fig. 15, the irradiation light emitted from the laser 302 is made into substantially parallel light by the optical system 303. Then, a part of the reflected light of the irradiation light by the object is received by a light receiving element 306A such as a photodiode. The light receiving circuit 306 can acquire the reflected light amount information by the AD converter performing AD conversion on the output value of the light receiving element 306A to digitalize it.

[0108] (An example of the acquisition timing of the correction light amount) Fig. 16 is a diagram showing a configuration example (third example) of a frame in the distance measurement module 11 according to an embodiment. As shown in Fig. 16, in each of the plurality of phases Q0 to Q3, the distance measurement module 11 according to an embodiment can acquire the correction light amount by the light receiving element 306A. The distance measurement module 11 according to an embodiment may acquire the correction light amount in a period different from the integration period, or may acquire the correction light amount in the integration period.

[0109] (Hardware configuration of the processing unit) FIG. 17 is a hardware configuration diagram of the processing unit 3 of the distance measurement module 11 and the processing unit 5 of the information processing apparatus according to one embodiment.

[0110] As shown in FIG. 5, the processing unit 3 of the distance measurement module 11 and the processing unit 5 of the information processing apparatus (hereinafter referred to as "processing units 3, 5") include a CPU 501, a ROM 502, a RAM 503, an SSD 505, a recording medium 506, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, and a bus line 510.

[0111] The CPU 501 controls the overall operation of the processing units 3, 5. The ROM 502 stores programs used for driving the CPU 501 such as IPL. The RAM 503 is used as a work area for the CPU 501. The SSD 505 stores various data such as programs.

[0112] The media I / F 507 controls the reading or writing (storage) of data to / from the recording medium 506 such as a flash memory. The display 508 displays various information such as a cursor, a menu, a window, characters, or images. The network I / F 509 is an interface for data communication using the communication network 10. The keyboard 511 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The mouse 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc.

[0113] The speaker 515 outputs a sound signal under the control of the CPU 501. The bus line 510 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 501 shown in FIG. 5.

[0114] Each of the above programs may be in an installable or executable file format and may be recorded on a computer-readable recording medium and distributed. Examples of the recording medium include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray Disc, SD card, etc. Further, the recording medium can be provided as a Program Product, either domestically or abroad.

[0115] FIG. 18 is a diagram showing an example in which a distance measurement system according to another embodiment of the present disclosure is applied to a portable information terminal. FIG. 19 is a diagram showing an example in which a distance measurement system according to another embodiment of the present disclosure is applied to an autonomous driving system of a moving body.

[0116] Application examples of using the distance measurement module 11 as a distance measurement system in various detection systems will be described with reference to FIGS. 18 and 19. The detection systems in these application examples have respective functional blocks described later in addition to the distance measurement module 11. In FIGS. 18 and 19, functional blocks such as a determination unit included in the detection system are described outside the detection system for convenience of drawing. Each of the various detection systems shown in FIGS. 18 to 19 has a control unit that receives information from the distance measurement module 11 and controls the various detection systems based on the information from the distance measurement module 11.

[0117] FIG. 18 is an example of a shape measurement system as a detection system and shows an application example in which the distance measurement module 11 is used for user authentication of an electronic device.

[0118] The portable information terminal 60X, which is an electronic device, has a user authentication function. The authentication function may be realized by dedicated hardware or may be realized by a CPU (Central Processing Unit) that controls the portable information terminal 60X executing a program such as a ROM (Read Only Memory).

[0119] When authenticating a user, light is projected from the light emitting unit 12 of the distance measuring module 11 mounted on the mobile information terminal 60X toward the user 61X who uses the mobile information terminal 60X.

[0120] The light reflected by the user 61X and its surroundings is received by the light receiving unit 14 of the distance measuring module 11, and image data is generated (imaging is performed) by the image processing unit 62X. The determination unit 63X determines the degree of coincidence between the image information of the user 61X imaged by the distance measuring module 11 and the previously registered user information, and determines whether the user is a registered user.

[0121] Specifically, the shapes (contours and unevenness) of the face, ears, head, etc. of the user 61X can be measured and used as user information.

[0122] In the application example of FIG. 18, regarding the detection of the user 61X by the distance measuring module 11, distance measurement can be performed with high accuracy similar to the distance measuring module 11, and an improvement in recognition accuracy can be realized.

[0123] FIG. 18 shows an example in which the distance measuring module 11 is mounted on the mobile information terminal 60X. However, user authentication using the distance measuring module 11 can also be used for stationary personal computers, OA devices such as printers, building security systems, and the like.

[0124] In terms of functions, it can be used not only for personal authentication functions but also for scanning three-dimensional shapes such as faces. Also in this case, high-precision scanning can be realized by mounting the distance measuring module 11.

[0125] FIG. 19 shows an application example in which the distance measuring module 11 is used in an autonomous driving system in a moving body which is an example of a detection system.

[0126] In the application example of FIG. 19, the distance measuring module 11 is used for sensing an object outside the moving body 70X. The moving body 70X is an autonomous driving type moving body that can automatically travel while recognizing the external situation.

[0127] The mobile body 70X is equipped with a ranging module 11, and the ranging module 11 irradiates light toward the traveling direction of the mobile body 70X and its surrounding area. In the room 71X which is the moving area of the mobile body 70X, a desk 72X is installed in the traveling direction of the mobile body 70X.

[0128] Among the light projected from the light emitting unit 12 of the ranging module 11 mounted on the mobile body 70X, the light reflected by the desk 72X and its surroundings is received by the light receiving unit 14 of the ranging module 11, and the electrical signal subjected to photoelectric conversion is sent to the signal processing unit 73X.

[0129] Based on the electrical signal sent from the light receiving unit 14 and the like, the signal processing unit 73X calculates information regarding the layout of the room 71X, such as the distance to the desk 72X, the position of the desk 72X, and the surrounding situation other than the desk 72X.

[0130] Based on this calculated information, the determination unit 74X determines the moving route and moving speed of the mobile body 70X, and based on the determination result of the determination unit 74X, the driving control unit 75X controls the traveling of the mobile body 70X (such as the operation of the motor which is the drive source).

[0131] In the application example of FIG. 19, regarding the layout detection of the room 71X by the ranging module 11, ranging can be performed with high accuracy similar to the ranging module 11, and the accuracy improvement of the autonomous driving of the mobile body 70X can be realized.

[0132] FIG. 19 shows an example in which the ranging module 11 is mounted on the autonomous mobile body 70X traveling in the room 71X, but it can also be applied to an autonomous vehicle (so-called self-driving vehicle) traveling outdoors.

[0133] In addition, it can also be applied to a driving support system in a mobile body such as an automobile driven by a driver, rather than an autonomous driving type. In this case, the ranging module 11 can be used to detect the surrounding situation of the mobile body, and the driving of the driver can be supported according to the detected surrounding situation.

[0134] In addition to the above, the distance measurement module 11 may be applied to an article inspection system in a factory or the like. Specifically, based on the information acquired by the distance measurement module 11, the determination unit of the article inspection system determines the state of each article.

[0135] Further, the distance measurement module 11 may be applied to the operation control of a movable device.

[0136] The articulated arm as a movable device has a plurality of arms connected by bendable joints and is provided with a hand portion at its tip. The articulated arm is used, for example, in an assembly line of a factory, and grips an object with the hand portion during inspection, conveyance, and assembly of the object.

[0137] The distance measurement module 11 detects an object and its surrounding area, and the determination unit of the movable device determines various information about the object, such as the distance to the object, the shape of the object, the position of the object, and the positional relationship between a plurality of objects when they exist, based on the information acquired by the distance measurement module 11. Then, based on the determination result of the determination unit, the drive control unit controls the articulated arm operation.

[0138] Further, the distance measurement module 11 may be applied to a driving support system in a moving body such as an automobile.

[0139] The distance measurement module 11 mounted inside the automobile detects the driver driving the automobile and the surrounding area thereof. The determination unit of the driving support system determines information such as the face (expression) and posture of the driver based on the information acquired by the distance measurement module 11. Then, based on the determination result of the determination unit, the control unit performs appropriate driving support according to the situation of the driver.

[0140] The shape measurement system, the moving body, the article inspection system, the movable device, and the driving support system are all examples of detection systems. In the distance measurement module 11 in the present embodiment, the spatial resolution of the distance image can be improved. Therefore, in the detection system to which the distance measurement module 11 is applied, high-precision detection can be realized.

[0141] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications or changes can be made within the scope of the gist of the present invention described in the claims.

Description of Reference Numerals

[0142] 11 Distance measurement module (distance measurement system) 12 Light emitting unit 13 Light emission control unit 14 Light receiving unit 15 Distance measurement arithmetic processing unit 21 Pixel circuit 22 Pixel array unit 23 Drive control circuit 31 Photodiode 32A, 32B Taps 33A, 33B Signal lines 41A, 41B Transfer transistors 42A, 42B FD units 43A, 43B Selection transistors 44A, 44B Reset transistors 201 Optical system 202 Light receiving circuit unit 203 Image data transmission unit 204 Temperature sensor 205 Correction value calculation unit 206 Storage unit 301 Laser driver 302 Laser 303 Optical system 304 Temperature sensor 305 Correction value calculation unit 306 Light receiving circuit 306A Light receiving element 307 Correction value calculation unit 3, 5 Processing unit 60X Portable information terminal 61X User 62X Image processing unit 63X Judgment unit 70X Moving body 71X Indoor 72X Machine 73X Signal Processing Unit 74X Judgment Unit 75X Operation Control Unit

Prior Art Documents

Patent Documents

[0143]

Patent Document 1

Claims

1. A ranging operation processing unit that calculates the distance to an object based on the charge amount of each of the plurality of taps obtained from a light receiving unit having a plurality of taps for each of a plurality of phases having a phase difference from each other comprising wherein the ranging operation processing unit calculates the distance to the object based on the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases processing device.

2. A ranging operation processing unit that calculates the distance to an object based on the charge amount of each of the plurality of taps obtained from a light receiving unit having a plurality of taps for each of a plurality of phases having a phase difference from each other, and a correction value calculation unit that calculates a correction value of a parameter proportional to the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases, which is used for the calculation of the distance by the ranging operation processing unit processing device.

3. The correction value calculation unit calculates, as the correction value of the parameter, a correction value of the gain of the light receiving unit based on the temperature of the light receiving unit acquired by a temperature sensor The processing device according to claim 2.

4. The correction value calculation unit calculates, as the correction value of the parameter, a correction value of the amount of light of the irradiation light irradiated from a light emitting unit based on the illuminance of external light acquired by an illuminance sensor The processing device according to claim 2.

5. The correction value calculation unit calculates, as the correction value of the parameter, a correction value of the amount of light of the irradiation light irradiated from the light emitting unit based on the temperature of the light emitting unit acquired by a temperature sensor The processing device according to claim 2.

6. The correction value calculation unit switches whether to correct the parameter based on the temperature of the light receiving unit acquired by a temperature sensor The processing device according to claim 2.

7. The ranging operation processing unit calculates the distance to the object based on the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases and the tap value of the external light received by the light receiving unit The processing device according to claim 1.

8. The ranging operation processing unit stores the amount of variation of the sum of the charge amounts of each of the plurality of taps for each predetermined frame, and estimates the amount of variation of the sum of the frames in which the sum is not stored based on the amount of variation of the sum of the frames in which the amount of variation of the sum is stored The processing device according to claim 1.

9. A computer A ranging operation processing unit that calculates the distance to an object based on the charge amount of each of the plurality of taps obtained from a light receiving unit having a plurality of taps for each of a plurality of phases having a phase difference from each other A program that causes it to function as The ranging operation processing unit calculates the distance to the object based on the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases Program

10. A computer A ranging operation processing unit that calculates the distance to an object based on the charge amount of each of the plurality of taps obtained from a light receiving unit having a plurality of taps for each of a plurality of phases having a phase difference from each other, and A correction value calculation unit that calculates a correction value of a parameter proportional to the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases, which is used for the calculation of the distance by the ranging operation processing unit A program that causes it to function as

11. A light emitting unit that irradiates an object with irradiation light, A light receiving unit that receives the reflected light of the irradiation light by the object, A processing device Comprising The processing device A ranging operation processing unit that calculates the distance to an object based on the charge amount of each of the plurality of taps obtained from the light receiving unit having a plurality of taps for each of a plurality of phases having a phase difference from each other Comprising The ranging operation processing unit calculates the distance to the object based on the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases Ranging system

12. A light emitting unit that irradiates an object with irradiation light, A light receiving unit that receives the reflected light of the irradiation light by the object, A processing device Comprising The processing device A ranging operation processing unit that calculates the distance to an object based on the charge amount of each of the plurality of taps obtained from a light receiving unit having a plurality of taps for each of a plurality of phases having a phase difference from each other, and A correction value calculation unit that calculates a correction value of a parameter proportional to the sum of the charge amounts of each of the plurality of taps for each of the plurality of phases, which is used for the calculation of the distance by the ranging operation processing unit Ranging system

Images