Distance measuring device

Abstract

A distance measuring device (1) is provided with a light source (2), a light receiving unit (5), a control unit (6), and a processing unit (7). The light receiving unit (5) includes a photodiode and a charge storage unit that stores charges generated by the photodiode. The control unit (6) supplies to the light receiving unit (5) a control pattern comprising M frames indicating whether or not to accumulate charges in the charge accumulation unit in each of N periods divided by a certain time T from the light pulse output timing of the light source (2). When the control pattern is represented by a matrix of M rows and N columns, and the value of an element of the mth row and the nth column of the matrix is set to be 1 when charge accumulation is indicated in the nth period in the mth frame and is set to be 0 when non-accumulation is indicated in the mth frame, the condition that the Hamming distance is 1 or the like is satisfied for all combinations of two adjacent column vectors among N column vectors constituting the matrix. As a result, it is possible to implement a device for reliably performing distance measurement by a TOF method using a compressive sensing technique.

Description

Technical Field

[0001] The present disclosure relates to an apparatus and a method for measuring a distance to an object by a time-of-flight method. Background Art

[0002] The distance measurement technology using the time-of-flight (TOF) method measures the distance to the object by finding the time for the light pulse output from the light source to be reflected by the object and return to the light receiving unit. As a method of distance measurement based on the TOF method, a two-phase method or a phase shift method is known, and a method using compressed sensing technology is also known (Patent Document 1, Non-Patent Document 1). In any method, the distance measuring device includes: a light source that irradiates a light pulse to the object; and a light receiving unit including a photodiode and a charge storage unit.

[0003] In the distance measurement technology based on the two-stage method, a light pulse with a pulse width T output from a light source is irradiated to an object, and among the charges generated by a photodiode that receives the light pulse reflected by the object, the charges generated in a first period of time T equal to the pulse width are accumulated in one charge accumulation unit, and the charges generated in a second period of the next time T are accumulated in another charge accumulation unit. Then, based on the ratio of the amounts of charges accumulated in the two charge accumulation units, the time from the time when the light pulse is output from the light source to the time when the light pulse is received by the photodiode is calculated, and the distance to the object is obtained.

[0004] In the distance measurement technology based on the two-stage method, if the measurable distance is to be extended, the pulse width of the light pulse and the time of the charge accumulation period need to be extended, so the measurable distance and the distance resolution become a trade-off.

[0005] In the distance measurement technology based on the phase shift method, a light pulse of pulse width T outputted from a light source is irradiated to an object, and among the charges generated by a photodiode receiving the light pulse reflected by the object, the charges generated in a first period of time T equal to the pulse width are accumulated in a charge accumulation unit. Then, charges are accumulated in a second period of time T following the first period. Thereafter, charges are accumulated in a similar manner in an nth period of time T following the n-1th period.

[0006] In this way, the charge accumulation period is shifted by time T, and the charge generated by the photodiode is accumulated in each of the multiple periods divided by time T. Then, based on the amount of charge accumulated in each of these multiple periods, the time from the time when the light pulse of the light source is output to the time when the light pulse of the photodiode is received is calculated, and the distance to the object is obtained.

[0007] In the distance measurement technology based on the phase shift method, the measurable distance can be extended without reducing the distance resolution by increasing the number of periods divided by the time T. However, when the number of periods divided by the time T increases, the number of measurements can also be increased.

[0008] The distance measurement technology using the compressed sensing technology is based on the fact that a reflected light pulse appears in a limited period after the light pulse output of the light source, and there is no reflected light in other time periods, so the reflected light intensity as a time function has sparseness. That is, after the light pulse output of the light source, the charge generated by the photodiode is accumulated in the charge accumulation unit in one or more periods according to a random frame pattern. And, for each of the multiple frame patterns that are different from each other, the distance to the object is calculated based on the amount of charge accumulated in the charge accumulation unit by the compressed sensing technology.

[0009] Compared with the two-stage method, the distance measurement technology using compressed sensing technology can extend the measurable distance without reducing the distance resolution. In addition, compared with the phase shift method, the distance measurement technology using compressed sensing technology can measure the distance to the object with fewer measurements. The distance measurement technology using compressed sensing technology can be said to be a technology that speeds up or improves the performance of the distance measurement technology based on the phase shift method.

[0010] Prior art literature

[0011] Patent Literature

[0012] Patent Document 1: International Publication No. 2016 / 133053

[0013] Non-patent literature

[0014] Non-patent document 1: Keiichiro Kagawa, et al., "A Dual-Mode303-Mega frames-per-Second Charge-Domain Time-Compressive Computational CMOS Image Sensor", Sensors, 22(5), 1953, pp.1-16, 2022

[0015] Non-patent document 2: Joel A. Tropp, et al., "Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit", IEEE TRANS ACTIONS ON INFORMATION THEORY, Vol. 53, No. 12, pp. 4655-4666, 2007 Summary of the invention

[0016] Problems to be solved by the invention

[0017] The inventors of the present invention have discovered the following problems in the process of studying distance measurement technology using compressed sensing technology. That is, sometimes it is impossible to find the distance to the object based on the pattern indicating the period during which the charge generated by the photodiode is accumulated in the charge accumulation unit. Or sometimes it is not easy to find a pattern that can find the distance to the object.

[0018] An object of the embodiment is to provide an apparatus and method that can reliably perform distance measurement based on the TOF method using compressed sensing technology.

[0019] Means used to solve problems

[0020] An embodiment is a distance measuring device. A distance measuring device, comprising: (1) a light source for irradiating a light pulse having a pulse width P to an object; (2) a light receiving unit including: a photodiode for generating a charge by receiving the light pulse irradiated from the light source to the object and reflected by the object; and a charge storage unit for storing the charge generated by the photodiode; (3) a control unit for providing the light receiving unit with a control pattern consisting of M frames, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light pulse is output from the light source to the charge storage unit and store the charge; and (4) a processing unit for calculating the distance to the object by a compression sensing technique based on the amount of charge stored in the charge storage unit, wherein the distance measuring device is configured to measure the distance to the object by a time-of-flight method, wherein the pulse width P is less than the certain time T, and the control unit provides the control pattern to the light receiving unit, wherein the control pattern is represented by a matrix having M rows and N columns, and the value a of the element in the mth row and nth column of the matrix having M rows and N columns is set to the value a of the element in the mth row and nth column of the matrix having M rows and N columns. m，n When the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, and the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1.

[0021] The embodiment is a distance measuring device. A distance measuring device, comprising: (1) a light source that irradiates a light pulse with a pulse width P to an object; (2) a light receiving unit that includes: a photodiode that receives the light pulse irradiated from the light source to the object and reflected by the object to generate a charge; and a charge storage unit that stores the charge generated by the photodiode; (3) a control unit that provides the light receiving unit with a control pattern consisting of M frames, the M frames indicating whether to generate a charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light pulse is output from the light source. The charge is transferred to the charge storage unit and the charge is stored; and (4) a processing unit, which calculates the distance to the object by a compression sensing technology based on the amount of charge stored in the charge storage unit, the distance measuring device is configured to measure the distance to the object by a time-of-flight method, the pulse width P is greater than k-1 times and less than k times (k is an integer greater than 2) a certain time T, and the control unit provides a control pattern to the light receiving unit, the control pattern is when the control pattern is represented by a matrix of M rows and N columns, the value a of the element of the mth row and nth column of the matrix of M rows and N columns m，n When the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1, the Hamming distance of all combinations of two column vectors among the N column vectors that are separated from each other by only k+1 columns is greater than k, and for all combinations of consecutive k+1 columns among the N column vectors, in the matrix of M rows and k+1 columns composed of k+1 column vectors, the value of any element in the M row vectors is 1, and there are more than k+1 rows of row vectors that are different from each other.

[0022] An embodiment is a distance measurement method. A distance measurement method, wherein the method is a method of measuring the distance to the object by a time-of-flight method using: (1) a light source that irradiates a light pulse of a pulse width P to an object; and (2) a light receiving unit that includes: a photodiode that generates a charge by receiving the light pulse irradiated from the light source to the object and reflected by the object; and a charge storage unit that stores the charge generated by the photodiode, and comprising: (3) a control step of providing a control pattern consisting of M frames to the light receiving unit, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light pulse is output from the light source to the charge storage unit and store the charge; and (4) a processing step of calculating the distance to the object by a compression sensing technique based on the amount of charge stored in the charge storage unit, wherein the pulse width P is less than the certain time T, and in the control step, the control pattern is provided to the light receiving unit, the control pattern is represented by a matrix of M rows and N columns, and the value a of the element of the mth row and nth column of the matrix of M rows and N columns is set to the value a of the element of the mth row and nth column of the matrix of M rows and N columns. m，n When the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, and the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1.

[0023] The embodiment is a distance measurement method. A distance measurement method, wherein a method is a method of measuring the distance to the object by a time-of-flight method using: (1) a light source that irradiates a light pulse of pulse width P to an object; and (2) a light receiving unit that includes: a photodiode that generates a charge by receiving the light pulse irradiated from the light source to the object and reflected by the object; and a charge accumulation unit that accumulates the charge generated by the photodiode, and comprising: (3) a control step that provides a control pattern consisting of M frames to the light receiving unit, the M frames indicating whether to start from the time when the light pulse from the light source is output. The charge generated by the photodiode in each of N periods divided by a certain time T is transferred to a charge storage unit and the charge is stored; and (4) a processing step, which calculates the distance to the object by a compression sensing technology based on the amount of charge stored in the charge storage unit, wherein the pulse width P is greater than k-1 times and less than k times (k is an integer greater than 2) the certain time T. In the control step, a control pattern is provided to the light receiving unit, and the control pattern is represented by a matrix of M rows and N columns, and the value a of the element of the mth row and nth column of the matrix of M rows and N columns is m，nWhen the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1, the Hamming distance of all combinations of two column vectors among the N column vectors that are separated from each other by only k+1 columns is greater than k, and for all combinations of consecutive k+1 columns among the N column vectors, in the matrix of M rows and k+1 columns composed of k+1 column vectors, the value of any element in the M row vectors is 1, and there are more than k+1 rows of row vectors that are different from each other.

[0024] Effects of the Invention

[0025] According to the distance measurement device and the distance measurement method of the embodiment, it is possible to reliably perform distance measurement based on the TOF method using the compression sensing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 1 is a diagram showing the structure of the distance measurement device 1 .

[0027] Figure 2 (a) is a diagram schematically showing the structure of the light receiving unit 5 of the distance measuring device 1, and (b) is a diagram schematically showing the situation in which the charge generated in the photodiode PD is transferred to the second charge storage unit C2 via the switch SW2 when the switches SW1, SW3, and SW4 are in the off state and the switch SW2 is in the on state.

[0028] Figure 3 It is a diagram showing a control pattern of a comparative example.

[0029] Figure 4 This figure shows the control pattern when M=3 and N=7 in waveform form.

[0030] Figure 5 This is a diagram showing the control pattern when M=3 and N=7 in a table format.

[0031] Figure 6 This is a diagram showing the control pattern when M=3 and N=7 in a table format.

[0032] Figure 7 This is a diagram showing the control pattern when M=3 and N=7 in a table format.

[0033] Figure 8 This is a diagram showing the control pattern when M=4 and N=15 in a table format.

[0034] Fig. 9This is a diagram showing a control pattern in the case of k=2, M=4, and N=11 in a waveform form.

[0035] Fig.10 This is a diagram showing control patterns when k=2, M=4, and N=11 in a table format.

[0036] Fig.11 This is a diagram showing control patterns in a table format when k=2, M=4, and N=12.

[0037] Fig.12 This is a diagram showing control patterns when k=2, M=5, and N=31 in a table format.

[0038] Fig.13 This is a diagram showing, in a table format, a control pattern when k=3, M=4, and N=7.

[0039] Fig.14 This is a diagram showing control patterns when k=3, M=6, and N=25 in a table format.

[0040] Fig.15 This is a diagram showing control patterns when k=4, M=6, and N=31 in a table format.

[0041] Fig.16 The waveform shows the adjustment of the charge accumulation times r n Figure 4 shows a control pattern of the control pattern.

[0042] Fig.17 The table shows the adjusted charge accumulation times r n Figure 4 shows a control pattern of the control pattern.

[0043] Fig.18 This is a diagram showing an example of a control pattern in which a plurality of frames that do not simultaneously instruct charge accumulation in the same period can be simultaneously provided to the light receiving unit 5 .

[0044] Fig.19 This is a diagram for explaining an example of solving the L0 optimization problem by applying the OMP algorithm. DETAILED DESCRIPTION

[0045] Hereinafter, embodiments of the distance measuring device and the distance measuring method will be described in detail with reference to the accompanying drawings. In addition, in the description of the accompanying drawings, the same elements are marked with the same reference numerals, and repeated descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all changes within the meaning and scope equivalent to the claims.

[0046] Figure 11 is a diagram showing the structure of an image measuring device 1. The distance measuring device 1 is a device for measuring the distance to an object by a time-of-flight (TOF) method, and includes a light source 2, an irradiation optical system 3, a focusing optical system 4, a light receiving unit 5, a control unit 6, and a processing unit 7. The distance measuring method uses the light source 2, the irradiation optical system 3, the focusing optical system 4, and the light receiving unit 5 to perform a control step and a processing step.

[0047] The light source 2 outputs light pulses to be irradiated to the object. The light source 2 outputs light pulses with a certain pulse width P at a certain repetition frequency. The light source 2 can be any light source as long as it can output light pulses, and can be, for example, a laser diode, a light emitting diode, or the like.

[0048] The irradiation optical system 3 is an optical system that irradiates the object with the light output from the light source 2. When the light output from the light source 2 is divergent light, the irradiation optical system 3 efficiently irradiates the object with the light.

[0049] The focusing optical system 4 receives a light pulse (reflected light pulse) irradiated from the light source 2 to the object via the irradiation optical system 3 and reflected by the object, and focuses the reflected light pulse.

[0050] The light receiving unit 5 receives the reflected light pulse that arrives via the focusing optical system 4. The light receiving unit 5 includes a photodiode that generates electric charge by receiving the reflected light pulse, and a charge storage unit that stores the electric charge generated by the photodiode.

[0051] The control unit 6 provides the control pattern to the light receiving unit 5 (control step). The control pattern indicates whether the charge generated by the photodiode of the light receiving unit 5 in each of the N periods divided by a certain time T from the time when the light pulse of the light source 2 is output is transferred to the charge accumulation unit and accumulated. The control pattern is composed of M frames, and indicates the accumulation / non-accumulation of the charge in each frame and each period. M and N are integers greater than 2.

[0052] The processing unit 7 obtains the distance to the object by the compression sensing technology based on the amount of charge generated by the photodiode of the light receiving unit 5 and accumulated by the charge accumulation unit (processing step).

[0053] The control unit 6 and the processing unit 7 may be a computer. The control unit 6 and the processing unit 7 include a computing unit (e.g., a CPU, etc.) for performing calculation processing, a storage unit (e.g., a hard disk drive, a RAM, a ROM, etc.) for storing control patterns or charge accumulation amounts, a display unit (e.g., a liquid crystal display, etc.) for displaying control patterns, etc., an input unit (e.g., a keyboard, a mouse, etc.) for receiving an instruction to start measurement or input of measurement conditions, etc., and the like.

[0054] The control unit 6 and the processing unit 7 may be not only a computer but also an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0055] Figure 2 1 is a diagram schematically showing the structure of the light receiving unit 5 of the distance measuring device 1. In this figure, the light receiving unit 5 has two charge storage units. The light receiving unit 5 includes a photodiode PD that generates charges by receiving light, and a first charge storage unit C1 and a second charge storage unit C2 that store charges.

[0056] The light receiving unit 5 includes a switch SW1 for transferring the charge generated in the photodiode PD to the first charge storage unit C1, a switch SW2 for transferring the charge generated in the photodiode PD to the second charge storage unit C2, a switch SW3 for outputting the charge stored in the first charge storage unit C1, and a switch SW4 for outputting the charge stored in the second charge storage unit C2. The switches SW1 and SW2 are set to either an on state or an off state according to the values ​​VTX1 and VTX2 of the control pattern provided from the control unit 6.

[0057] Figure 2 (a) shows the circuit structure of the light receiving unit 5. Figure 2 (b) schematically shows that when switches SW1, SW3, and SW4 are in the off state and switch SW2 is in the on state, the charge generated in the photodiode PD is transferred to the second charge storage unit C2 via switch SW2. When the charge transfer to the second charge storage unit C2 is completed, switch SW2 is in the off state, switch SW4 is in the on state, and the charge stored in the second charge storage unit C2 is output via switch SW4.

[0058] The number of charge storage units may be one or more than two. Any of the multiple charge storage units may be used as a charge discarding unit, or a charge discarding unit may be provided separately. The charge discarding unit accumulates the charge generated by the photodiode PD during the period when the control pattern does not indicate charge accumulation, and the charge does not need to be output. In addition, the light receiving unit 5 includes a switch for initializing the charge accumulation in each of the charge storage unit and the charge discarding unit.

[0059] The light receiving unit 5 may also be an imaging element in which a plurality of pixels including a photodiode and a charge storage unit are arranged two-dimensionally on the light receiving surface. In this case, the focusing optical system 4 may be an imaging optical system that inputs a reflected light pulse from the object and forms an image. The processing unit 7 can obtain a distance image of the object by calculating the distance to the object for each of the plurality of pixels. An imaging element in which a plurality of pixels having the structure of the light receiving unit 5 are arranged two-dimensionally is sold by Hamamatsu Photonics Co., Ltd. as a product "distance measurement area image sensor".

[0060] The distance measuring device and the distance measuring method of the present embodiment measure the distance to the object by using the light source 2 and the light receiving unit 5 through the compression sensing technology as described above, and are characterized by the control pattern provided by the control unit 6 to the light receiving unit 5, and also by the algorithm for calculating the distance by the processing unit 7. The control pattern provided by the control unit 6 to the light receiving unit 5 in the control step will be described below, and then the content of the processing performed by the processing unit 7 in the processing step will be described.

[0061] Figure 3 : is a diagram showing a control pattern of a comparative example. In this figure, the waveform of the irradiation light pulse output from the light source, the waveform of the reflected light pulse reaching the light receiving unit, and the patterns of the first to fourth frames in the control pattern indicating the period during which the charge generated by the photodiode in the light receiving unit is accumulated in the charge accumulation unit are shown in order from top to bottom. The waveform of the irradiation light pulse and the waveform of the reflected light pulse actually have noise or deformation, but they are schematically represented by rectangles in this figure (and subsequent figures). In addition, one reflected light pulse reaches the light receiving unit for each irradiation light pulse.

[0062] The pulse width of each of the irradiation light pulse output from the light source and the reflected light pulse reaching the light receiving unit is set to P. The time of each of the multiple periods divided after the light pulse output time of the light source is set to T. In this figure, P=T. The control pattern indicates the value 1 when indicating the accumulation of charge in each frame and each period, and indicates the value 0 when indicating non-accumulation.

[0063] There is a time difference Δt corresponding to the distance to the object between the light pulse output time of the light source and the arrival time of the reflected light pulse of the light receiving part. If the time difference Δt is detected, the distance to the object can be calculated. In the case where the period after the light pulse output time of the light source is divided into 8 periods, the control pattern requires 8 frames in the phase shift method. In contrast, in the case of using compressed sensing technology, Figure 3 As shown, the control pattern may be 4 frames.

[0064] As shown in the figure, a reflected light pulse appears in a limited time period after the light pulse output of the light source, and there is no reflected light in other time periods, so the reflected light intensity as a time function is sparse. Therefore, the time from the output of the irradiated light pulse to the arrival of the reflected light pulse can be calculated using the compressed sensing technology, and the distance to the object can be calculated. In addition, compared with the number of control patterns required in the case of the phase shift method, the number of control patterns required in the case of using the compressed sensing technology can be reduced.

[0065] However, when using compressed sensing technology to perform distance measurement based on the TOF method, it is sometimes impossible to find the distance to the object based on the pattern indicating the period during which the charge generated by the photodiode is accumulated in the charge accumulation unit. Or, it is sometimes difficult to find a pattern that can find the distance to the object. The distance measurement device and distance measurement method described below provide a control pattern that satisfies certain conditions to the light receiving unit 5 through the control unit 6, and can reliably perform distance measurement based on the TOF method using compressed sensing technology.

[0066] To explain the conditions that the control pattern should satisfy, the control pattern is represented by a matrix Φ of M rows and N columns as shown in the following formula (1). M is the number of frames included in the control pattern. N is the number of divided periods after the light source outputs a light pulse.

[0067] [Formula 1]

[0068]

[0069] The value a of the element in the mth row and nth column of the matrix Φ m，n In the nth period among the N periods in the mth frame among the M frames, the value is set to 1 when the charge accumulation unit is instructed to accumulate charges, and is set to 0 when the charge accumulation unit is instructed not to accumulate charges. In the matrix Φ, the column vector of the nth column As shown in the following formula (3), the matrix Φ consists of N column vectors constitute.

[0070] [Formula 2]

[0071]

[0072] [Formula 3]

[0073] Φ=(φ1…φ n …φ N )……(3)

[0074] The conditions that the control pattern should satisfy vary depending on the ratio of the pulse width P of the light pulse to the time T of each period. When the pulse width P is less than or equal to the time T (P≤T), the control pattern should satisfy the following first to third conditions.

[0075] The first condition is that in N column vectors The value of any element in all is 1. That is, the matrix Φ does not contain a column vector in which all the elements have a value of 0. This condition is necessary to obtain all the information of N periods. If the matrix Φ contains a column vector in which all the elements have a value of 0, no information about the period corresponding to the column vector can be obtained.

[0076] The second condition is N column vectors All of are different from each other. That is, the same column vector is not included in the matrix Φ. This condition is required in order to identify the position of the reflected light pulse with the same pulse width P as the time T. If the matrix Φ contains the same column vector, the position of the reflected light pulse with the same pulse width P as the time T cannot be identified.

[0077] The third condition is about N column vectors The Hamming distance of all combinations of two adjacent column vectors in is 1. The Hamming distance refers to the number of positions where the values ​​of the elements located at the same position between two column vectors are different when the values ​​are compared. For example, between the two column vectors shown in the following formula (4), the number of positions where the values ​​are different is 3, so the Hamming distance is 3. This condition is necessary to distinguish the reflected light pulse from other positions when it spans two periods.

[0078] [Formula 4]

[0079]

[0080] Figures 4 to 8 This is a diagram showing an example of a control pattern that satisfies the first to third conditions when the pulse width P is equal to or shorter than the time T (P≤T).

[0081] Figure 4 The figure shows the waveform of the irradiation light pulse output from the light source and the control pattern when M=3 and N=7 in the form of a waveform. Figure 5 It is expressed in table form. Figure 4 A diagram of a control pattern represented in waveform form. Figure 6 , Figure 7 This is a diagram showing the control pattern when M=3 and N=7 in a table format. Figure 8 The diagram shows in a table format a control pattern when M = 4 and N = 15. When the pulse width P is equal to or shorter than time T (P ≤ T), M and N have the following relationship (5).

[0082] [Formula 5]

[0083] N=2 M-1……(5)

[0084] When k is an integer greater than 2 and the pulse width P is greater than k-1 times and less than k times the time T ((k-1)T<P≤kT), the control pattern needs to satisfy the following fourth and fifth conditions in addition to the first to third conditions described above.

[0085] The fourth condition is that, with respect to N column vectors For all combinations of two column vectors separated by only k+1 columns, the Hamming distance is greater than k. and column vector For all combinations of , the Hamming distance is greater than k. This condition is necessary in order to distinguish the presence of a reflected light pulse in any of the two periods corresponding to the two column vectors separated by only k+1 columns.

[0086] The fifth condition is that with respect to N column vectors For all combinations of consecutive k+1 column vectors, in a matrix of M rows and k+1 columns consisting of k+1 column vectors, there are at least k+1 rows of row vectors whose values ​​of any elements in the M row vectors are 1 and are different from each other.

[0087] That is, if we set the column vector of a continuous k+1 columns to Then the column vectors of these k+1 columns The matrix of M rows and k+1 columns is represented by the following formula (6). The value of any element in the M row vectors contained in the matrix is ​​1, and there are more than k+1 rows of row vectors that are different from each other. When the pulse width P of the reflected light pulse is less than kT, the reflected light pulse exists over a maximum of k+1 periods. In order to calculate the amount of accumulated charge in each of the k+1 periods, at least k+1 information is required, so this condition is required.

[0088] [Formula 6]

[0089]

[0090] When k is an integer greater than 2 and the pulse width P is greater than k-1 times and less than k times the time T ((k-1)T<P≤kT), the control pattern is preferably such that, in addition to satisfying the first to

[0091] In addition to the fifth condition, the following sixth condition is also satisfied.

[0092] The sixth condition is that, with respect to N column vectors All combinations of consecutive column vectors with less than k+1 columns in the , the sum of these column vectors with less than k+1 columns, that is, the inner product of the sum column vector and each column vector Divide by the column vector The column vector with the largest value obtained by adjusting the size of is any one of the column vectors with k+1 or less columns.

[0093] That is, if N column vectors The column vectors below the continuous k+1 columns in are set as Then these column vectors The sum of the sum column vector S A It is expressed by the following formula (7). A With each column vector The inner product of the column vector The calculation of the size of is expressed by the following formula (8). The column vector with the largest calculated value is the column vector Any of .

[0094] [Formula 7]

[0095]

[0096] [Formula 8]

[0097]

[0098] In the case of k = 2 or k = 3, the sixth condition is automatically satisfied by satisfying the other conditions. Furthermore, the sixth condition is necessary when the orthogonal matching pursuit algorithm (non-patent document 2) is used when calculating the distance by the processing unit 7 described later, but is unnecessary when other algorithms (such as the round robin method) are used.

[0099] Figures 9 to 15 The diagram shows an example of a control pattern satisfying the first to sixth conditions when k is an integer greater than 2 and the pulse width P is greater than k-1 times and less than k times the time T ((k-1)T<P≤kT).

[0100] Fig. 9 The diagram shows the waveforms of the irradiation light pulse and the reflection light pulse and the control pattern when k=2, M=4, and N=11 in waveform form. Fig.10 It is expressed in table form. Fig. 9 A diagram of a control pattern represented in waveform form. Fig.11 This is a diagram showing control patterns in a table format when k=2, M=4, and N=12. Fig.12 This is a diagram showing control patterns when k=2, M=5, and N=31 in a table format. Fig.13 This is a diagram showing, in a table format, a control pattern when k=3, M=4, and N=7. Fig.14 This is a diagram showing control patterns when k=3, M=6, and N=25 in a table format. Fig.15 This is a diagram showing control patterns when k=4, M=6, and N=31 in a table format.

[0101] By providing the above control pattern to the light receiving unit 5 , it is possible to reliably perform distance measurement based on the TOF method using the compressed sensing technology.

[0102] When k is an integer greater than 2 and the pulse width P is greater than k-1 times and less than k times the time T ((k-1)T<P≤kT), the control pattern may be repeated for each of the M frames. n A control pattern for indicating the accumulation of electric charge in the n-th period. In this case, the control pattern preferably further satisfies the following seventh condition.

[0103] The seventh condition is that the value of the element at the mth row and nth column is r n a m，n The matrix with M rows and N columns satisfies the same content as the sixth condition above. That is, the value of the element in the mth row and nth column is r n a m，n The matrix Φ with M rows and N columns r It is represented by the following formula (9).

[0104] [Formula 9]

[0105]

[0106] If the matrix Φ is constructed r N column vectors The continuous column vectors with less than k+1 columns in (Equation (10)) are set to Then these column vectors The sum of the sum column vector S rA It is expressed by the following formula (11). rA With each column vector The inner product of the column vector The column vector with the largest value is the column vector Any of .

[0107] [Formula 10]

[0108]

[0109] [Formula 11]

[0110]

[0111] Fig.16 , Fig.17 The number of charge accumulation times r when M=4 and N=8 is adjusted n FIG. 1 is an example of a control pattern.

[0112] Fig.16The waveform shows the adjustment of the charge accumulation times r n Figure 4 shows a control pattern of the control pattern. Fig.17 It is expressed in table form. Fig.16 The control pattern is shown in the waveform form. The numbers in the figure represent the number of charge accumulation times r n .

[0113] In this way, by making the number of accumulations different for each of the M frames according to the period of charge accumulation, it is possible to stably perform distance measurement independently of the distance to the object. That is, generally speaking, the longer the distance to the object, the smaller the intensity of the reflected light pulse reaching the light receiving unit 5, and the worse the SN. Therefore, the longer the distance to the object (that is, the longer the time from the light pulse output of the light source to the period of charge accumulation), the greater the number of charge accumulations r. n The larger the setting, the more stable the distance measurement can be performed regardless of the distance to the object.

[0114] Furthermore, if it is known in advance that there is an object with low reflectivity near a certain distance, the charge accumulation number r is set to the value corresponding to the distance. n By setting it to be larger, stable distance measurement can be performed even for objects with low reflectivity. In addition, the number of accumulation times (e.g., r n =1000), the signal amount can be adjusted according to the intensity of the reflected light pulse.

[0115] The light receiving part 5 may also be Figure 2 As shown in the structure of the light receiving unit 5, a plurality of charge storage units are included for one photodiode. In this case, the control unit 6 can simultaneously provide a plurality of frames that do not simultaneously instruct charge storage in the same period among the M frames of the control pattern to the light receiving unit 5. Thus, the time required for distance measurement can be shortened.

[0116] Fig.18 is a diagram showing an example of such a control pattern. In this example, Figure 2 In the structure of the light receiving unit 5 shown in the figure, the first frame in the control pattern can be supplied to the switch SW1, and the second frame can be supplied to the switch SW2 at the same time. Also, the third frame in the control pattern can be supplied to the switch SW1, and the fourth frame can be supplied to the switch SW2 at the same time. In addition, during the period when charge accumulation is not performed in all of the multiple frames supplied to the light receiving unit 5 at the same time, the charge generated in the photodiode PD is discarded to the charge discarding unit.

[0117] In addition, the light receiving unit 5 may also include multiple groups of photodiodes and charge accumulation units. In this case, the control unit 6 can provide multiple frames of the M frames of the control pattern to the light receiving unit 5 at the same time. The multiple frames provided to the light receiving unit 5 at the same time can also indicate charge accumulation at the same time during the same period. That is, a certain frame is provided to a certain group of photodiodes and charge accumulation units, and at the same time, other arbitrary frames are provided to other groups. Thus, the time required for distance measurement can also be shortened.

[0118] Next, the processing content of the processing unit 7 will be described. Let the power of the reflected light reaching the light receiving unit 5 in the nth period among N periods after the light pulse output from the light source be x n , will be x1~x N The column vector whose elements are x is represented by the following equation (12). The amount of charge accumulated in the charge accumulation section of the light receiving section 5 in the m-th frame among the M frames of the control pattern is represented by y. m , will be y1~y M The column vector of the elements is assumed to be y (the following formula (13)).

[0119] [Formula 12]

[0120]

[0121] [Formula 13]

[0122]

[0123] There is a relationship represented by the following equation (14) between the M-row, N-column matrix Φ (the above equation (1)) representing the control pattern, the column vector x (equation (12)), and the column vector y (equation (13)).

[0124] [Formula 14]

[0125] y=Φx……(14)

[0126] If M = N and there exists an inverse matrix Φ of the matrix Φ -1 , then x can be analytically determined by the following equation (15). However, when M<N, x cannot be analytically determined.

[0127] [Formula 15]

[0128] x=Φ -1 y……(15)

[0129] Therefore, by solving the optimization problem represented by the following formula (16) using an iterative method, x can be obtained. The second term of this formula is the L1 norm (the sum of the absolute values ​​of each element). By solving this optimization problem, x that satisfies the above conditions can be obtained with a number of measurements (the number of frames M) that is less than the number of unknowns (the number of periods N). However, the amount of calculation for this optimization problem is not fixed and the calculation time is long.

[0130] [Formula 16]

[0131]

[0132] Preferably, x is obtained by applying the Orthogonal Matching Pursuit (OMP) algorithm to solve the optimization problem represented by the following equation (17). The second term of the equation is the L0 norm (the number of elements with non-zero values). This can stabilize and shorten the calculation time.

[0133] [Formula 17]

[0134]

[0135] The method of solving the L0 optimization problem (Expression (17)) by applying the OMP algorithm is as follows. First, for the N column vectors constituting the matrix Φ The following formula (18) is used to calculate the column vector with the largest calculated value. The numerator of this formula represents the column vector y and the column vector The inner product of size.

[0136] [Formula 18]

[0137]

[0138] Generate a column vector consisting only of the The matrix Φ of M rows and N columns is composed (that is, the values ​​of all elements in the columns other than the n1th column are 0) S Then, using this matrix Φ S , calculate the column vector x representing the least square solution of x by the following formula (19) S , the column vector r representing the residual is calculated by the following formula (20).

[0139] [Formula 19]

[0140] x S =(Φ S T Φ S ) -1 Φ s Ty……(19)

[0141] [Formula 20]

[0142] r=y-Φ S x S ……(20)

[0143] Secondly, for the N column vectors that constitute the matrix Φ The column vector that has been found in The adjacent column vectors of are calculated as follows (21), and the column vector with the largest calculated value is found. The numerator of this formula represents the column vector r and the column vector The inner product of size.

[0144] [Formula 21]

[0145]

[0146] Update only the column vector calculated so far The matrix Φ S Then, using this matrix Φ S , calculate the column vector x representing the least square solution of x by using the above formula (19) S , update the column vector r representing the residual by the above equation (20). Repeat this process.

[0147] This iterative process is performed until the size of the column vector r representing the residual becomes equal to or smaller than a certain value, or until the number of column vectors obtained by calculation using equation (18) or equation (21) reaches k+1.

[0148] Fig.19 This is a diagram for explaining an example of applying the OMP algorithm to solve the L0 optimization problem. This diagram shows the control pattern of k=2, M=4, N=7 in a table format, and also shows the values ​​of each element of the column vector x and the column vector y.

[0149] In this example, when the 7 column vectors constituting the 4-row 7-column matrix Φ When the calculation formula (18) is used, the column vector with the largest calculated value is By using equation (19), we can obtain the column vector x representing the least squares solution of x S =(0,0,8.3,0,0,0,0) T , through formula (20), we can get the column vector r = (0, -2.3, 1.7, 0.7) representing the residual T .

[0150] Next, if the column vectors found so far are The two column vectors next to The calculation formula (21) of each of the above formulas, then the column vector with the largest calculated value is By using equation (19), we can obtain the column vector x representing the least squares solution of x S =(0,3.5,6,0,0,0,0) T , we can get the column vector r = (0, 0, 0.5, -0.5) representing the residual by equation (20) T .

[0151] Next, if the column vectors found so far are The two column vectors next to The calculation formula (21) of each of the above formulas, then the column vector with the largest calculated value is By using equation (19), we can obtain the column vector x representing the least squares solution of x S =(0,4,5,1,0,0,0) T , we can get the column vector r = (0,0,0,0) representing the residual by equation (20) T .

[0152] The column vector x represents the least squares of x obtained after 3 (= k + 1) repetitions. S This is consistent with the column vector x. In addition, the magnitude of the column vector r representing the residual at this time is zero.

[0153] By applying the OMP algorithm to solve the L0 optimization problem in this way, the calculation time can be stabilized and shortened, and thereby the computing power and power consumption required according to the required specifications can be minimized.

[0154] In addition, not only the reflected light pulse but also the background light enters the light receiving unit 5. In this case, the processing unit 7 preferably performs correction based on the intensity of the background light when calculating the distance to the object.

[0155] In order to reduce the influence of the background light, the signal value obtained when the reflected light pulse is measured can be corrected by hardware or software based on the amount of charge accumulated in the charge accumulation section or the charge discarding section during the period when only the background light is incident on the light receiving section 5 (a period when no light pulse is output from the light source before or after the reflected light pulse is measured, or a period when the reflected light pulse is not incident on the light receiving section 5 even when the reflected light pulse is measured). In addition, even if the matrix Φ is generated in consideration of the background light intensity, the signal value obtained when the reflected light pulse is measured can be corrected.

[0156] The distance measurement device and the distance measurement method are not limited to the above-described embodiments and configuration examples, and various modifications are possible.

[0157] The first mode of the above-mentioned embodiment is a distance measuring device, which comprises: (1) a light source that irradiates a light pulse with a pulse width P to an object; (2) a light receiving unit including: a photodiode that generates a charge by receiving the light pulse irradiated from the light source to the object and reflected by the object; and a charge storage unit that stores the charge generated by the photodiode; (3) a control unit that provides a control pattern consisting of M frames to the light receiving unit, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light pulse is output from the light source to the charge storage unit and store the charge; and (4) a processing unit that calculates the distance to the object by a compression sensing technology based on the amount of charge stored in the charge storage unit, wherein the distance measuring device is configured to measure the distance to the object by a time-of-flight method, the pulse width P being less than the certain time T, and the control unit provides the control pattern to the light receiving unit, the control pattern being a matrix with M rows and N columns, and the value a of the element in the mth row and nth column of the matrix with M rows and N columns being the same as the control pattern. m，n When the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, and the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1.

[0158] The distance measuring device of the second aspect may be configured such that, in the configuration of the first aspect, the control unit repeats r for each of the M frames. n A control pattern for instructing the accumulation of electric charges in the n-th period is provided to the light receiving unit.

[0159] In the distance measuring device of the third aspect, in the configuration of the second aspect, the control unit may provide the light receiving unit with a control pattern, wherein the value of the element constituting the mth row and the nth column is r n a m，n Among all combinations of consecutive column vectors with less than k+1 columns among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the sum of the column vectors with less than k+1 columns, that is, the inner product of the sum column vector and each of the N column vectors by the size of the column vector, is any one of the column vectors with less than k+1 columns.

[0160] A distance measuring device according to a fourth aspect of the above-mentioned embodiment comprises: (1) a light source for irradiating a light pulse having a pulse width P to an object; (2) a light receiving unit including: a photodiode for generating a charge by receiving the light pulse irradiated from the light source to the object and reflected by the object; and a charge accumulating unit for accumulating the charge generated by the photodiode; and (3) a control unit for providing the light receiving unit with a control pattern consisting of M frames, the M frames indicating whether to emit a light pulse from the light source to the object in each of N periods divided by a certain time T. The charge generated by the diode is transferred to the charge storage unit and the charge is stored; and (4) a processing unit, which calculates the distance to the object by a compression sensing technology based on the amount of charge stored by the charge storage unit, the distance measuring device is configured to measure the distance to the object by a time-of-flight method, the pulse width P is greater than k-1 times and less than k times (k is an integer greater than 2) a certain time T, and the control unit provides a control pattern to the light receiving unit, the control pattern is when the control pattern is represented by a matrix of M rows and N columns, the value a of the element of the mth row and nth column of the matrix of M rows and N columns m，n When the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1, the Hamming distance of all combinations of two column vectors among the N column vectors that are separated from each other by only k+1 columns is greater than k, and for all combinations of consecutive k+1 columns among the N column vectors, in the matrix of M rows and k+1 columns composed of k+1 column vectors, the value of any element in the M row vectors is 1, and there are more than k+1 rows of row vectors that are different from each other.

[0161] In the distance measuring device of the fifth aspect, in the configuration of the fourth aspect, the control unit may repeat r for each of the M frames. n A control pattern for instructing the accumulation of electric charges in the n-th period is provided to the light receiving unit.

[0162] In the distance measuring device of the sixth aspect, in the configuration of the fifth aspect, the control unit may provide the light receiving unit with a control pattern, wherein the value of the element constituting the mth row and the nth column is r n a m，n Among all combinations of consecutive column vectors with k+1 columns or less among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the sum of the column vectors with k+1 columns or less, that is, the inner product of the sum column vector and each of the N column vectors by the size of the column vector, is any of the column vectors with k+1 columns or less.

[0163] In the distance measuring device of the seventh embodiment, it can be constructed that, in the structure of any one of the first to sixth embodiments, the light receiving unit as a photodiode and a charge storage unit includes one photodiode and a plurality of charge storage units, and the control unit simultaneously provides a plurality of frames among M frames of the control pattern that do not simultaneously indicate charge storage in the same period to the light receiving unit.

[0164] In the distance measuring device of the eighth embodiment, it can be constructed that, in the structure of any one of the first to sixth embodiments, the light receiving unit serves as a photodiode and the charge storage unit, includes multiple groups of photodiodes and charge storage units, and the control unit simultaneously provides multiple frames of M frames of the control pattern to the light receiving unit.

[0165] In the distance measurement device of the ninth aspect, in the configuration of any one of the first to eighth aspects, the processing unit may calculate the distance to the object using an orthogonal matching pursuit algorithm.

[0166] In the distance measurement device of a tenth aspect, in the configuration of any one of the first to ninth aspects, the processing unit may be configured to perform correction based on the intensity of background light when obtaining the distance to the object.

[0167] In the distance measuring device of the eleventh embodiment, it can be constructed as follows: in the structure of any one of the first to tenth embodiments, there is also an imaging optical system that inputs a light pulse irradiated from a light source to an object and reflected by the object to form an image, the light receiving unit has a plurality of pixels each including a photodiode and a charge storage unit arranged in two dimensions on a light receiving surface that receives the light pulse passing through the imaging optical system, and the processing unit obtains a distance image of the object by calculating the distance to the object for each of the plurality of pixels.

[0168] According to the first mode of the above-mentioned embodiment, a distance measurement method is a method for measuring the distance to the object by a time-of-flight method using: (1) a light source that irradiates a light pulse of a pulse width P to an object; and (2) a light receiving unit that includes: a photodiode that generates a charge by receiving the light pulse irradiated from the light source to the object and reflected by the object; and a charge storage unit that stores the charge generated by the photodiode, and comprising: (3) a control step of providing a control pattern consisting of M frames to the light receiving unit, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light pulse is output from the light source to the charge storage unit and store the charge; and (4) a processing step of calculating the distance to the object by a compression sensing technology based on the amount of charge stored in the charge storage unit, the pulse width P being less than the certain time T, and in the control step, providing the control pattern to the light receiving unit, the control pattern being a value a of an element of the mth row and nth column of the Mth row and Nth column matrix when the control pattern is represented by a matrix of M rows and N columns. m，n When the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, and the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1.

[0169] In the distance measurement method of the second aspect, in the configuration of the first aspect, in the control step, each of the M frames is repeated r. n A control pattern for instructing the accumulation of electric charges in the n-th period is provided to the light receiving unit.

[0170] In the distance measurement method of the third aspect, in the structure of the second aspect, in the control step, a control pattern is provided to the light receiving unit, and the control pattern is a value r of an element constituting the mth row and the nth column. n a m，n Among all combinations of consecutive column vectors with less than k+1 columns among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the sum of the column vectors with less than k+1 columns, that is, the inner product of the sum column vector and each of the N column vectors by the size of the column vector, is any one of the column vectors with less than k+1 columns.

[0171] According to the fourth mode of the distance measurement method of the above-mentioned embodiment, there is a method of measuring the distance to the object by a time-of-flight method using: (1) a light source, which irradiates a light pulse of pulse width P to an object; and (2) a light receiving unit, which includes: a photodiode, which receives a light pulse irradiated from the light source to the object and reflected by the object to generate an electric charge; and a charge accumulation unit, which accumulates the charge generated by the photodiode, and comprises: (3) a control step, which provides a control pattern consisting of M frames to the light receiving unit, and the M frames indicate whether to add or remove the light pulse from the light source to the object. The charge generated by the photodiode in each of N periods divided by a certain time T from the output moment is transferred to the charge storage unit and the charge is stored; and (4) a processing step, which is based on the amount of charge stored in the charge storage unit, and the distance to the object is calculated by a compression sensing technology, the pulse width P is greater than k-1 times and less than k times (k is an integer greater than 2) the certain time T, and in the control step, a control pattern is provided to the light receiving unit, and the control pattern is when the control pattern is represented by a matrix of M rows and N columns, the value a of the element of the mth row and nth column of the matrix of M rows and N columns is m，n When the value is set to 1 in the m-th frame among M frames and the n-th period among N periods to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 to indicate non-accumulation, the value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1, the Hamming distance of all combinations of two column vectors among the N column vectors that are separated from each other by only k+1 columns is greater than k, and for all combinations of consecutive k+1 columns among the N column vectors, in the matrix of M rows and k+1 columns composed of k+1 column vectors, the value of any element in the M row vectors is 1, and there are more than k+1 rows of row vectors that are different from each other.

[0172] In the distance measurement method of the fifth aspect, in the configuration of the fourth aspect, in the control step, each of the M frames is repeated r. n A control pattern for instructing the accumulation of electric charges in the n-th period is provided to the light receiving unit.

[0173] In the distance measuring method of the sixth aspect, in the structure of the fifth aspect, in the control step, a control pattern is provided to the light receiving unit, and the control pattern is a value r of an element constituting the mth row and the nth column. n a m，n Among all combinations of consecutive column vectors with less than k+1 columns among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the sum of the column vectors with less than k+1 columns, that is, the inner product of the sum column vector and each of the N column vectors by the size of the column vector, is any one of the column vectors with less than k+1 columns.

[0174] In the distance measurement method of the seventh aspect, it can be constructed that in the structure of any one of the first to sixth aspects, the light receiving unit as a photodiode and a charge storage unit includes one photodiode and a plurality of charge storage units, and in the control step, a plurality of frames of the M frames of the control pattern that do not simultaneously indicate charge accumulation in the same period are provided to the light receiving unit at the same time.

[0175] In the distance measurement method of the eighth mode, it can be constructed that, in the structure of any one of the first to sixth modes, the light receiving unit serves as a photodiode and the charge storage unit, including multiple groups of photodiodes and charge storage units, and in the control step, multiple frames of M frames of the control pattern are simultaneously provided to the light receiving unit.

[0176] In the distance measurement method of a ninth aspect, in the configuration of any one of the first to eighth aspects, the distance to the object may be obtained using an orthogonal matching pursuit algorithm in the processing step.

[0177] In the distance measurement method of a tenth aspect, in the configuration of any one of the first to ninth aspects, correction based on the intensity of background light may be performed when the distance to the object is obtained in the processing step.

[0178] In the distance measurement method of the eleventh aspect, it can be constructed as follows: in the structure of any one of the first to tenth aspects, an imaging optical system that uses input of a light pulse irradiated from a light source to an object and reflected by the object to form an image is also used, and the light receiving unit has a plurality of pixels each including a photodiode and a charge storage unit arranged in two dimensions on a light receiving surface that receives the light pulse passing through the imaging optical system, and in a processing step, the distance to the object is obtained by calculating the distance to the object for each of the plurality of pixels.

[0179] Industrial Applicability

[0180] The embodiments can be used as an apparatus and method for reliably performing distance measurement based on the TOF method using a compression sensing technology.

[0181] The distance measuring device or distance measuring method of the above-mentioned embodiment can be used in application fields such as 3D face recognition, AR, vehicle-mounted use, surveillance cameras, and picking robots. Among them, the distance measuring device is installed in a portable terminal device for 3D face recognition and AR technology, and in vehicle-mounted use, the distance measuring device can be used as a distance measurement to an object that is indispensable for autonomous driving technology.

[0182] Description of Reference Numerals

[0183] 1 ... distance measuring device, 2 ... light source, 3 ... irradiation optical system, 4 ... focusing optical system, 5 ... light receiving unit, 6 ... control unit, 7 ... processing unit.

Claims

1. A distance measuring device, wherein: have: A light source that irradiates a light pulse with a pulse width P toward an object; a light receiving unit including: a photodiode that receives a light pulse irradiated from the light source to the object and reflected by the object to generate a charge; and a charge storage unit that stores the charge generated by the photodiode; a control unit that provides the light receiving unit with a control pattern consisting of M frames, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light source outputs the light pulse to the charge accumulation unit and accumulate the charge; and a processing unit that obtains the distance to the object by a compression sensing technique based on the amount of charge accumulated by the charge accumulation unit, The distance measuring device is configured to measure the distance to the object by a time-of-flight method. The pulse width P is less than the certain time T, The control unit provides the control pattern to the light receiving unit, The control pattern is represented by a matrix of M rows and N columns, and the value a of the element in the mth row and nth column of the matrix of M rows and N columns is m，n When the value is set to 1 in the nth period among the N periods in the mth frame among the M frames to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 in the nth period among the N periods to indicate non-accumulation, The value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, and the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1.

2. The distance measuring device according to claim 1, wherein: The control unit repeats each of the M frames r n The control pattern indicating the accumulation of electric charges in the n-th period is provided to the light receiving unit.

3. The distance measuring device according to claim 2, wherein: The control unit provides the control pattern to the light receiving unit, wherein the control pattern is a value r of an element constituting the mth row and the nth column. n a m，n Among all combinations of consecutive column vectors below k+1 columns among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the inner product of the sum column vector which is the sum of the column vectors below k+1 columns and each of the N column vectors by the size of the column vector is any of the column vectors below k+1 columns.

4. A distance measuring device, wherein: have: A light source that irradiates a light pulse with a pulse width P toward an object; a light receiving unit including: a photodiode that receives a light pulse irradiated from the light source to the object and reflected by the object to generate a charge; and a charge storage unit that stores the charge generated by the photodiode; a control unit that provides the light receiving unit with a control pattern consisting of M frames, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light source outputs the light pulse to the charge accumulation unit and accumulate the charge; and a processing unit that obtains the distance to the object by a compression sensing technique based on the amount of charge accumulated by the charge accumulation unit, The distance measuring device is configured to measure the distance to the object by a time-of-flight method. The pulse width P is greater than k-1 times and less than k times the certain time T, wherein k is an integer greater than 2, The control unit provides the control pattern to the light receiving unit, The control pattern is represented by a matrix of M rows and N columns, and the value a of the element in the mth row and nth column of the matrix of M rows and N columns is m，n When the value is set to 1 in the nth period among the N periods in the mth frame among the M frames to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 in the nth period among the N periods to indicate non-accumulation, The value of any element among all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1, the Hamming distance of all combinations of two column vectors among the N column vectors that are only separated from each other by k+1 columns is greater than k, and for all combinations of consecutive k+1 column vectors among the N column vectors, in the matrix of M rows and k+1 columns constituted by the column vectors of the k+1 columns, the value of any element among the M row vectors is 1 and there are more than k+1 rows of row vectors that are different from each other.

5. The distance measuring device according to claim 4, wherein: The control unit repeats each of the M frames r n The control pattern indicating the accumulation of electric charges in the n-th period is provided to the light receiving unit.

6. The distance measuring device according to claim 5, wherein: The control unit provides the control pattern to the light receiving unit, wherein the control pattern is a value r of an element constituting the mth row and the nth column. n a m，n Among all combinations of consecutive column vectors below k+1 columns among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the inner product of the sum column vector which is the sum of the column vectors below k+1 columns and each of the N column vectors by the size of the column vector is any of the column vectors below k+1 columns.

7. The distance measuring device according to any one of claims 1 to 6, wherein: The light receiving unit includes one photodiode and a plurality of charge storage units as the photodiode and the charge storage unit. The control unit simultaneously supplies a plurality of frames of the M frames of the control pattern that do not simultaneously instruct charge accumulation during the same period to the light receiving unit.

8. The distance measuring device according to any one of claims 1 to 6, wherein: The light receiving unit includes a plurality of sets of photodiodes and charge storage units as the photodiodes and the charge storage units, The control unit simultaneously provides a plurality of frames among the M frames of the control pattern to the light receiving unit.

9. The distance measuring device according to any one of claims 1 to 8, wherein: The processing unit obtains the distance to the object using an orthogonal matching tracking algorithm.

10. The distance measuring device according to any one of claims 1 to 9, wherein: The processing unit performs correction based on the intensity of background light when calculating the distance to the object.

11. The distance measuring device according to any one of claims 1 to 10, wherein: The optical system further comprises an imaging optical system which inputs a light pulse emitted from the light source to the object and reflected by the object to form an image. The light receiving unit has a plurality of pixels each including the photodiode and the charge storage unit arranged two-dimensionally on a light receiving surface that receives a light pulse passing through the imaging optical system. The processing unit acquires a distance image of the object by obtaining a distance to the object for each of the plurality of pixels.

12. A distance measurement method, wherein: It is a method that utilizes: a light source that irradiates a light pulse with a pulse width P toward an object; as well as a light receiving unit including: a photodiode that receives a light pulse irradiated from the light source to the object and reflected by the object to generate a charge; and a charge storage unit that stores the charge generated by the photodiode, and measuring the distance to the object by a time-of-flight method, have: a control step of providing a control pattern consisting of M frames to the light receiving unit, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light pulse of the light source is output to the charge accumulation unit and accumulate the charge; and a processing step of obtaining the distance to the object by a compression sensing technique based on the amount of charge accumulated by the charge accumulation unit, The pulse width P is less than the certain time T, In the control step, the control pattern is supplied to the light receiving unit. The control pattern is represented by a matrix of M rows and N columns, and the value a of the element in the mth row and nth column of the matrix of M rows and N columns is m，n When the value is set to 1 in the nth period among the N periods in the mth frame among the M frames to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 in the nth period among the N periods to indicate non-accumulation, The value of any element in all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, and the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1.

13. The distance measurement method according to claim 12, wherein: In the control step, each of the M frames is repeated r n The control pattern indicating the accumulation of electric charges in the n-th period is provided to the light receiving unit.

14. The distance measurement method according to claim 13, wherein: In the control step, the control pattern is provided to the light receiving unit, wherein the value of the element constituting the mth row and the nth column is r n a m，n Among all combinations of consecutive column vectors below k+1 columns among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the inner product of the sum column vector which is the sum of the column vectors below k+1 columns and each of the N column vectors by the size of the column vector is any of the column vectors below k+1 columns.

15. A distance measurement method, wherein: It is a method that utilizes: a light source that irradiates a light pulse with a pulse width P toward an object; as well as a light receiving unit including: a photodiode that receives a light pulse irradiated from the light source to the object and reflected by the object to generate a charge; and a charge storage unit that stores the charge generated by the photodiode, and measuring the distance to the object by a time-of-flight method, have: a control step of providing a control pattern consisting of M frames to the light receiving unit, the M frames indicating whether to transfer the charge generated by the photodiode in each of N periods divided by a certain time T from the time when the light pulse of the light source is output to the charge accumulation unit and accumulate the charge; and a processing step of obtaining the distance to the object by a compression sensing technique based on the amount of charge accumulated by the charge accumulation unit, The pulse width P is greater than k-1 times and less than k times the certain time T, wherein k is an integer greater than 2, In the control step, the control pattern is provided to the light receiving unit, The control pattern is represented by a matrix of M rows and N columns, and the value a of the element in the mth row and nth column of the matrix of M rows and N columns is m，n When the value is set to 1 in the nth period among the N periods in the mth frame among the M frames to indicate the accumulation of charges in the charge accumulation unit, and is set to 0 in the nth period among the N periods to indicate non-accumulation, The value of any element among all of the N column vectors constituting the matrix of M rows and N columns is 1, all of the N column vectors are different from each other, the Hamming distance of all combinations of two adjacent column vectors among the N column vectors is 1, the Hamming distance of all combinations of two column vectors among the N column vectors that are only separated from each other by k+1 columns is greater than k, and for all combinations of consecutive k+1 column vectors among the N column vectors, in the matrix of M rows and k+1 columns constituted by the column vectors of the k+1 columns, the value of any element among the M row vectors is 1 and there are more than k+1 rows of row vectors that are different from each other.

16. The distance measurement method according to claim 15, wherein: In the control step, each of the M frames is repeated r n The control pattern indicating the accumulation of electric charges in the n-th period is provided to the light receiving unit.

17. The distance measurement method according to claim 16, wherein: In the control step, the control pattern is provided to the light receiving unit, wherein the value of the element constituting the mth row and the nth column is r n a m，n Among all combinations of consecutive column vectors below k+1 columns among the N column vectors of the matrix with M rows and N columns, the column vector with the largest value obtained by dividing the inner product of the sum column vector which is the sum of the column vectors below k+1 columns and each of the N column vectors by the size of the column vector is any of the column vectors below k+1 columns.

18. The distance measurement method according to any one of claims 12 to 17, wherein: The light receiving unit includes one photodiode and a plurality of charge storage units as the photodiode and the charge storage unit. In the control step, a plurality of frames of the M frames of the control pattern that do not simultaneously instruct charge accumulation during the same period are simultaneously provided to the light receiving unit.

19. The distance measurement method according to any one of claims 12 to 17, wherein: The light receiving unit includes a plurality of sets of photodiodes and charge storage units as the photodiodes and the charge storage units, In the control step, a plurality of frames among the M frames of the control pattern are simultaneously provided to the light receiving section.

20. The distance measurement method according to any one of claims 12 to 19, wherein: In the processing step, the distance to the object is obtained using an orthogonal matching tracking algorithm.

21. The distance measurement method according to any one of claims 12 to 20, wherein: In the processing step, when obtaining the distance to the object, correction is performed based on the intensity of the background light.

22. The distance measurement method according to any one of claims 12 to 21, wherein: further using an imaging optical system that forms an image by inputting a light pulse emitted from the light source to the object and reflected by the object, The light receiving unit has a plurality of pixels each including the photodiode and the charge storage unit arranged two-dimensionally on a light receiving surface that receives a light pulse passing through the imaging optical system. In the processing step, a distance image of the object is acquired by obtaining the distance to the object for each of the plurality of pixels.

Images