DISTANCE MEASURING DEVICE

Abstract

A distance measuring device 1 comprises a light source 2, a light receiving unit 5, a control unit 6, and a processing unit 7. The light receiving unit 5 contains a photodiode and a charge accumulation region for accumulating charges generated in the photodiode. The control unit 6 applies a control pattern to the light receiving unit 5, which contains M frames that specify whether or not the charges in the charge accumulation region should be accumulated in each of N periods divided by a predetermined time T from a light pulse output time of the light source 2.If the control pattern is represented by a matrix of M rows and N columns, and a value of an element in an m-th row and n-th column of the matrix is ​​set to 1 if charge accumulation in an n-th period in an m-th frame is indicated, and set to 0 if non-accumulation is indicated, then the conditions are that a Hamming distance of 1 is satisfied for all combinations of two adjacent column vectors from the N column vectors forming the matrix, and so on. In this way, a device is realized that is capable of performing a reliable distance measurement using a time-of-flight (TOF) method with a compressive sensor technique.

Description

Technical area

[0001] The present disclosure relates to a device and a method for measuring a distance to an object using a time of flight method. State of the art

[0002] A distance measurement method using a time-of-flight (TOF) method measures a distance to an object by determining the time until a light pulse emitted from a light source is reflected by the object and returns to a light receiving unit. As methods for measuring distance using the TOF method, a two-phase method and a phase-shift method are known, and a method using a compression scanning technique is also known (Patent Document 1, Non-Patent Document 1). In each of the above methods, a distance measuring device includes the light source for irradiating the object with the light pulse and the light receiving unit including a photodiode and a charge storage region.

[0003] In the distance measurement technique using the two-phase method, the object is irradiated with the light pulse emitted by the light source with a pulse width T. Of the charges generated in the photodiode that receives the light pulse reflected from the object, the charges generated in a first period of the same time T as the pulse width are accumulated in one charge accumulation area, and the charges generated in a subsequent second period of time T are accumulated in another charge accumulation area. Furthermore, based on a ratio of the amounts of charges accumulated by the above-mentioned two charge accumulation areas, the time from a light pulse output time by the light source to a light pulse reception time by the photodiode is calculated to determine the distance to the object.

[0004] In distance measurement technology using the 2-phase method, the pulse width of the light pulse and the time of the charge accumulation period must be increased in order to increase the measurable distance, so that the measurable distance and the distance resolution are in a tension relationship.

[0005] In distance measurement using the phase-shift method, the object is irradiated with a light pulse of pulse width T emitted by the light source. The charges generated in the photodiode that receives the light pulse reflected from the object are accumulated in a charge accumulation region during a first period of time T equal to the pulse width. Next, the charges are accumulated in the same manner during a second period of time T following the first period. After that, the charges are accumulated in a similar manner during an nth period of time T following an (n-1)th period.

[0006] As described above, the charge accumulation period is shifted by time T, and the charges generated in the photodiode are accumulated in each of a plurality of periods divided by time T. Further, based on the amount of charges accumulated in each of the plurality of periods, the time from the light pulse output by the light source to the light pulse reception by the photodiode is calculated to determine the distance to the object.

[0007] In distance measurement technology that uses the phase-shift method, the measurable distance can be increased without decreasing the distance resolution by increasing the number of periods divided by time T. However, as the number of periods divided by time T increases, the number of measurements also increases.

[0008] The distance measurement technique using compressive sensing is based on the fact that reflected light intensity is sparse as a function of time because the reflected light pulse appears in a limited period after the light pulse is emitted from the light source, and the reflected light does not exist in other time periods. That is, after the light pulse is emitted from the light source, the charges generated in the photodiode accumulate in the charge accumulation region for one or more periods according to a random frame pattern. Furthermore, the distance to the object is determined using the compressive sensing technique based on the amount of charges accumulated in the charge accumulation region for each of several different image patterns.

[0009] Compared to the two-phase method, distance measurement using compressive sensing can increase the measurable distance without reducing the distance resolution. Furthermore, compared to the phase-shift method, distance measurement using compressive sensing can measure the distance to an object with fewer measurements. The distance measurement method using compressive sensing can be considered a high-speed or high-performance version of the distance measurement method using the phase-shift method. Citation listPatent literature

[0010] Patent Document 1: International Publication No. WO 2016 / 133053 A Non-patent literature Non-Patent Document 1: Keiichiro Kagawa, et al., “A Dual-Mode 303-Megaframes-per-Second Charge-Domain Time-Compressive Computational CMOS Image Sensor,” Sensors, 22(5), 1953, pp.1-16, 2022 Non-Patent Document 2: Joel A. Tropp, et al., “Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit,” IEEE TRANSACTIONS ON INFORMATION THEORY, Vol.53, No.12, pp.4655-4666, 2007 Summary of the inventionTechnical problem

[0011] While investigating distance measurement technology using the compression scanning technique, the inventors found that the above technique has the following problems. Namely, the distance to the object cannot be determined based on a pattern indicating a period of time during which the charges generated in the photodiode are accumulated in the charge storage part. Furthermore, it may not be easy to find a pattern that can be used to determine the distance to the object.

[0012] An object of an embodiment is to provide a device and a method capable of performing a reliable distance measurement by a TOF method using a compression sensing technique. Solution to the problem

[0013] One embodiment is a distance measuring device. The distance measuring device includes (1) a light source for irradiating an object with a light pulse having a pulse width P; (2) a light receiving unit including a photodiode for receiving the light pulse irradiated by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode.(3) a control unit for applying to the light receiving unit a control pattern including M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and (4) a processing unit for determining a distance to the object using a compression sampling technique based on an amount of charges accumulated by the charge accumulation region, and the device is configured to measure the distance to the object using a time-of-flight method, wherein the pulse width P is set to the predetermined time T or less, and in the control unit, when the control pattern is represented by a matrix of M rows and N columns and a value a; m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when accumulation of the charges in the charge accumulation region is indicated in an n-th period out of the N periods in an m-th frame out of the M frames, and is set to 0 when non-accumulation is indicated, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, and a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors, is applied to the light receiving unit.

[0014] One embodiment is a distance measuring device. The distance measuring device includes (1) a light source for irradiating an object with a light pulse having a pulse width P; (2) a light receiving unit including a photodiode for receiving the light pulse irradiated by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode.(3) a control unit for applying to the light receiving unit a control pattern including M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and (4) a processing unit for determining a distance to the object using a compression detection method based on an amount of charges accumulated by the charge accumulation region, and the device is configured to measure the distance to the object using a time-of-flight method, wherein the pulse width P is set to more than k-1 times and k times or less of the predetermined time T (k is an integer of 2 or more), and in the control unit, when the control pattern is represented by a matrix of M rows and N columns and a value a;m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when the accumulation of charges in the charge accumulation area in an n-th period of the N periods in an m-th frame of the M frames is specified, and is set to 0 when no accumulation is specified, wherein the control pattern in which a value of at least one element is 1 for all N column vectors forming the matrix of the M rows and the N columns, all N column vectors are different from each other, a Hamming distance is 1 for all combinations of two adjacent column vectors from the N column vectors, a Hamming distance is k or more for all combinations of two column vectors separated from each other by k+1 columns from the N column vectors, and for all combinations of consecutive k+1 column vectors from the N column vectors, in a matrix of M rows and k+1 columns,formed by the k+1 column vectors, there are k+1 or more row vectors that are different from each other and in which a value of at least one element is 1 out of M row vectors, is applied to the light receiving unit.

[0015] One embodiment is a distance measurement method. The distance measurement method is a method that includes (1) a light source for irradiating an object with a light pulse having a pulse width P;and (2) a light receiving unit having a photodiode for receiving the light pulse irradiated onto the object from the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode, the method being for measuring a distance to the object using a time-of-flight method, and the method including (3) a control step of applying to the light receiving unit a control pattern comprising M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output timing of the light source;and (4) a processing step of determining the distance to the object using a compression scanning technique based on an amount of charges accumulated by the charge accumulation region, and wherein the pulse width P is set to the predetermined time T or less, and in the control step, when the control pattern is represented by a matrix of M rows and N columns and a value a; m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when the accumulation of charges in the charge accumulation region is indicated in an n-th period of the N periods in an m-th frame of the M frames, and is set to 0 when no accumulation is indicated, wherein the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, and a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors, is applied to the light receiving unit.

[0016] One embodiment is a distance measurement method. The distance measurement method is a method that includes (1) a light source for irradiating an object with a light pulse having a pulse width P;and (2) a light receiving unit having a photodiode for receiving the light pulse irradiated onto the object by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode, the method being for measuring a distance to the object using a time-of-flight method, and the method comprising (3) a control step of applying to the light receiving unit a control pattern including M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output timing of the light source;and (4) a processing step of determining the distance to the object using a compression scanning technique based on an amount of charges accumulated by the charge accumulation region, and wherein the pulse width P is set to more than k-1 times and k times or less of the predetermined time T (k is an integer of 2 or more), and in the control step, when the control pattern is represented by a matrix of M rows and N columns and a value a; m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when an accumulation of charges in the charge accumulation area is indicated in an n-th period of the N periods in an m-th frame of the M frames, and is set to 0 when no accumulation is indicated, the control pattern in which a value of at least one element is 1 for all N column vectors forming the matrix of the M rows and the N columns, all N column vectors are different from each other, a Hamming distance is 1 for all combinations of two adjacent column vectors from the N column vectors, a Hamming distance is k or more for all combinations of two column vectors separated by k+1 columns from the N column vectors, and for all combinations of consecutive k+1 column vectors from the N column vectors, in a matrix of M rows and k+1 columns,formed by the k+1 column vectors, k+1 or more row vectors are present which differ from each other and in which a value of at least one element is 1 out of M row vectors, is applied to the light receiving unit., Advantageous effects of the invention

[0017] According to the distance measuring device and the distance measuring method of the embodiments, it is possible to perform reliable distance measurement by a TOF method using a pressure sensor technique. Short description of the drawings [ Fig. 1] Fig. 1 is a diagram illustrating the configuration of a distance measuring device 1. [ Fig. 2] Fig. 2 includes diagrams schematically illustrating a configuration of a light receiving unit 5 of the distance measuring device 1, and includes (a) a diagram illustrating a circuit configuration of the light receiving unit 5, and (b) a diagram schematically illustrating a state in which, when the switches SW1, SW3, and SW4 are in an OFF state and a switch SW2 is in an ON state, charges generated in a photodiode PD are transferred to a second charge accumulation region C2 through the switch SW2. [ Fig. 3] Fig. 3 is a diagram illustrating a control pattern of a comparative example. [ Fig. 4] Fig. Figure 4 is a diagram illustrating the control pattern in a waveform format for the case where M = 3 and N = 7. [ Fig. 5] Fig. Figure 5 is a diagram illustrating the control pattern in a table format for the case where M = 3 and N = 7. [ Fig. 6] Fig. Figure 6 is a diagram illustrating the control pattern in table format in the case where M = 3 and N = 7. [ Fig. 7] Fig. Figure 7 is a diagram illustrating the control pattern in table format in the case where M = 3 and N = 7. [ Fig. 8] Fig. Figure 8 is a diagram illustrating the control pattern in table format in the case where M = 4 and N = 15. [ Fig. 9] Fig. Figure 9 is a diagram illustrating the control pattern in waveform format for the case where k = 2, M = 4, and N = 11. [ Fig. 10] Fig. Figure 10 is a diagram illustrating the control pattern in table format in the case where k = 2, M = 4, and N = 11. [ Fig. 11] Fig. Figure 11 is a diagram illustrating the control pattern in the table format in the case where k = 2, M = 4, and N = 12. [ Fig. 12] Fig. Figure 12 is a diagram illustrating the control pattern in tabular format for the case where k = 2, M = 5, and N = 31. [ Fig. 13] Fig. Figure 13 is a diagram illustrating the control pattern in tabular format for the case where k = 3, M = 4, and N = 7. [ Fig. 14] Fig. Figure 14 is a diagram illustrating the control pattern in tabular format for the case where k = 3, M = 6, and N = 25. [ Fig. 15] Fig. Figure 15 is a diagram illustrating the control pattern in tabular format for the case where k = 4, M = 6, and N = 31. [ Fig. 16] Fig. Figure 16 is a diagram illustrating the control pattern in waveform format in the case where the number of charge accumulations r n is adjusted. [ Fig. 17] Fig. Figure 17 is a diagram illustrating the control pattern in table format for the case where the number of charge accumulations r n is adjusted. [ Fig. 18] Fig. 18 is a diagram illustrating an example of the control pattern in the case where a plurality of frames not simultaneously indicating charge accumulation in a same period can be simultaneously applied to the light receiving unit 5. [ Fig. 19] Fig. Figure 19 is a diagram illustrating an example of solving an L0 optimization problem by applying an OMP algorithm. Description of embodiments

[0018] Embodiments of a distance measuring device and a distance measuring method will be described in detail below with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. The present invention is not limited to these examples, and the claims, their equivalents, and any changes within the scope of the present invention are to be understood as falling within the scope of the present invention.

[0019] Fig. 1 is a diagram showing a configuration of a distance measuring device 1. The distance measuring device 1 is a device for measuring a distance to an object using a time-of-flight (TOF) method and includes a light source 2, an irradiation optical system 3, a focusing optical system 4, a light receiving unit 5, a control unit 6, and a processing unit 7. A distance measuring method is a method for performing a control step and a processing step using the light source 2, the irradiation optical system 3, the focusing optical system 4, and the light receiving unit 5.

[0020] Light source 2 emits a light pulse with which the object is to be irradiated. Light source 2 emits the light pulse with a specified pulse width P at a specified repetition frequency. Light source 2 can be any type as long as it can emit the light pulse, and can be, for example, a laser diode, an LED, or the like.

[0021] The irradiation optical system 3 is an optical system for irradiating the object with the light emitted from the light source 2. When the light output from the light source 2 is diverging light, the irradiation optical system 3 efficiently applies the light to the object.

[0022] The optical focusing system 4 inputs the light pulse (reflection light pulse) with which the object is irradiated from the light source 2 through the irradiation optics 3 and reflected by the object, and focuses the reflected light pulse.

[0023] The light receiving unit 5 receives the reflected light pulse arriving through the focusing optical system 4. The light receiving unit 5 includes a photodiode for receiving the reflected light pulse to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode.

[0024] The control unit 6 applies a control pattern to the light receiving unit 5 (the control step). The control pattern is a pattern indicating whether or not the charges generated in the photodiode of the light receiving unit 5 should be transferred to the charge accumulation region and accumulated in each of N periods divided by a predetermined time T from a light pulse output timing of the light source 2. The control pattern includes M frames and indicates the accumulation / non-accumulation of charges in each frame and each period. Each of M and N is an integer of 2 or more.

[0025] The processing unit 7 determines a distance to the object by applying a compression measurement method based on the amount of charges generated in the photodiode of the light receiving unit 5 and accumulated by the charge accumulation region (the processing step).

[0026] The control unit 6 and the processing unit 7 may be a computer. The control unit 6 and the processing unit 7 include an operation unit (e.g., a CPU, etc.) for performing a calculation process, etc., a storage unit (e.g., a hard disk drive, a RAM, a ROM, etc.) for storing the control pattern, the charge accumulation amount, etc., a display unit (e.g., a liquid crystal display, etc.) for displaying the control pattern, etc., an input unit (e.g., a keyboard, a mouse, etc.) for receiving an instruction to start a measurement, an input of a measurement condition, etc., and the like.

[0027] The control unit 6 and the processing unit 7 may be not only a computer, but also an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or the like.

[0028] Fig. Figure 2 contains diagrams schematically illustrating a configuration of the light receiving unit 5 of the distance measuring device 1. In this diagram, the light receiving unit 5 has a configuration with two charge accumulation regions. The light receiving unit 5 includes a photodiode PD for generating charges in response to light reception, and a first charge accumulation region C1 and a second charge accumulation region C2 for storing the charges.

[0029] The light receiving unit 5 further includes a switch SW1 for transferring the charges generated in the photodiode PD to the first charge accumulation region C1, a switch SW2 for transferring the charges generated in the photodiode PD to the second charge accumulation region C2, a switch SW3 for outputting the charges accumulated in the first charge accumulation region C1, and a switch SW4 for outputting the charges accumulated in the second charge accumulation region C2. The switches SW1 and SW2 are set to an ON or OFF state according to the values ​​VTX1 and VTX2 of the control pattern applied by the control unit 6.

[0030] (a) in Fig. 2 illustrates a circuit configuration of the light receiving unit 5. (b) in Fig. 2 schematically shows a state in which, when switches SW1, SW3, and SW4 are in the OFF state and switch SW2 is in the ON state, the charges generated in photodiode PD are transferred to the second charge accumulation region C2 through switch SW2. When the charge transfer to the second charge accumulation region C2 is completed, switch SW2 is set to the OFF state and switch SW4 is set to the ON state, and the charges stored in the second charge accumulation region C2 are output via switch SW4.

[0031] The number of charge accumulation regions can be set to one, two, or more. Each of the plurality of charge accumulation regions can be used as a charge removal region, or a charge removal region can be provided separately. The charge removal region is a region for accumulating the charges generated in the photodiode PD during a period in which charge accumulation is not indicated by the control pattern, and it is not necessary to output the above charges. Furthermore, the light receiving unit 5 includes a switch for initializing charge accumulation in each of the charge accumulation regions and the charge removal region.

[0032] The light receiving unit 5 may be an imaging element in which a plurality of pixels, each including the photodiode and the charge accumulation region, are arranged two-dimensionally on a light receiving surface. In this case, the focusing optical system 4 may be an imaging optical system for inputting and forming an image of the light pulse reflected from the object. The processing unit 7 may acquire a distance image of the object by determining the distance to the object for each of the plurality of pixels. The imaging element in which the plurality of pixels, each having the above-described configuration of the light receiving unit 5, are arranged two-dimensionally is sold as a product called a "distance surface image sensor" by Hamamatsu Photonics KK.

[0033] The distance measuring device and method according to the present embodiment are for measuring the distance to the object by the compression scanning technique using the light source 2 and the light receiving unit 5 as described above, and have a feature in the control pattern applied to the light receiving unit 5 by the control unit 6 and also have a feature in the distance calculation algorithm by the processing unit 7. Next, the control pattern applied to the light receiving unit 5 by the control unit 6 in the control step will be described, and then the content of the processing performed by the processing unit 7 in the processing step will be described.

[0034] Fig. 3 is a diagram illustrating the control pattern according to a comparative example. In this diagram, in order from the top, a waveform of the irradiation light pulse output from the light source, a waveform of the reflected light pulse reaching the light receiving unit, and patterns from the first to fourth frames in the control pattern indicating a period in which the charges generated in the photodiode in the light receiving unit are accumulated in the charge accumulation region are illustrated. The waveform of the irradiation light pulse and the waveform of the reflected light pulse practically have noise and distortion, but the waveforms are schematically represented as rectangles in this diagram (and in the following diagrams). Furthermore, it is assumed that one reflected light pulse reaches the light receiving unit for one irradiation light pulse.

[0035] A pulse width of each irradiation light pulse output from the light source and the reflected light pulse reaching the light receiving unit is set to P. A time of each of a plurality of divided periods after the light pulse output timing of the light source is set to T. In this diagram, it is set to P = T. The control pattern is represented as a pattern in which a value is set to 1 when specifying accumulation of charges in each frame and period, and a value is set to 0 when specifying non-accumulation.

[0036] With respect to the light pulse time emitted by the light source, a reflected light pulse arriving at the light receiving unit has a time difference Δt, which depends on the distance to the object. By detecting the above time difference Δt, the distance to the object can be determined. In the case of division into eight periods according to the light pulse output time of the light source, eight images are required for the control pattern in the phase shift method. On the other hand, in the case where the compressed sampling technique is used, the control pattern can be Fig. 3, four images are shown.

[0037] As illustrated in the diagram, the reflected light pulse appears in a limited time period after the light pulse output time of the light source, and the reflected light does not exist in other time periods. Thus, the reflected light intensity as a function of time has a sparse property. Therefore, the time between the time of output of the irradiation light pulse and the time of arrival of the reflected light pulse can be determined using the compressed sampling technique, and the distance to the object can also be determined. Furthermore, the number of control patterns required in the case where the compressed sampling technique is used can be made smaller than that required in the case where the phase-shift method is used.

[0038] In the case where distance measurement using the TOF method is performed with the compression scanning technique, the distance to the object may not be determined depending on the pattern indicating the period of time for which the charges generated in the photodiode are accumulated in the charge storage part. Furthermore, it may not be easy to find the pattern that can determine the distance to the object. In the distance measuring device and method described below, the control unit 6 applies the control pattern that satisfies the predetermined condition to the light receiving unit 5, and thus the distance measurement using the TOF method can be reliably performed using the compression scanning technique.

[0039] To describe the condition to be satisfied by the control pattern, the control pattern is represented by a matrix Φ with M rows and N columns, as shown in the following formula (1). M is the number of frames contained in the control pattern. N is the number of periods divided by the light pulse output timing of the light source. [Formula 1] Φ=(a1,1⋯a 1,n⋯a1,N⋮⋱⋮am,1am,nam,N⋮⋱⋮aM,1⋯aM,n⋯aM,N)

[0040] A value a m,n of an element in an m-th row and an n-th column of the matrix Φ is set to 1 if the accumulation of charges in the charge accumulation region is indicated in an n-th period of the N periods in an m-th frame of the M frames, and is set to 0 if the non-accumulation is indicated. In the matrix Φ, a column vector ϕ n the n-th column is represented by the following formula (2). The matrix Φ is represented by the N column vectors ϕ1 to ϕN formed as represented in the following formula (3). [Formula 2] ϕn=(a1,n⋮am,n⋮aM,n) [Formula 3] Φ=(ϕ1⋯ϕn⋯ϕN)

[0041] The condition that the control pattern must satisfy depends on the relationship between the pulse width P of the light pulse and the time T of each period. In the case where the pulse width P is set to time T or less (P≤T), the control pattern must satisfy the following first to third conditions.

[0042] The first condition is that the value of at least one element for all N column vectors ϕ1 to ϕ Nis equal to 1. This means that the matrix Φ does not contain a column vector in which the values ​​of all elements are 0. This condition is necessary to obtain information for all N periods. If the column vector in which the values ​​of all elements are 0 is contained in the matrix Φ, no information can be obtained for the period corresponding to the column vector.

[0043] The second condition is that all N column vectors ϕ1 to ϕ N are different from each other. This means that the matrix Φ does not contain the same column vector. This condition is necessary to determine the position of the reflected light pulse with the same pulse width P as the time T. If the same column vectors are contained in the matrix Φ, the position of the reflected light pulse with the same pulse width P as the time T cannot be determined.

[0044] The third condition is that a Hamming distance for all combinations of two adjacent column vectors from the N column vectors ϕ1 to ϕ N is equal to 1. The Hamming distance indicates the number of positions where the values ​​differ from each other when comparing the values ​​of the elements at the same position between the two column vectors. For example, between the two column vectors shown in the following formula (4), the number of positions where the values ​​differ from each other is 3, so the Hamming distance is 3. This condition is necessary to distinguish one position from the other in the case where the reflected light pulse extends over two periods. [Formula 4] (1101),(0110)

[0045] Fig. 4 to Fig. 8 are diagrams illustrating examples of the control pattern satisfying the first to third conditions when the pulse width P is set to time T or less (P≤T).

[0046] Fig. 4 is a diagram illustrating the control pattern in waveform format in the case where M = 3 and N = 7 with the waveform of the irradiation light pulse output from the light source. Fig. 5 is a diagram illustrating the control pattern in table format, which is shown in waveform format in Fig. 4 is illustrated. Fig. 6 and Fig. 7 are each a diagram illustrating the control pattern in table format for the case where M = 3 and N = 7. Fig. Figure 8 is a diagram illustrating the control pattern in table format for the case where M = 4 and N = 15. In the case where the pulse width P is set to time T or less (P≤T), a relationship of the following formula (5) exists between M and N. [Formula 5] N=2M−1

[0047] In the case where k is an integer of 2 or more and the pulse width P is set to more than k-1 times and k times or less of the time T ((k-1)T < P ≤ kT), the control pattern must satisfy the following fourth and fifth conditions in addition to the first to third conditions described above.

[0048] The fourth condition is that the Hamming distance is k or more for all combinations of two column vectors separated by k+1 columns from the N column vectors ϕ1 to ϕ Nseparated from each other. This means that the Hamming distance k or greater for all combinations of the column vector ϕ n and the column vector ϕ n+k+1 This condition is necessary to distinguish in which of the two periods, corresponding to the two column vectors separated by k+1 columns, the reflected light pulse is present.

[0049] The fifth condition is that for all combinations of k+1 consecutive column vectors from the N column vectors ϕ1 to ϕ N in a matrix of M rows and k+1 columns formed by the k+1 column vectors, there are k+1 or more row vectors which differ from each other and in which the value of at least one element is 1 out of M row vectors.

[0050] That is, if the determined consecutive k+1 column vectors on ϕ n to ϕ n+k, the matrix of M rows and k+1 columns defined by the above k+1 column vectors ϕ n to ϕ n+k formed by the following formula (6). Of the M row vectors included in this matrix, there are k+1 or more row vectors that are different from each other and in which the value of at least one element is 1. In the case where the pulse width P of the reflected light pulse is set to kT or less, the reflected light pulse extends for a maximum of k+1 periods, and at least k+1 pieces of information are required to obtain the accumulation charge amount of each of the k+1 periods, and therefore this condition is necessary. [Formula 6] (a1,n⋯a1,n+k⋮⋮am,n⋯am,n+k⋮⋮aM,n⋯aM,n+k)

[0051] In the case where k is an integer of 2 or more and the pulse width P is set to more than k-1 times and k times or less of the time T ((k-1)T < P ≤ kT), it is preferable that the control pattern satisfies the following sixth condition in addition to the first to fifth conditions described above.

[0052] The sixth condition is that for all combinations of consecutive k+1 or fewer column vectors from the N column vectors ϕ1 to ϕ N a column vector in which a value obtained by dividing an inner product of a sum column vector, which is a sum of the k+1 or fewer column vectors, and each column vector ϕ n by a size of the column vector ϕ n obtained is a maximum value, any of the k+1 or fewer column vectors.

[0053] That is, if the consecutive k+1 or fewer column vectors from the N column vectors ϕ1 to ϕN on ϕ n1 to ϕ n2 are set, the sum column vector S A , which is the sum of the above column vectors ϕ n1 to ϕ n2 is represented by the following formula (7). The calculation of the division of the inner product of the sum column vector S A and each column vector ϕ n by the size of the column vector ϕ n is represented by the following formula (8). The column vector with the highest calculated value is one of the column vectors ϕ n1 to ϕ n2 . [Formula 7] SA=∑n=n1n2ϕn=(∑n=n1n2a1,n⋮∑n=n1n2am,n⋮∑n=n1n2aM,n) [Formula 8] 〈ϕn,SA〉‖ϕn‖2

[0054] In the case where k = 2 or k = 3, the above sixth condition is automatically satisfied if the other conditions are met. Furthermore, the above sixth condition is required in the case where an orthogonal matching pursuit algorithm (Non-Patent Document 2) is used in the distance calculation by the processing unit 7 described later, and furthermore, it is not required in the case where another algorithm (for example, a brute force method or the like) is used.

[0055] Fig. 9 to Fig. 15 are diagrams illustrating examples of the control pattern satisfying the first to sixth conditions in the case where k is an integer of 2 or more and the pulse width P is set to more than k-1 times and k times or less of the time T ((k-1)T < P ≤ kT).

[0056] Fig. 9 is a diagram illustrating the control pattern in waveform format in the case where k = 2, M = 4, and N = 11, with the waveforms of the irradiation light pulse and the reflected light pulse. Fig. 10 is a diagram illustrating the control pattern in table format, which is shown in waveform format in Fig. 9 is illustrated. Fig. Figure 11 is a diagram illustrating the control pattern in tabular format for the case where k = 2, M = 4, and N = 12. Fig. Figure 12 is a diagram illustrating the control pattern in table format in the case where k = 2, M = 5, and N = 31. Fig. Figure 13 is a diagram illustrating the control pattern in table format for the case where k = 3, M = 4, and N = 7. Fig. Figure 14 is a diagram illustrating the control pattern in table format for the case where k = 3, M = 6, and N = 25. Fig. Figure 15 is a diagram illustrating the control pattern in table format for the case where k = 4, M = 6, and N = 31.

[0057] By applying the control pattern described above to the light receiving unit 5, it is possible to reliably perform the distance measurement according to the TOF method using the compression sensor technology.

[0058] In the case where k is an integer of 2 or more and the pulse width P is set to more than k-1 times and k times or less of the time T ((k-1)T < P ≤ kT), the control pattern may be a control pattern in which the display of the accumulation of charges in the n-th period r n times for each of the M frames. In this case, it is preferable that the control pattern also satisfies the following seventh condition.

[0059] The seventh condition is that the same content as the sixth condition above is satisfied for the matrix of M rows and N columns, in which the value of the element in the m-th row and the n-th column is set to r n a m,n is set. That is, the matrix Φ r of M rows and N columns, in which the value of the element in the m-th row and the n-th column is set to r n a m,n is represented by the following formula (9). [Formula 9] Φr=(r1a1,1⋯rna1,n⋯rNa1,N⋮⋱⋮r1,am,1rnam,nrNam,N⋮⋱⋮r1aM,1⋯rnaM,n⋯rNaM,N)

[0060] If the consecutive k+1 or fewer column vectors from the N column vectors ϕ r,1 to ϕ r,N (the following formula (10)), which the above matrix Φ r form, on ϕ r,n1 to ϕ r,n2 , the sum column vector S rA , which is the sum of the above column vectors Φ r,n1 to ϕ r,n2is represented by the following formula (11). The column vector with the maximum value obtained by dividing the inner product of the sum column vector S rA and each column vector ϕ r,n by the size of the column vector ϕ r,n is obtained, one of the column vectors Φ r,n1 to ϕ( r,n2 ).\ [Formula 10] ϕr,n=(rna1,n⋮rnam,n⋮rnaM,n) [Formula 11] SrA=∑n=n1n2ϕr,n=(∑n=n1n2rna1,n⋮∑n=n1n2rnam,n⋮∑n=n1n2rnaM,n)

[0061] Fig. 16 and Fig. 17 are diagrams illustrating an example of the control pattern in which the number of charge accumulations r n in the case where M = 4 and N = 8.

[0062] Fig. 16 is a diagram illustrating the control pattern in waveform format in the case where the number of charge accumulations r n is set. Fig. 17 is a diagram illustrating the control pattern in table format, which is shown in waveform format in Fig. 16. The number in the diagram indicates the number of charge storages r n to.

[0063] As described above, it is possible to perform distance measurement stably regardless of the distance to the object by varying the number of accumulations depending on the period of charge accumulation for each of the M frames. That is, the greater the distance to the object, the lower the intensity of the reflected light pulse reaching the light receiving unit 5, and the worse the SN. The greater the distance to the object (i.e., the longer the time between the light pulse output time of the light source and the period of charge accumulation), the greater the number of charge accumulations r. n, and in this case the distance measurement can be carried out stably regardless of the distance to the object.

[0064] Furthermore, in the case where it is known in advance that the object with a low reflectance exists near a certain distance, it is possible to stably perform the distance measurement even for the above-mentioned object with the low reflectance by determining the number of charge accumulations r n for the period corresponding to the above-mentioned distance is set larger. By uniformly increasing the number of accumulations (e.g. r n = 1000), regardless of the period of charge accumulation, the signal quantity can be adjusted according to the intensity of the reflected light pulse.

[0065] As with the Fig. In the configuration of the light receiving unit 5 illustrated in Figure 2, the light receiving unit 5 may include a plurality of charge accumulation regions for a photodiode. In this case, the control unit 6 may simultaneously apply to the light receiving unit 5 a plurality of frames that do not simultaneously indicate charge accumulation in the same period of the M frames of the control pattern. In this case, the time required for distance measurement can be shortened.

[0066] Fig. Figure 18 is a diagram illustrating an example of the control pattern in the above case. In this example, in the Fig. In the configuration of the light receiving unit 5 shown in Figure 2, the first frame of the control pattern may be applied to the switch SW1, and at the same time, the second frame may be applied to the switch SW2. Further, the third frame of the control pattern may be applied to the switch SW1, and at the same time, the fourth frame may be applied to the switch SW2. Moreover, in the period in which charge accumulation is not performed in all of the multiple frames simultaneously applied to the light receiving unit 5, the charges generated in the photodiode PD are removed to the charge removal region.

[0067] Furthermore, the light receiving unit 5 may include a plurality of sets of photodiodes and charge accumulation regions. In this case, the control unit 6 may simultaneously apply a plurality of frames from the M images of the control pattern to the light receiving unit 5. The plurality of frames simultaneously applied to the light receiving unit 5 can simultaneously indicate the charge accumulation in the same period. That is, a specific frame is applied to a specific set of photodiodes and charge accumulation regions, and at the same time, another arbitrary frame is applied to another set. In this case, too, the time required for distance measurement can be shortened.

[0068] Next, the processing content of the processing unit 7 will be described. A power of the reflected light reaching the light receiving unit 5 in the n-th period of N periods after the light pulse output timing of the light source is set to x n set, and a column vector with x1 to x N as elements is set to x (the following formula (12)). The amount of charges accumulated in the charge accumulation area of ​​the light receiving unit 5 in the m-th frame out of the M frames of the control pattern is set to y m set, and a column vector with y1 to y M as elements is set to y (the following formula (13)). [Formula 12] x=(x1⋮xn⋮xN) [Formula 13] y=(y1⋮ym⋮yM)

[0069] The matrix Φ (formula (1) above) with the M rows and the N columns representing the control pattern, the column vector x (formula (12)) and the column vector y (formula (13)) have a relationship represented by the following formula (14). [Formula 14] y=Φx

[0070] In the case where M = N is set and the inverse matrix Φ -1 of the matrix Φ exists, x can be determined analytically by the following formula (15). However, in the case where it is set to M < N, x cannot be determined analytically. [Formula 15] x=Φ−1y

[0071] In this case, x can be determined by solving an optimization problem represented by the following formula (16) using an iterative method. A second term of this formula is an L1 norm (sum of the absolute values ​​of the respective elements). By solving this optimization problem, it is possible to determine x that satisfies the above condition, where the number of measurements (the number of frames M) is smaller than the unknown number (the number of periods N). However, in this optimization problem, the amount of computation is indeterminate and the computation time is long. [Formula 16] minimize12‖Φx−y‖22+λ‖x‖1

[0072] Preferably, x is determined by solving an optimization problem represented by the following formula (17) using an Orthogonal Matching Pursuit (OMP) algorithm. A second term of this formula is an L0 norm (the number of non-zero elements). In this case, the computation time can be stabilized and shortened. [Formula 17] minimize12‖Φx−y‖22+λ‖x‖0

[0073] A procedure for solving the L0 optimization problem (formula (17)) by applying the OMP algorithm is as follows. First, the following formula (18) is applied for each of the N column vectors ϕ1 to ϕ N which form the matrix Φ, and the column vector ϕ n1 with the maximum calculated value is determined. In this formula, a numerator represents the inner product of the column vector y and the column vector ϕ n and a denominator represents the size of the column vector ϕ n represents. [Formula 18] 〈ϕn,y〉‖ϕn‖2

[0074] A matrix Φ S with M rows and N columns containing only the column vector ϕ n1 which was determined as described above (ie the values ​​of all elements of the columns except the n1-th column are zero). Furthermore, using the above matrix Φ S a column vector x S , which represents the least squares solution of x, is calculated using the following formula (19), and a column vector r, which represents the residual error, is calculated using the following formula (20). [Formula 19] xs=(ΦSTΦS)−1ΦSTy [Formula 20] r=y−ΦSxS

[0075] Then, for the column vector corresponding to the already determined column vector ϕ n1 is adjacent, from the N column vectors ϕ1 to ϕ N, which form the matrix Φ, the following formula (21) is calculated and the column vector ϕ n2 with the maximum calculated value. In this formula, the numerator represents the inner product of the column vector r and the column vector ϕ n and the denominator is the size of the column vector ϕ n . [Formula 21] 〈ϕn,r〉‖ϕn‖2

[0076] The matrix Φ S , which only contains the column vectors ϕ n1 and ϕ n2 which were determined as described above, is updated. Furthermore, using the above matrix Φ S the column vector x S , which represents the least squares solution of x, is calculated using the above formula (19), and the column vector r, which represents the residual error, is updated using the above formula (20). The above processing is performed repeatedly.

[0077] The above repeated processing is performed until the size of the column vector r representing the residual error reaches the predetermined value or less, or until the number of column vectors determined by the calculation of Formula (18) or Formula (21) becomes k+1.

[0078] Fig. Figure 19 is a diagram showing an example of solving the L0 optimization problem using the OMP algorithm. This diagram illustrates the control pattern in tabular format for the case where k = 2, M = 4, and N = 7, and also illustrates the values ​​of the respective elements of the column vector x and column vector y.

[0079] In this example, if formula (18) is calculated for each of the seven column vectors ϕ1 to ϕ7 that form the matrix Φ from the four rows and the seven columns, the column vector ϕ3 has the maximum calculated value. The column vector x S = (0, 0, 8.3, 0, 0, 0, 0) T, which is the least squares solution of x, is determined using formula (19), and the column vector r = (0, -2.3, 1.7, 0.7) T , which represents the residual error, is determined using formula (20).

[0080] If formula (21) is then calculated for each of the two column vectors ϕ2 and ϕ4 that are adjacent to the column vector ϕ3 determined as described above, the column vector ϕ2 has the highest calculated value. The column vector x S = (0, 3.5, 6, 0, 0, 0, 0) T , which is the least squares solution of x, is determined using formula (19), and the column vector r = (0, 0, 0.5, -0.5) T , which represents the residual error, is determined using formula (20).

[0081] If formula (21) is then calculated for each of the two column vectors ϕ1 and ϕ4, in addition to the column vectors ϕ2 and ϕ3 determined as described above, the column vector ϕ4 has the maximum calculated value. The column vector x S = (0, 4, 5, 1, 0, 0, 0) T , which is the least squares solution of x, is determined using formula (19), and the column vector r = (0, 0, 0, 0) T , which represents the residual error, is determined using formula (20).

[0082] The column vector x S , which represents the least-squares solution of x obtained after repeating the processing three times (= k+1 times), coincides with the column vector x. Furthermore, the size of the column vector r, which represents the residual error, is 0 at this time.

[0083] As described above, solving the L0 optimization problem using the OMP algorithm can stabilize and shorten the computation time. It can also minimize the operating power and power consumption required to meet the required specifications.

[0084] Furthermore, in addition to the reflected light pulse, the background light also falls on the light receiving unit 5. In this case, it is advantageous for the processing unit 7 to make a correction based on the background light intensity at the time of determining the distance to the object.

[0085] To reduce the influence of background light, the signal value acquired at the time of reflected light pulse measurement can be corrected using hardware or software based on the amount of charge accumulated in the charge accumulation region or the charge removal region during a period in which only background light is incident on the light receiving unit 5 (a period in which the light pulse is not output from the light source before or after the time of reflected light pulse measurement, or a period in which the reflected light pulse is not incident on the light receiving unit 5 even at the time of reflected light pulse measurement). Furthermore, the signal value acquired at the time of reflected light pulse measurement can also be corrected by creating the matrix Φ taking the background light intensity into account.

[0086] The distance measuring device and the distance measuring method are not limited to the embodiments and configuration examples described above, and various modifications are possible.

[0087] The distance measuring device of a first aspect according to the above embodiment includes (1) a light source for irradiating an object with a light pulse having a pulse width P; (2) a light receiving unit including a photodiode for receiving the light pulse irradiated onto the object from the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode;(3) a control unit for applying to the light receiving unit a control pattern including M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and (4) a processing unit for determining a distance to the object using a compression sampling technique based on an amount of charges accumulated by the charge accumulation region. The device is configured to measure the distance to the object using a time-of-flight method, wherein the pulse width P is set to the predetermined time T or less, and in the control unit, when the control pattern is represented by a matrix of M rows and N columns and a value a; m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when accumulation of the charges in the charge accumulation region in an n-th period out of the N periods in an m-th frame out of the M frames is specified, and is set to 0 when non-accumulation is specified, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, and a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors is applied to the light receiving unit.

[0088] In the distance measuring device of a second aspect, the control unit in the configuration of the first aspect may apply to the light receiving unit the control pattern in which an indication of the accumulation of charges in the n-th period r n times for each of the M frames.

[0089] In the distance measuring device of a third aspect, in the configuration of the second aspect, the control unit may apply to the light receiving unit the control pattern in which, for all combinations of consecutive k+1 or less column vectors out of the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to r n a m,n, a column vector in which a value obtained by dividing an inner product of a column vector, which is a sum of the k+1 or fewer column vectors, and each of the N column vectors by a size of the column vector is a maximum value, any of the k+1 or fewer column vectors.

[0090] The distance measuring device of a fourth aspect according to the above embodiment includes (1) a light source for irradiating an object with a light pulse having a pulse width P; (2) a light receiving unit including a photodiode for receiving the light pulse irradiated by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode;(3) a control unit for applying to the light receiving unit a control pattern including M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and (4) a processing unit for determining a distance to the object using a compression detection method based on an amount of charges accumulated by the charge accumulation region, and the device is configured to measure the distance to the object using a time of flight method, wherein the pulse width P is set to more than k-1 times and k times or less of the predetermined time T (k is an integer of 2 or more), and in the control unit, when the control pattern is represented by a matrix of M rows and N columns and a value a;m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when the accumulation of charges in the charge accumulation area in an n-th period of the N periods in an m-th frame of the M frames is specified, and is set to 0 when no accumulation is specified, wherein the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, a Hamming distance is 1 for all combinations of two adjacent column vectors from the N column vectors, a Hamming distance is k or more for all combinations of two column vectors separated from each other by k+1 columns from the N column vectors, and for all combinations of consecutive k+1 column vectors from the N column vectors, in a matrix of M rows and k+1 columns,formed by the k+1 column vectors, there are k+1 or more row vectors that are different from each other and in which a value of at least one element is 1 out of M row vectors, is applied to the light receiving unit.,

[0091] In the distance measuring device of a fifth aspect, the control unit in the configuration of the fourth aspect may apply to the light receiving unit the control pattern in which an indication of the accumulation of charges in the n-th period r n times for each of the M frames.

[0092] In the distance measuring device of a sixth aspect, in the configuration of the fifth aspect, the control unit may apply to the light receiving unit the control pattern in which, for all combinations of consecutive k+1 or less column vectors out of the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to r n a m,n , a column vector in which a value obtained by dividing an inner product of a sum column vector, which is a sum of the k+1 or fewer column vectors, and each of the N column vectors by a size of the column vector, is a maximum value, each of the k+1 or fewer column vectors.

[0093] In the distance measuring device of a seventh aspect, in the configuration of any one of the first to sixth aspects, the light receiving unit may include a photodiode and a plurality of charge accumulation regions as the photodiode and the charge accumulation region, and the control unit may simultaneously apply to the light receiving unit a plurality of frames that do not simultaneously indicate charge accumulation in a same period out of the M frames of the control pattern.

[0094] In the distance measuring device of an eighth aspect, in the configuration of any one of the first to sixth aspects, the light receiving unit may include a plurality of sets of photodiodes and charge accumulation regions as the photodiode and the charge accumulation region, and the control unit may simultaneously apply a plurality of frames from the M frames of the control pattern to the light receiving unit.

[0095] In the distance measuring device of a ninth aspect, the processing unit in the configuration of any one of the first to eighth aspects may determine the distance to the object using an orthogonal matching pursuit algorithm.

[0096] In the distance measuring device of a tenth aspect, the processing unit in the configuration of any one of the first to ninth aspects may perform a correction based on a background light intensity when determining the distance to the object.

[0097] In the distance measuring device of an eleventh aspect, the device in the configuration of any one of the first to tenth aspects may further include an imaging optical system for inputting and forming an image of the light pulse irradiated onto the object by the light source and reflected from the object, and in the light receiving unit, a plurality of pixels each including the photodiode and the charge accumulation region may be arranged two-dimensionally on a light receiving surface for receiving the light pulse passed through the imaging optical system, a plurality of pixels each including the photodiode and the charge accumulation region may be arranged two-dimensionally on a light receiving surface for receiving the light pulse passed through the imaging optical system,and the processing unit can acquire a distance image of the object by determining the distance to the object for each of the plurality of pixels.,

[0098] The distance measuring method of a first aspect according to the above embodiment is a method comprising (1) a light source for irradiating an object with a light pulse having a pulse width P;and (2) a light receiving unit having a photodiode for receiving the light pulse irradiated onto the object by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode, the method being for measuring a distance to the object using a time-of-flight method, and the method comprising (3) a control step of applying to the light receiving unit a control pattern comprising M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output timing of the light source;and (4) a processing step of determining the distance to the object using a compression scanning technique based on an amount of charges accumulated by the charge accumulation region, and wherein the pulse width P is set to the predetermined time T or less, and in the control step, when the control pattern is represented by a matrix of M rows and N columns and a value a; m,nof an element in an m-th row and an n-th column of the matrix of M rows and N columns is set to 1 when the accumulation of charges in the charge accumulation region in an n-th period out of the N periods in an m-th frame out of the M frames is specified, and is set to 0 when no accumulation is specified, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of M rows and N columns, all N column vectors are different from each other, and a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors is applied to the light receiving unit.

[0099] In the distance measuring method of a second aspect, in the configuration of the first aspect, in the control step, the control pattern in which an indication of the accumulation of charges in the n-th period r ntimes for each of the M frames, applied to the light receiving unit.

[0100] In the distance measuring method of a third aspect, in the configuration of the second aspect, in the control step, the control pattern in which for all combinations of consecutive k+1 or less column vectors from the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to r n a m,n is, a column vector in which a value obtained by dividing an inner product of a sum column vector, which is a sum of the k+1 or less column vectors, and each of the N column vectors by a size of the column vector is a maximum value, may be applied to the light receiving unit.

[0101] The distance measuring method of a fourth aspect according to the above embodiment is a method using (1) a light source for irradiating an object with a light pulse having a pulse width P;and (2) a light receiving unit having a photodiode for receiving the light pulse irradiated onto the object from the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode, the method being for measuring a distance to the object using a time-of-flight method, and the method including (3) a control step of applying to the light receiving unit a control pattern comprising M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output timing of the light source;and (4) a processing step of determining the distance to the object using a compression scanning technique based on an amount of charges accumulated by the charge accumulation region, and wherein the pulse width P is set to more than k-1 times and k times or less of the predetermined time T (k is an integer of 2 or more), and wherein in the control step, when the control pattern is represented by a matrix of M rows and N columns and a value a; m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when an accumulation of charges in the charge accumulation area is indicated in an n-th period of the N periods in an m-th frame of the M frames, and is set to 0 when no accumulation is indicated, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, a Hamming distance is 1 for all combinations of two adjacent column vectors from the N column vectors, a Hamming distance is k or more for all combinations of two column vectors separated from each other by k+1 columns from the N column vectors, and for all combinations of consecutive k+1 column vectors from the N column vectors, in a matrix of M rows and k+1 columns,formed by the k+1 column vectors, k+1 or more row vectors are present which differ from each other and in which a value of at least one element is 1 out of M row vectors, is applied to the light receiving unit.,

[0102] In the distance measuring method of a fifth aspect, in the configuration of the fourth aspect, in the control step, the control pattern in which an indication of the accumulation of charges in the n-th period r n times for each of the M frames is repeated, applied to the light receiving unit.

[0103] In the distance measuring method of a sixth aspect, in the configuration of the fifth aspect, in the control step, the control pattern in which for all combinations of consecutive k+1 or less column vectors from the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to rn a m,n is set, a column vector in which a value obtained by dividing an inner product of a sum column vector, which is a sum of the k+1 or less column vectors, and each of the N column vectors by a size of the column vector is a maximum value, may be applied to the light receiving unit.

[0104] In the distance measuring method of a seventh aspect, in the configuration of any one of the first to sixth aspects, the light receiving unit may include a photodiode and a plurality of charge accumulation regions as the photodiode and the charge accumulation region, and in the control step, a plurality of frames not simultaneously indicating charge accumulation in a same period out of the M frames of the control pattern may be simultaneously applied to the light receiving unit.

[0105] In the distance measuring method of an eighth aspect, the light receiving unit in the configuration of any one of the first to sixth aspects may include a plurality of sets of photodiodes and charge accumulation regions as the photodiode and the charge accumulation region, and in the control step, a plurality of frames out of the M frames of the control pattern may be applied to the light receiving unit at the same time.

[0106] In the distance measuring method of a ninth aspect, in the configuration of any one of the first to eighth aspects, in the processing step, the distance to the object may be determined using an orthogonal matching pursuit algorithm.

[0107] In the distance measuring method of a tenth aspect, in the configuration of any one of the first to ninth aspects, in the processing step, a correction based on a background light intensity may be performed when determining the distance to the object.

[0108] In the distance measuring method of an eleventh aspect, in the configuration of any one of the first to tenth aspects, the method may further use an imaging optical system for inputting and forming an image of the light pulse irradiated onto the object from the light source and reflected from the object, and in the light receiving unit, a plurality of pixels each including the photodiode and the charge accumulation region may be two-dimensionally arranged on a light receiving surface to receive the light pulse passing through the imaging optical system, and in the processing step, a distance image of the object may be acquired by determining the distance to the object for each of the plurality of pixels. Industrial applicability

[0109] The embodiments can be used as a device and method capable of performing reliable distance measurement by a TOF method using a compression sensing technique.

[0110] The distance measuring device or method of the above embodiment can be used in applications such as 3D facial recognition, AR, in-vehicle applications, surveillance cameras, and robotic sensing. In the above-mentioned applications, in the 3D facial recognition and AR technologies, the distance measuring device can be provided in a mobile terminal. Furthermore, in the in-vehicle application, the distance measuring device can be used for distance measurement with the object, which is necessary for autonomous driving technology. List of reference symbols

[0111] 1 - distance measuring device, 2 - light source, 3 - optical irradiation system, 4 - optical focusing system, 5 - light receiving unit, 6 - control unit, 7 - processing unit. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] WO 2016 / 133053 A

[0010] Cited non-patent literature

[0000] Keiichiro Kagawa, et al., “A Dual-Mode 303-Megaframes-per-Second Charge-Domain Time-Compressive Computational CMOS Image Sensor,” Sensors, 22(5), 1953, pp.1-16

[0010] Joel A. Tropp, et al., „Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit“, IEEE TRANSACTIONS ON INFORMATION THEORY, Bd.53, Nr.12, S.4655-4666,

[0010]

Claims

[1] Distance measuring device, comprising: a light source for irradiating an object with a light pulse having a pulse width P; a light receiving unit including a photodiode for receiving the light pulse irradiated onto the object by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode; a control unit for applying to the light receiving unit a control pattern including M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and a processing unit for determining a distance to the object using a compression sensing technique based on an amount of charges accumulated by the charge accumulation region, wherein the device is configured to measure the distance to the object using a time-of-flight method, the pulse width P is set to the specified time T or less, and in the control unit, if the control pattern is represented by a matrix of M rows and N columns and a value a m,n of an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 when an accumulation of charges in the charge accumulation area is indicated in an n-th period out of the N periods in an m-th frame out of the M frames, and is set to 0 when a non-accumulation is indicated, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, and a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors, is applied to the light receiving unit. [2] A distance measuring device according to claim 1, wherein the control unit applies to the light receiving unit the control pattern in which an indication of the accumulation of charges in the n-th period r n times for each of the M frames. [3] A distance measuring device according to claim 2, wherein the control unit applies to the light receiving unit the control pattern in which, for all combinations of consecutive k+1 or less column vectors out of the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to r n a m,n , a column vector in which a value obtained by dividing an inner product of a sum column vector, which is a sum of the k+1 or fewer column vectors, and each of the N column vectors by a size of the column vector, is a maximum value, each of the k+1 or fewer column vectors. [4] Distance measuring device, comprising: a light source for irradiating an object with a light pulse having a pulse width P; a light receiving unit having a photodiode for receiving the light pulse irradiated onto the object by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode; a control unit for applying to the light receiving unit a control pattern containing M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and a processing unit for determining a distance to the object using a compression sensing technique based on an amount of charges accumulated by the charge accumulation region, wherein the device is configured to measure the distance to the object using a time-of-flight method, the pulse width P is set to more than k-1 times and k times or less of the predetermined time T (k is an integer of 2 or more), and in the control unit, if the control pattern is represented by a matrix of M rows and N columns and a value a m,n of an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 if an accumulation of the charges in the charge accumulation area is specified in an n-th period of the N periods in an m-th frame of the M frames, and is set to 0 if no accumulation is specified, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors, a Hamming distance is k or more for all combinations of two column vectors separated by k+1 columns out of the N column vectors, and for all combinations of consecutive k+1 column vectors out of the N column vectors, in a matrix of M rows and k+1 columns formed by the k+1 column vectors, there are k+1 or more row vectors that are different from each other and in which a value of at least one element is 1 out of M row vectors, is applied to the light receiving unit. [5] A distance measuring device according to claim 4, wherein the control unit applies to the light receiving unit the control pattern in which an indication of the accumulation of charges in the n-th period r n times for each of the M frames. [6] A distance measuring device according to claim 5, wherein the control unit applies to the light receiving unit the control pattern in which, for all combinations of consecutive k+1 or less column vectors out of the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to r n a m,n , a column vector in which a value obtained by dividing an inner product of a sum column vector, which is a sum of the k+1 or fewer column vectors, and each of the N column vectors by a size of the column vector, is a maximum value, each of the k+1 or fewer column vectors. [7] A distance measuring device according to any one of claims 1 to 6, wherein the light receiving unit comprises a photodiode and a plurality of charge accumulation regions as the photodiode and the charge accumulation region, and the control unit simultaneously applies a plurality of frames to the light receiving unit which do not simultaneously indicate charge accumulation in a same period of the M frames of the control pattern. [8] A distance measuring device according to any one of claims 1 to 6, wherein the light receiving unit includes a plurality of sets of photodiodes and charge accumulation regions as the photodiode and charge accumulation region, and the control unit simultaneously applies a plurality of images from the M images of the control pattern to the light receiving unit. [9] Distance measuring device according to one of claims 1 to 8, wherein the processing unit determines the distance to the object using an orthogonal tracking algorithm. [10] Distance measuring device according to one of claims 1 to 9, wherein the processing unit performs a correction based on a background light intensity when determining the distance to the object. [11] Distance measuring device according to one of claims 1 to 10, further comprising an optical imaging system for inputting and generating an image of the light pulse irradiated by the light source and reflected by the object, wherein in the light receiving unit, a plurality of pixels each including the photodiode and the charge accumulation region are arranged two-dimensionally on a light receiving surface to receive the light pulse passing through the imaging optical system, and the processing unit acquires a distance image of the object by determining the distance to the object for each of the plurality of pixels. [12] Distance measurement methods, including: a light source for irradiating an object with a light pulse having a pulse width P; and a light receiving unit including a photodiode for receiving the light pulse irradiated by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode, the method for measuring the distance to an object using a time-of-flight method, comprising: a control step of applying to the light receiving unit a control pattern containing M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and a processing step of determining the distance to the object using a compression scanning technique based on an amount of charges accumulated by the charge accumulation region, wherein the pulse width P is set to the specified time T or less, and in the control step, if the control pattern is represented by a matrix of M rows and N columns and a value a m,n of an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 if an accumulation of the charges in the charge accumulation area is specified in an n-th period of the N periods in an m-th frame of the M frames, and is set to 0 if no accumulation is specified, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, and a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors, is applied to the light receiving unit. [13] A distance measuring method according to claim 12, wherein in the control step, the control pattern in which an indication of the accumulation of charges in the n-th period r n times for each of the M frames, is applied to the light receiving unit. [14] A distance measuring method according to claim 13, wherein in the control step, the control pattern in which for all combinations of consecutive k+1 or less column vectors out of the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to rn a m,n is set, a column vector in which a value obtained by dividing an inner product of a sum column vector, which is a sum of the k+1 or less column vectors, and each of the N column vectors by a size of the column vector is a maximum value, is applied to the light receiving unit. [15] Distance measurement methods, including: a light source for irradiating an object with a light pulse having a pulse width P; and a light receiving unit having a photodiode for receiving the light pulse irradiated by the light source and reflected by the object to generate charges, and a charge accumulation region for accumulating the charges generated in the photodiode, the method for measuring the distance to an object using a time-of-flight method, comprising: a control step of applying to the light receiving unit a control pattern containing M frames indicating whether or not the charges generated in the photodiode should be transferred to and accumulated in the charge accumulation region in each of N periods divided by a predetermined time T from a light pulse output time of the light source; and a processing step of determining the distance to the object using a compression scanning technique based on an amount of charges accumulated by the charge accumulation region, wherein the pulse width P is set to more than k-1 times and k times or less of the predetermined time T (k is an integer of 2 or more), and in the control step, if the control pattern is represented by a matrix of M rows and N columns and a value a m,nof an element in an m-th row and an n-th column of the matrix of the M rows and the N columns is set to 1 if accumulation of the charges in the charge accumulation area in an n-th period of the N periods in an m-th frame of the M frames is specified, and is set to 0 if no accumulation is specified, the control pattern in which a value of at least one element is 1 for all N column vectors constituting the matrix of the M rows and the N columns, all N column vectors are different from each other, a Hamming distance is 1 for all combinations of two adjacent column vectors out of the N column vectors, a Hamming distance is k or more for all combinations of two column vectors separated by k+1 columns out of the N column vectors, and for all combinations of consecutive k+1 column vectors out of the N column vectors, in a matrix of M rows and k+1 columns formed by the k+1 column vectors, there are k+1 or more row vectors that are different from each other and in which a value of at least one element out of M row vectors is 1, is applied to the light receiving unit. [16] A distance measuring method according to claim 15, wherein in the control step, the control pattern in which an indication of the accumulation of charges in the n-th period r n times for each of the M frames, is applied to the light receiving unit. [17] A distance measuring method according to claim 16, wherein in the control step, the control pattern in which for all combinations of consecutive k+1 or less column vectors out of the N column vectors constituting the matrix of the M rows and the N columns in which the value of the element in the m-th row and the n-th column is set to r n a m,n is set, a column vector in which a value obtained by dividing an inner product of a sum column vector which is a sum of the k+1 or less column vectors and each of the N column vectors by a size of the column vector is a maximum value is applied to the light receiving unit. [18] A distance measuring method according to any one of claims 12 to 17, wherein the light receiving unit comprises a photodiode and a plurality of charge accumulation regions as the photodiode and the charge accumulation region, and in the control step, a plurality of images not simultaneously indicating charge accumulation in a same period from the M images of the control pattern are simultaneously applied to the light receiving unit. [19] A distance measuring method according to any one of claims 12 to 17, wherein the light receiving unit comprises a plurality of sets of photodiodes and charge accumulation regions as the photodiode and charge accumulation region, and in the control step, a plurality of images from the M images of the control pattern are applied to the light receiving unit at the same time. [20] A distance measuring method according to any one of claims 12 to 19, wherein in the processing step the distance to the object is determined using an orthogonal matching pursuit algorithm. [21] A distance measuring method according to any one of claims 12 to 20, wherein in the processing step, in determining the distance to the object, a correction is performed based on a background light intensity. [22] A distance measuring method according to any one of claims 12 to 21, further comprising using an imaging optical system for inputting and generating an image of the light pulse irradiated by the light source onto the object and reflected by the object, wherein in the light receiving unit, a plurality of pixels each containing the photodiode and the charge accumulation region are arranged two-dimensionally on a light receiving surface to receive the light pulse passed through the imaging optical system, and in the processing step, a distance image of the object is acquired by determining the distance to the object for each of the plurality of pixels.

Images